From ea5a2b702d9affa15e72a6226817cfb55da8443b Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:42 +0000 Subject: [PATCH 01/16] =?UTF-8?q?feat(httpapi):=20wire-shape=20signal=20?= =?UTF-8?q?=E2=80=94=20wire=5Frevision,=20floor=20header,=20golden=20diges?= =?UTF-8?q?t,=20param=20tokens?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit S0 of the HTTP role-parity program: give a client a single place to learn whether this build's wire shape has moved under it, and give CI a gate that catches an undocumented move. - GET /v0/beads/context reports wire_revision (bumps only on a non-additive response-shape change) and min_client_wire_revision. An optional Bd-Wire-Revision request header lets a client declare the revision it was built for; checkWireRevision refuses a declared revision below the floor with 400 invalid_argument/reason=wire_revision_unsupported and a min_wire_revision field, on every route but /healthz (wireRevisionExempt) — including the identity handshake, where catching the mismatch is cheapest. Absent header is today's behavior; malformed value is invalid_value, never wire_revision_unsupported. The floor lives in Server.minClientWireRevision (defaulting to the MinClientWireRevision constant) so a test can raise it. - internal/httpapi/wireshape: a golden digest over every response's every content-type schema (including application/problem+json) across the whole spec, keyed by schema+member. TestWireShapeDigest fails closed when a member's JSON name or type changes without wire_revision bumping to match; regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`. - GET /v0/beads/issues' `sort` parameter now carries the issues.listSort capability token, the same convention `capabilities` entries use. TestNewParameterOnExistingOperationHasABehaviorToken requires every new parameter on an existing operation to carry one, checked against a write-once grandfather baseline (testdata/pretoken_parameters.json) of parameters that predate the rule; TestUntokenedParameterRuleFires proves the checker itself against a synthetic fixture. - Docs: engdocs/design/bd-serve-v0.md gets a "The wire-shape signal" section; CHANGELOG.md gets an Added entry. Live-mutated and reverted three RED demonstrations during development: retyping api_version in the spec failed TestWireShapeDigest; short-circuiting checkWireRevision's floor comparison failed TestWireRevisionBelowFloorRefusesEveryEnforcedRoute; adding an untokened query parameter to listIssues failed TestNewParameterOnExistingOperationHasABehaviorToken. All three now pass against the unmutated tree. Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 22 + engdocs/design/bd-serve-v0.md | 78 + internal/httpapi/apigen/types.gen.go | 39 +- internal/httpapi/handlers.go | 8 +- internal/httpapi/problem.go | 32 + internal/httpapi/routes.go | 38 +- internal/httpapi/server.go | 32 +- internal/httpapi/server_test.go | 2 +- internal/httpapi/spec/openapi.v0.yaml | 165 +- internal/httpapi/spec_parity_test.go | 127 +- .../httpapi/testdata/pretoken_parameters.json | 182 + internal/httpapi/wire_revision.go | 72 + internal/httpapi/wire_revision_test.go | 222 ++ .../httpapi/wireshape/cmd/gendigest/main.go | 49 + internal/httpapi/wireshape/doc.go | 21 + .../httpapi/wireshape/testdata/golden.json | 3050 +++++++++++++++++ internal/httpapi/wireshape/wireshape.go | 285 ++ internal/httpapi/wireshape/wireshape_test.go | 104 + 18 files changed, 4498 insertions(+), 30 deletions(-) create mode 100644 internal/httpapi/testdata/pretoken_parameters.json create mode 100644 internal/httpapi/wire_revision.go create mode 100644 internal/httpapi/wire_revision_test.go create mode 100644 internal/httpapi/wireshape/cmd/gendigest/main.go create mode 100644 internal/httpapi/wireshape/doc.go create mode 100644 internal/httpapi/wireshape/testdata/golden.json create mode 100644 internal/httpapi/wireshape/wireshape.go create mode 100644 internal/httpapi/wireshape/wireshape_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d809cba9..c03a88781 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 recognizes a registered remote backend (`backends.IsRemote`) before any Dolt-mode check, so a remote-backend workspace is never misclassified as having a local Dolt root to gate. +- `bd serve`'s `GET /v0/beads/context` now reports `wire_revision` (the + non-additive wire-shape counter for this build) and + `min_client_wire_revision` (the oldest revision this build still answers + correctly). A client may declare the revision it was built for on any + request via the optional `Bd-Wire-Revision` header; a declared revision + below `min_client_wire_revision` is refused with + `400 invalid_argument`/`reason: "wire_revision_unsupported"` and a + `min_wire_revision` field naming the floor. The header is absent by + default and an absent header is served exactly as before — this is + additive, not a new precondition on existing clients. See + `internal/httpapi/wire_revision.go` and the `wire_revision` property in + `internal/httpapi/spec/openapi.v0.yaml` for the full revision history. +- `GET /v0/beads/issues`'s `sort` parameter now advertises the + `issues.listSort` capability token, the same way an operation itself + does, and a CI rule + (`TestNewParameterOnExistingOperationHasABehaviorToken`) requires every + new parameter on an existing operation to carry one. +- A CI golden digest (`internal/httpapi/wireshape`) pins the JSON name and + type of every response member across the whole HTTP spec and fails if any + changes without `wire_revision` bumping to match; regenerate it with + `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, + revision-bumped change. ### Changed diff --git a/engdocs/design/bd-serve-v0.md b/engdocs/design/bd-serve-v0.md index 7e5d050da..b0bd0204e 100644 --- a/engdocs/design/bd-serve-v0.md +++ b/engdocs/design/bd-serve-v0.md @@ -529,3 +529,81 @@ it cannot discover before connecting. Refusing keeps the published surface a property of the build, and matches how `bd` already answers this question one layer down, where a backend that cannot guarantee mutation-free access is turned away rather than opened anyway. + +## The wire-shape signal + +`capabilities` answers which OPERATIONS a build serves. Nothing answered +whether an operation's response SHAPE had changed underneath a client already +built against it — a client decoding a strict schema against a future server +that renamed or retyped a field it depends on would fail wherever it happened +to read that field, with no single place to check first. `wire_revision` and +`min_client_wire_revision`, both on `GET /v0/beads/context`, are that single +place. + +`wire_revision` is a counter, not a semver: it bumps only for a NON-ADDITIVE +change to an existing response member's JSON name or type (a new, purely +additive member needs no bump). The full history is the `wire_revision` +property's description in `openapi.v0.yaml` — `0` for every build before this +field existed (an integer `revision`, pre-#6053), `1` for the brief +string-typed `revision` (#6053), `2` for the first build carrying +`wire_revision` itself. A client talking to a build too old to answer this +field at all infers the value from `bd_version`, which is always present; the +mapping is documented beside the property rather than assumed, because +`bd_version` alone does not sort the same way a counter does. + +`min_client_wire_revision` is the other half: the oldest revision a build +still answers correctly. A client that knows the revision it was compiled +against may assert it on ANY request via the optional `Bd-Wire-Revision` +header — not only the handshake — and `checkWireRevision` refuses a request +naming a revision below the floor with `400 invalid_argument`/ +`reason: "wire_revision_unsupported"` and a `min_wire_revision` field pinning +the floor that was violated, before the server does any work building a body +shaped for a revision the client has already said it cannot decode. The header +is optional and absent by default, which is what keeps this additive: an older +client that never sends it is served exactly as before. A malformed value +(not a non-negative integer) is a different mistake — `reason: "invalid_value"` +— and must never disclose `min_wire_revision`, which is reserved for a +revision the server actually read and understood to be too old. + +The check runs on every route but `GET /healthz` (`wireRevisionExempt`, +`routes.go`) — including the identity handshake itself, unlike the +`Bd-Project-Id` stamp, which exempts the handshake because that is how a +client LEARNS the id it must stamp with. A client that already knows the wire +revision it was built for has no equivalent reason to omit it on the very +first request, and the handshake is where a floor violation is cheapest to +catch: before the client has acted on anything shaped for a revision it +cannot decode. Liveness is exempt for the same reason `Bd-Project-Id` is: a +probe carries no notion of either header, and refusing it for one it never +had a reason to send would make `/healthz` lie about the process being alive. + +**The golden digest.** A revision bump is a promise a human makes; nothing +enforced that the promise was kept, or that it was even necessary. CI pins +one against the other with a golden digest, `internal/httpapi/wireshape`: +`wireshape.Compute` walks every response's every content-type schema across +the whole spec — not only `application/json`, so `application/problem+json` +and therefore `Problem` itself is covered too — recording the JSON name and +type of every member it finds, keyed by schema and member name. +`TestWireShapeDigest` fails one of two ways: a changed or removed entry at an +UNCHANGED `wire_revision` is drift nobody signed off on; a changed +`wire_revision` with an unchanged entry set means the golden is simply stale. +Either way the fix is the same explicit command, +`go run ./internal/httpapi/wireshape/cmd/gendigest` — never run to make a +failing test pass on an accidental shape change, only after the revision bump +and the `openapi.v0.yaml` history entry it belongs beside. + +**The new-parameter token rule.** `capabilities` already required a +`resource.verb` token for a new OPERATION; nothing required one for a new +PARAMETER on an EXISTING operation, so a client had no single place to learn +that, say, `GET /v0/beads/issues`'s `sort` parameter had started accepting a +value. `issues.listSort` is the first deliberate use of the same convention +one level down: the token lives in the parameter's own description, exactly as +`capabilities` members carry theirs, enforced by +`TestNewParameterOnExistingOperationHasABehaviorToken` against a frozen +baseline (`internal/httpapi/testdata/pretoken_parameters.json`) of every +parameter the document already had the day the rule was written — written +once and never regenerated, since only a pre-existing, untokened parameter +ever belongs on it. `TestUntokenedParameterRuleFires` proves the checker +itself still distinguishes tokened, grandfathered and untokened parameters, +against a synthetic fixture rather than the real spec, so the rule's own logic +is covered independently of whether the current document happens to exercise +every branch. diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 9ad2dc9be..573d43fcd 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1010,7 +1010,7 @@ type ContextResponse struct { // OPTIONAL, and absent means only that this server does not disclose its filesystem layout — never that it has no workspace. A client MUST NOT require it, MUST NOT treat absence as an error, and has no use for the value beyond display: it is a path on the SERVER's filesystem, which the client cannot open. Identify the workspace by `project_id` and `database`, which are required. BeadsDir *string `json:"beads_dir,omitempty"` - // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, and `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. + // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.listSort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. // // THIS LIST IS BUILD-LEVEL, NOT WORKSPACE-LEVEL. It says which operations this binary serves, and for every entry but two that is the whole answer. `events.list` and `events.watch` are the exceptions: the durable events journal is a per-workspace setting that is OFF by default, so a server that advertises them may still refuse every request to both with 409 `events_journal_disabled` — correctly, because the operations exist and the workspace has no journal. A consumer of either MUST treat the capability as "this server speaks it" and the 409 as "not on this workspace", and must not read the capability as a promise that records will arrive. Capabilities []string `json:"capabilities"` @@ -1021,6 +1021,11 @@ type ContextResponse struct { // DoltMode Which storage mode this workspace is served from. DoltMode string `json:"dolt_mode"` + // MinClientWireRevision The OLDEST client `wire_revision` this server still answers correctly. A client whose own compiled `wire_revision` is below this number should refuse to dial at all, the same way it refuses an unknown `api_version`: nothing it does with a response shaped for this revision is safe to assume. + // + // A client MAY declare the revision it was built for on every request with the optional `Bd-Wire-Revision` header (see the document-level rule above). When it does, and the declared value is below this member, the server refuses with `400` / `code: invalid_argument` / `param: "Bd-Wire-Revision"` / `reason: "wire_revision_unsupported"` rather than risk serving a body the caller has already said it cannot decode. The header is OPTIONAL and an absent one is served exactly as today — this is additive wire surface, not a new precondition on requests already in the field, exactly as `Bd-Project-Id` is. + MinClientWireRevision int `json:"min_client_wire_revision"` + // ProjectId Logical project identifier. ProjectId string `json:"project_id"` @@ -1029,6 +1034,31 @@ type ContextResponse struct { // SchemaVersion The shared JSON schema version — the same constant the CLI's stdout JSON envelope reports. Diagnostic only: it can move for CLI-only reasons with no HTTP wire change, so clients MUST NOT branch on it. SchemaVersion int `json:"schema_version"` + + // WireRevision A counter bumped on every release whose wire format changed in a way an unaware client could MISDECODE: an existing member's JSON type or meaning changed, or a member was removed or renamed. Adding a brand-new member to an existing schema, or a brand-new operation, is ADDITIVE and never bumps it — an old client simply never asks for the new member and is unaffected. + // + // Revision history, and it only ever grows by appending a row: + // + // * `0` — every server before 1.3.0. `revision` and + // `expected_version` were JSON integers, and this field did not + // exist: an absent `wire_revision` together with a `bd_version` + // below `1.3.0` IS a `0`. + // * `1` — 1.3.0 up to, but not including, the release that adds this + // field. `revision` and `expected_version` moved to DECIMAL-STRING + // tokens (#6053) under the same `api_version: "v0"`, with nothing + // in the handshake to tell an integer-expecting client the shape + // had moved underneath it. That silent gap is the whole reason + // this field exists. + // * `2` — the first release whose `ContextResponse` carries + // `wire_revision` and `min_client_wire_revision` at all (this one). + // + // + // A client talking to a server that omits this member entirely is talking to a pre-signal server and INFERS the revision rather than reading one: `1` when `bd_version` is `>= 1.3.0`, else `0`. A client that does not know how to speak the revision it reads or infers must refuse to proceed rather than guess at the shape. + // + // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member's (schema, member, type, format, enum, required) fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass. + // + // Every request PARAMETER (query, path or header — not a request body member) added to an EXISTING operation after this document must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against a frozen baseline (`internal/httpapi/testdata/pretoken_parameters.json`) of every parameter this document already had the day the rule was written. A parameter on that baseline is grandfathered and never needs one; any parameter not on it — present now or added later — must carry a token today. The baseline is written once and never regenerated: only a pre-existing, untokened parameter ever belongs on it, and a newly tokened parameter satisfies the rule without needing an entry at all. `issues.listSort` is the first deliberate use of the rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. + WireRevision int `json:"wire_revision"` } // CreateIssueDependency One edge created with the issue. It carries `reverse` where `BatchCreateDependency` does not, because that operation's items have no id a target could point back at and this one's issue does. @@ -1562,6 +1592,9 @@ type Problem struct { // ItemKind The `kind` of the item at `item_index`, so a client can dispatch on what the item was doing without walking its own request back. ItemKind *string `json:"item_kind,omitempty"` + // MinWireRevision With `invalid_argument` / `reason: "wire_revision_unsupported"` ONLY: this server's own `ContextResponse.min_client_wire_revision`, so a client that declared a `Bd-Wire-Revision` can log exactly how far behind it is without re-deriving the number from `bd_version`. It is set on that refusal and on no other. + MinWireRevision *int `json:"min_wire_revision,omitempty"` + // OpenChildren With `not_closable`: how many open children the transaction that refused the close observed, read inside that transaction rather than parsed out of `detail`. // // PRESENT ONLY for the open-children refusal. The other `not_closable` refusal is a live blocker and carries `blockers` instead, so member presence — not prose — is how a client tells the two apart. Both are bypassed by `force`. @@ -1572,7 +1605,7 @@ type Problem struct { // With `precondition_failed`: the body member carrying the guard that missed. It is the same spelling a 400 on the same operation would use, so a client reads one member to find the offending input whichever way the request was refused. Param *string `json:"param,omitempty"` - // Reason With `invalid_argument`: `unknown_parameter` (this server does not know that parameter — version skew; degrade or fall back), `invalid_value` (the value is not one this server will act on: malformed, out of vocabulary, or — for `limit=0` under `--allow-non-loopback` — legal but refused in this server's configuration; `detail` says which), or `project_mismatch` (the `Bd-Project-Id` header named a project this server does not serve — a document-level refusal like the Host-header 400, raised on every enforced route, and the one that carries `server_project_id`; see the document-level rule). Either way the recovery is to send something different, never to retry the same request. The set may grow; default-branch on unknown values. + // Reason With `invalid_argument`: `unknown_parameter` (this server does not know that parameter — version skew; degrade or fall back), `invalid_value` (the value is not one this server will act on: malformed, out of vocabulary, or — for `limit=0` under `--allow-non-loopback` — legal but refused in this server's configuration; `detail` says which), `project_mismatch` (the `Bd-Project-Id` header named a project this server does not serve — a document-level refusal like the Host-header 400, raised on every enforced route, and the one that carries `server_project_id`; see the document-level rule), or `wire_revision_unsupported` (the `Bd-Wire-Revision` header named a revision below `ContextResponse.min_client_wire_revision` — also a document-level refusal, raised on every route including the identity handshake, and the one that carries `min_wire_revision`; see the document-level rule and `ContextResponse.wire_revision`). Either way the recovery is to send something different, never to retry the same request. The set may grow; default-branch on unknown values. Reason *string `json:"reason,omitempty"` // RequestId Opaque correlation id for this request, echoed in the server's request log line. Never a dispatch key and never a retry key. (This server mints per-process ids that do not survive a restart; a deployment may substitute any identifier with the same log-correlation property, such as an edge trace id.) @@ -2197,7 +2230,7 @@ type ListIssuesParams struct { // // THE VOCABULARY IS CLOSED, and deliberately smaller than the nine values `bd list --sort` and `GET /v0/beads/issues:query` take. Each value here is a cursor contract, not a display preference: it needs a keyset predicate and a key proven total. See the operation description for why the other seven have neither. // - // A value outside the enum is a 400 `invalid_argument` with `param: "sort"` and `reason: "invalid_value"`. A server that predates this parameter answers `param: "sort"` with `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in `created` order and sorting client-side. + // A value outside the enum is a 400 `invalid_argument` with `param: "sort"` and `reason: "invalid_value"`. A server that predates this parameter answers `param: "sort"` with `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in `created` order and sorting client-side. This server also advertises the `issues.listSort` token in `ContextResponse.capabilities` for exactly that check, so a client with capabilities cached from the handshake never has to pay for the round trip an old server's `unknown_parameter` answer costs. Sort *ListIssuesParamsSort `form:"sort,omitempty" json:"sort,omitempty"` // Cursor Opaque keyset position, taken verbatim from a previous response's `next_cursor`. Clients MUST NOT construct, parse or mutate it: its encoding is server-private and versioned, and an undecodable or unknown-version value is refused with 400 `invalid_cursor`. The recovery for that refusal is normative: restart paging with no `cursor` at all — the position cannot be salvaged, and re-sending the same value cannot succeed. diff --git a/internal/httpapi/handlers.go b/internal/httpapi/handlers.go index 3f593966b..29a36e336 100644 --- a/internal/httpapi/handlers.go +++ b/internal/httpapi/handlers.go @@ -95,9 +95,11 @@ func contextResponse(info domain.ContextInfo, schemaVersion int, capabilities [] } published := domain.PublishedContext(info) return apigen.ContextResponse{ - ApiVersion: APIVersion, - SchemaVersion: schemaVersion, - Capabilities: capabilities, + ApiVersion: APIVersion, + SchemaVersion: schemaVersion, + Capabilities: capabilities, + WireRevision: CurrentWireRevision, + MinClientWireRevision: MinClientWireRevision, BdVersion: published.BdVersion, Backend: published.Backend, diff --git a/internal/httpapi/problem.go b/internal/httpapi/problem.go index cab58a9dc..7f500db04 100644 --- a/internal/httpapi/problem.go +++ b/internal/httpapi/problem.go @@ -306,6 +306,15 @@ const ( // `server_project_id`. The recovery is to stop stamping this server with // another workspace's id, never to retry the same request. ReasonProjectMismatch Reason = "project_mismatch" + // ReasonWireRevisionUnsupported means the request's Bd-Wire-Revision header + // named a revision below this server's ContextResponse.min_client_wire_revision. + // Like ReasonProjectMismatch it is a document-level 400 reachable on every + // enforced route — including the identity handshake itself — rather than + // per-operation behavior, and it is the one refusal that carries + // `min_wire_revision`. The recovery is to stop talking to this server with a + // client built for a revision it no longer supports, never to retry the same + // request. + ReasonWireRevisionUnsupported Reason = "wire_revision_unsupported" ) // staticDetail is the set of codes whose `detail` is FIXED, whatever the @@ -1345,6 +1354,29 @@ func ProjectMismatch(got, own string) Result { return res } +// WireRevisionUnsupported builds the 400 for a request whose Bd-Wire-Revision +// header names a revision below this server's own +// ContextResponse.min_client_wire_revision. got is the revision the client +// declared; min is this server's floor, disclosed in the `min_wire_revision` +// extension member for the same reason ProjectMismatch discloses +// `server_project_id`: so a client that already knows it declared a revision +// can tell exactly how far behind it is without re-deriving the number from +// `bd_version`. +// +// This is the ONLY refusal on the surface that sets `min_wire_revision`, and — +// unlike ProjectMismatch, which is exempt on the identity handshake because +// that is where a client LEARNS the id it must stamp with — it is raised on +// GET /v0/beads/context too: a client that already knows the revision it was +// built for gains nothing from being served a body it has already said it +// cannot decode, and loses the chance to fail before acting on it. +func WireRevisionUnsupported(got, min int) Result { + res := InvalidArgument(WireRevisionHeader, ReasonWireRevisionUnsupported, + "the "+WireRevisionHeader+" header names revision "+strconv.Itoa(got)+ + ", which is below the "+strconv.Itoa(min)+" this server still answers correctly") + res.Problem.MinWireRevision = &min + return res +} + // InvalidCursor builds the 400 for a cursor this server did not issue, cannot // decode, or issued under a different internal version. func InvalidCursor() Result { diff --git a/internal/httpapi/routes.go b/internal/httpapi/routes.go index acc417999..a56723172 100644 --- a/internal/httpapi/routes.go +++ b/internal/httpapi/routes.go @@ -59,6 +59,23 @@ const CapProjectEnforce = "project.enforce" // round trip an old server would refuse anyway. const CapBatchApplyLarge = "issues.batchApplyLarge" +// CapIssuesListSort is the behavior capability that advertises GET +// /v0/beads/issues' `sort` parameter (reads.go, spec `sort` on OpListIssues). +// Like CapBatchApplyLarge it names a PARAMETER added to an existing operation +// rather than a route of its own — `issues.list` is already the per-operation +// token for the route itself — so it rides the same behaviorCapabilities list. +// +// The parameter shipped (#5666) with no token at all: a client already had the +// per-parameter fallback (an old server answers `sort` with 400 +// `unknown_parameter`, which doubles as a capability probe), but that means +// paying a round trip an old server would refuse anyway. A client that checks +// this token at handshake time learns the same thing for free. This is also +// the spec/lint rule in TestNewParameterOnExistingOperationHasABehaviorToken: +// `sort` predates that rule and would have failed it, so it is the parity fix +// — ent already advertises the equivalent token — that the rule's own +// baseline is built from. +const CapIssuesListSort = "issues.listSort" + // customMethodTarget splits the custom method off the segment the router // matched, and reports the row that claims it. // @@ -173,6 +190,17 @@ type route struct { // reads that touch no workspace data are exempt and every other route — // streaming or not — is enforced. projectExempt bool + // wireRevisionExempt exempts an operation from the Bd-Wire-Revision floor + // check (checkWireRevision). Legitimate only for liveness: a kubelet probe + // carries no notion of a wire revision at all, and the check's refusal body + // is itself a `ContextResponse`-shaped disclosure the health row must never + // grow. It is deliberately NOT set on the identity handshake — unlike + // projectExempt, where the handshake is how a client LEARNS the id it must + // stamp with, a client that already knows the revision it was built for can + // and should declare it on the very first request, and the handshake is + // exactly where a floor violation is cheapest to catch: before the client + // has acted on anything shaped for a revision it cannot decode. + wireRevisionExempt bool // implemented gates the capability list, so a release between slices never // advertises an operation that does not work. Every v0 operation is // implemented as of the read-endpoints slice; the flag stays because the @@ -203,8 +231,12 @@ var routeTable = []route{ // liveness probe gated on a matching project stamp would go dark on a // misconfigured client exactly when an operator needs it most. projectExempt: true, - implemented: true, - handler: (*Server).handleHealth, + // Same reasoning extends to the wire-revision floor: a probe carries no + // `Bd-Wire-Revision` of its own, and must not be refused for a header it + // never had a reason to send. + wireRevisionExempt: true, + implemented: true, + handler: (*Server).handleHealth, }, { op: OpGetContext, @@ -778,7 +810,7 @@ func (r route) specPathOf() string { // route. project.enforce announces per-request Bd-Project-Id enforcement // (checkProjectStamp): a stamped client reads it to know the refusal is available // rather than silently dropped by an older server. -var behaviorCapabilities = []string{CapProjectEnforce, CapBatchApplyLarge} +var behaviorCapabilities = []string{CapProjectEnforce, CapBatchApplyLarge, CapIssuesListSort} // Capabilities lists what this build advertises in ContextResponse.capabilities: // the operations it actually implements, gated on `implemented` so a stub can diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index dfdff48f6..26de31cdb 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -491,6 +491,15 @@ type Server struct { writeStall time.Duration watchPoll time.Duration watchBeat time.Duration + // minClientWireRevision is the field checkWireRevision enforces against. + // It starts at the package constant MinClientWireRevision and exists as + // a field, rather than a direct read of that constant, only so a test can + // raise the floor in milliseconds the way it shrinks semTimeout above — + // production never overrides it. Unlike semTimeout et al it is an int + // with no zero-value ambiguity to paper over with orDefault: the + // constant's own real-world default is 0, so a freshly zeroed Server + // behaves identically whether or not Listen ran yet. + minClientWireRevision int // drainTimeout overrides the package-level drainTimeout constant used as // drainBudget's floor. A test shrinks this to prove Serve's drain // actually EXTENDS past it for an in-flight large apply, instead of @@ -632,11 +641,12 @@ func Listen(cfg Config) (*Server, error) { workspaceMemories: cfg.Memories, eventsJournal: cfg.EventsJournal, - sem: make(chan struct{}, maxInflight), - semTimeout: semAcquireTimeout, - semWarn: saturationWarn, - largeApplySem: make(chan struct{}, 1), - largeApplyCeiling: orDefault(cfg.LargeApplyCeiling, DefaultLargeApplyCeiling), + sem: make(chan struct{}, maxInflight), + semTimeout: semAcquireTimeout, + semWarn: saturationWarn, + minClientWireRevision: MinClientWireRevision, + largeApplySem: make(chan struct{}, 1), + largeApplyCeiling: orDefault(cfg.LargeApplyCeiling, DefaultLargeApplyCeiling), closing: make(chan struct{}), maxWatchStreams: maxWatchStreams, @@ -1872,6 +1882,18 @@ func (s *Server) route(rt route) http.Handler { } } + // Wire-floor before resources, same as the project stamp: a client that + // has declared a revision this build no longer supports is turned away + // before it can buy a database slot or open a unit of work. Checked on + // every route but health (see wireRevisionExempt) — including the + // identity handshake, where catching the mismatch is cheapest. + if !rt.wireRevisionExempt { + if res := s.checkWireRevision(r); res != nil { + s.fail(w, r, *res) + return + } + } + if !rt.bypassSemaphore { release, err := s.acquire(r.Context(), rec) if err != nil { diff --git a/internal/httpapi/server_test.go b/internal/httpapi/server_test.go index 6bae6ef99..1b1dff570 100644 --- a/internal/httpapi/server_test.go +++ b/internal/httpapi/server_test.go @@ -635,7 +635,7 @@ func TestCapabilitiesAdvertiseEveryImplementedOperation(t *testing.T) { // them; project.enforce and issues.batchApplyLarge are advertised in the // same list. Both are spelled literally here to keep this an independent // oracle rather than a second call to the code under test. - want = append(want, "project.enforce", "issues.batchApplyLarge") + want = append(want, "project.enforce", "issues.batchApplyLarge", "issues.listSort") slices.Sort(want) if !slices.Equal(got, want) { t.Errorf("capabilities = %v, want %v", got, want) diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index 71d23b419..ca0cb8bef 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -52,7 +52,7 @@ info: it decides every refusal before opening its stream. - ## Four 400s cut across operations, and are documented here once + ## Five 400s cut across operations, and are documented here once * **Host header.** A request whose `Host` header is not in the server's @@ -121,10 +121,34 @@ info: id to stamp with. A client detects that a server performs this check by the `project.enforce` capability (see `ContextResponse`). + * **Wire-revision floor.** A request MAY carry a `Bd-Wire-Revision` header + naming the integer `wire_revision` (see `ContextResponse.wire_revision`) + the CLIENT was built to decode. When the header is present, is a + non-negative integer, and names a revision BELOW + `ContextResponse.min_client_wire_revision`, the request is refused with + `400` / `code: invalid_argument` / `param: "Bd-Wire-Revision"` / + `reason: "wire_revision_unsupported"`, and that refusal carries this + server's own floor in the `min_wire_revision` member so the client can + log exactly how far behind it is without re-deriving the number from + `bd_version`. A header value that is not a non-negative integer is + refused the ordinary way a malformed value is: `400` / + `reason: "invalid_value"`. An ABSENT header is the backward-compatible + path, exactly as for `Bd-Project-Id`: a client that never declares a + revision is served exactly as today, so this is additive wire surface, + never a new precondition on requests already in the field. UNLIKE the + project-id stamp, this check is NOT skipped on `GET /v0/beads/context`: + the handshake is precisely where a client's floor matters most, because + serving a `ContextResponse` to a client that has already said it cannot + decode this revision would defeat the whole signal. It IS skipped on + `GET /healthz` alone, for the reason liveness is always exempt: it must + answer whatever client reaches it. See `ContextResponse.wire_revision` + for the full version-signal design. + The first two are uniform rules reachable on EVERY route including - `GET /healthz`; the third holds on every body-carrying operation; the fourth - holds on every route EXCEPT the two exempt reads named above. All four are + `GET /healthz`; the third holds on every body-carrying operation; the + fourth holds on every route EXCEPT the two exempt reads named above; the + fifth holds on every route except `GET /healthz` alone. All five are stated here instead of being repeated on every operation. The per-operation `responses` below therefore list what an operation produces BEYOND them: a generated client must treat `400 invalid_argument` as possible everywhere, @@ -192,6 +216,13 @@ info: * `schema_version` is NOT an HTTP gate. It is shared with the CLI's stdout JSON envelope, so it can move for CLI-only reasons with no HTTP wire change. Report it, log it, never branch on it. + * `wire_revision` and `min_client_wire_revision` gate WIRE SHAPE, not + behavior: a non-additive change to an EXISTING member's type, meaning, + presence or name bumps `wire_revision`; a brand-new member or operation + does not. A client that reads no `wire_revision` at all is talking to a + server that predates it and infers one from `bd_version` (`>= 1.3.0` + means `1`, else `0`); see `ContextResponse.wire_revision` for the full + table and the `Bd-Wire-Revision` request header. * Clients MUST default-branch on unknown `code` and `reason` values within a status class: both vocabularies grow additively, so an exhaustive switch breaks on the first addition. @@ -1281,7 +1312,11 @@ paths: predates this parameter answers `param: "sort"` with `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in - `created` order and sorting client-side. + `created` order and sorting client-side. This server also + advertises the `issues.listSort` token in + `ContextResponse.capabilities` for exactly that check, so a client + with capabilities cached from the handshake never has to pay for + the round trip an old server's `unknown_parameter` answer costs. schema: type: string enum: [created, priority] @@ -7385,7 +7420,7 @@ components: conforms to this document. required: [api_version, bd_version, schema_version, backend, dolt_mode, database, - project_id, capabilities] + project_id, capabilities, wire_revision, min_client_wire_revision] properties: api_version: type: string @@ -7458,7 +7493,7 @@ components: `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace - is refused here rather than silently ignored, and + is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB @@ -7472,10 +7507,13 @@ components: server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an - older server would refuse anyway. The list grows additively, and - an operation never appears here unless it is fully implemented. - This is how a client checks for an operation or a behavior — - never the version string. + older server would refuse anyway, and `issues.listSort`, which + announces that `GET /v0/beads/issues` accepts the `sort` query + parameter (two closed, pageable total orders — see that + parameter) rather than silently answering `unknown_parameter`. The + list grows additively, and an operation never appears here unless + it is fully implemented. This is how a client checks for an + operation or a behavior — never the version string. THIS LIST IS BUILD-LEVEL, NOT WORKSPACE-LEVEL. It says which @@ -7490,6 +7528,91 @@ components: not read the capability as a promise that records will arrive. items: type: string + wire_revision: + type: integer + description: >- + A counter bumped on every release whose wire format changed in a + way an unaware client could MISDECODE: an existing member's JSON + type or meaning changed, or a member was removed or renamed. + Adding a brand-new member to an existing schema, or a brand-new + operation, is ADDITIVE and never bumps it — an old client simply + never asks for the new member and is unaffected. + + + Revision history, and it only ever grows by appending a row: + + + * `0` — every server before 1.3.0. `revision` and + `expected_version` were JSON integers, and this field did not + exist: an absent `wire_revision` together with a `bd_version` + below `1.3.0` IS a `0`. + * `1` — 1.3.0 up to, but not including, the release that adds this + field. `revision` and `expected_version` moved to DECIMAL-STRING + tokens (#6053) under the same `api_version: "v0"`, with nothing + in the handshake to tell an integer-expecting client the shape + had moved underneath it. That silent gap is the whole reason + this field exists. + * `2` — the first release whose `ContextResponse` carries + `wire_revision` and `min_client_wire_revision` at all (this one). + + + A client talking to a server that omits this member entirely is + talking to a pre-signal server and INFERS the revision rather than + reading one: `1` when `bd_version` is `>= 1.3.0`, else `0`. A + client that does not know how to speak the revision it reads or + infers must refuse to proceed rather than guess at the shape. + + + This document's own drift gate is `TestWireShapeDigest` + (`internal/httpapi/wireshape`): a golden digest of every EXISTING + response member's (schema, member, type, format, enum, required) + fails CI the moment any one of them changes without this field's + Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) + increasing to match. Regenerate the golden with + `go run ./internal/httpapi/wireshape/cmd/gendigest` after a + deliberate, revision-bumped change, and commit the result — see + that package's doc.go for the exact command. A purely additive + member or operation also changes the golden (it is appended to, + never frozen in place) but needs no revision bump to pass. + + + Every request PARAMETER (query, path or header — not a request + body member) added to an EXISTING operation after this document + must carry a capability token in its own description, exactly as + `capabilities` members do above — + `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it + against a frozen baseline + (`internal/httpapi/testdata/pretoken_parameters.json`) of every + parameter this document already had the day the rule was written. + A parameter on that baseline is grandfathered and never needs one; + any parameter not on it — present now or added later — must carry + a token today. The baseline is written once and never + regenerated: only a pre-existing, untokened parameter ever + belongs on it, and a newly tokened parameter satisfies the rule + without needing an entry at all. `issues.listSort` is the first + deliberate use of the rule; it also closes the one gap the rule + was written to catch, since `sort` itself shipped (#5666) with no + token at all. + min_client_wire_revision: + type: integer + description: >- + The OLDEST client `wire_revision` this server still answers + correctly. A client whose own compiled `wire_revision` is below + this number should refuse to dial at all, the same way it refuses + an unknown `api_version`: nothing it does with a response shaped + for this revision is safe to assume. + + + A client MAY declare the revision it was built for on every + request with the optional `Bd-Wire-Revision` header (see the + document-level rule above). When it does, and the declared value + is below this member, the server refuses with `400` / + `code: invalid_argument` / `param: "Bd-Wire-Revision"` / + `reason: "wire_revision_unsupported"` rather than risk serving a + body the caller has already said it cannot decode. The header is + OPTIONAL and an absent one is served exactly as today — this is + additive wire surface, not a new precondition on requests already + in the field, exactly as `Bd-Project-Id` is. IssueCount: type: object @@ -11179,13 +11302,19 @@ components: `invalid_value` (the value is not one this server will act on: malformed, out of vocabulary, or — for `limit=0` under `--allow-non-loopback` — legal but refused in this server's - configuration; `detail` says which), or `project_mismatch` (the + configuration; `detail` says which), `project_mismatch` (the `Bd-Project-Id` header named a project this server does not serve — a document-level refusal like the Host-header 400, raised on every enforced route, and the one that carries `server_project_id`; see - the document-level rule). Either way the recovery is to send - something different, never to retry the same request. The set may - grow; default-branch on unknown values. + the document-level rule), or `wire_revision_unsupported` (the + `Bd-Wire-Revision` header named a revision below + `ContextResponse.min_client_wire_revision` — also a document-level + refusal, raised on every route including the identity handshake, + and the one that carries `min_wire_revision`; see the + document-level rule and `ContextResponse.wire_revision`). Either + way the recovery is to send something different, never to retry + the same request. The set may grow; default-branch on unknown + values. assignee: type: string description: >- @@ -11446,3 +11575,11 @@ components: and never on a refusal raised before the stamp is checked — the Host gate, or a deployment's authentication layer — so its PRESENCE is the signal that this specific check fired. + min_wire_revision: + type: integer + description: >- + With `invalid_argument` / `reason: "wire_revision_unsupported"` + ONLY: this server's own `ContextResponse.min_client_wire_revision`, + so a client that declared a `Bd-Wire-Revision` can log exactly how + far behind it is without re-deriving the number from `bd_version`. + It is set on that refusal and on no other. diff --git a/internal/httpapi/spec_parity_test.go b/internal/httpapi/spec_parity_test.go index bb7973a91..075894866 100644 --- a/internal/httpapi/spec_parity_test.go +++ b/internal/httpapi/spec_parity_test.go @@ -1,7 +1,9 @@ package httpapi import ( + "encoding/json" "os" + "path/filepath" "reflect" "regexp" "slices" @@ -470,6 +472,128 @@ func TestSpecCapabilityVocabularyMatchesTheRouteTable(t *testing.T) { } } +// untokenedParameters reports "operationId.paramName" for every parameter in +// ops whose own description carries no capabilityToken and whose +// (operationId, paramName) pair is not in baseline. doc resolves the $refs a +// parameter may point at (components.parameters); baseline is keyed by +// operationId to the list of its parameter names exempt from the rule. +// +// Factored out of TestNewParameterOnExistingOperationHasABehaviorToken so +// TestUntokenedParameterRuleFires can falsify it against a fixture with no +// dependency on the real document: the rule fires on its own fabricated +// miss, not just on whatever this spec currently happens to contain. +func untokenedParameters(t *testing.T, doc map[string]any, ops map[string]specOp, baseline map[string][]string) []string { + t.Helper() + grandfathered := map[string]bool{} + for opID, params := range baseline { + for _, p := range params { + grandfathered[opID+"\x00"+p] = true + } + } + var missing []string + for opID, so := range ops { + raw, ok := so.op["parameters"].([]any) + if !ok { + continue + } + for _, r := range raw { + pm, ok := r.(map[string]any) + if !ok { + t.Fatalf("%s: parameter is %T, want a mapping", opID, r) + } + pm = resolveRef(t, doc, pm) + name, _ := pm["name"].(string) + if name == "" { + t.Fatalf("%s: parameter with no name", opID) + } + if grandfathered[opID+"\x00"+name] { + continue + } + desc, _ := pm["description"].(string) + if len(capabilityToken.FindAllString(desc, -1)) == 0 { + missing = append(missing, opID+"."+name) + } + } + } + sort.Strings(missing) + return missing +} + +func loadPretokenBaseline(t *testing.T) map[string][]string { + t.Helper() + blob, err := os.ReadFile(filepath.Join("testdata", "pretoken_parameters.json")) + if err != nil { + t.Fatalf("read pretoken baseline: %v", err) + } + var baseline map[string][]string + if err := json.Unmarshal(blob, &baseline); err != nil { + t.Fatalf("decode %s: %v", "testdata/pretoken_parameters.json", err) + } + return baseline +} + +// TestNewParameterOnExistingOperationHasABehaviorToken is the lint rule S0 +// item 4 asks for: a parameter on an existing operation with no capability +// token in its own description is undocumented dispatch surface — a client +// has no way to learn THIS server accepts it without sending it and risking +// `unknown_parameter`, which is exactly what happened to `sort` (#5666) +// before `issues.listSort` closed the gap. +// +// The baseline (testdata/pretoken_parameters.json) is a WRITE-ONCE grandfather +// list: every parameter this document already had the day the rule was +// written, captured by walking the spec with this same token regex. A +// parameter on it is exempt permanently, and nothing here ever regenerates +// the file. Any parameter not on it — present today or added by any later +// change — must carry a token, which is what makes this a ratchet rather than +// a one-time sweep: `issues.listSort` is already the parity fix the rule's +// own baseline is built from, not an exemption to it. +func TestNewParameterOnExistingOperationHasABehaviorToken(t *testing.T) { + doc := loadSpec(t) + ops := specOps(t, doc) + baseline := loadPretokenBaseline(t) + + if missing := untokenedParameters(t, doc, ops, baseline); len(missing) > 0 { + t.Errorf("parameter(s) on an existing operation carry no capability token and are not on the "+ + "frozen baseline (internal/httpapi/testdata/pretoken_parameters.json): %v\n"+ + "add a `resource.verb` token to the parameter's own description, the way `sort` on "+ + "GET /v0/beads/issues advertises `issues.listSort`", missing) + } +} + +// TestUntokenedParameterRuleFires is the rule's own falsification, required +// because TestNewParameterOnExistingOperationHasABehaviorToken is permanently +// green against a spec written to satisfy it: a synthetic operation proves +// the CHECK itself still distinguishes the three cases it exists to tell +// apart, independent of anything the real document currently says. +func TestUntokenedParameterRuleFires(t *testing.T) { + doc := map[string]any{} // no $refs in the fixtures below; resolveRef never consults it + newOp := func(desc string) specOp { + return specOp{ + method: "GET", + path: "/synthetic", + op: map[string]any{ + "parameters": []any{ + map[string]any{"name": "widget", "description": desc}, + }, + }, + } + } + + ops := map[string]specOp{"syntheticOp": newOp("plain prose, no token at all")} + if got := untokenedParameters(t, doc, ops, map[string][]string{}); !slices.Equal(got, []string{"syntheticOp.widget"}) { + t.Fatalf("untokened, non-grandfathered parameter: got %v, want [syntheticOp.widget]", got) + } + + if got := untokenedParameters(t, doc, ops, map[string][]string{"syntheticOp": {"widget"}}); len(got) != 0 { + t.Fatalf("grandfathered parameter still reported: got %v, want none", got) + } + + ops["syntheticOp"] = newOp("carries the `synthetic.widget` token") + if got := untokenedParameters(t, doc, ops, map[string][]string{}); len(got) != 0 { + t.Fatalf("tokened parameter still reported: got %v, want none", got) + } +} + func specParam(t *testing.T, so specOp, name string) map[string]any { t.Helper() raw, ok := so.op["parameters"].([]any) @@ -682,7 +806,8 @@ func TestDefaultsMatchCLIFlags(t *testing.T) { // re-adding one costs a test edit. var contextResponseAllowlist = []string{ "api_version", "backend", "bd_version", "beads_dir", "capabilities", - "database", "dolt_mode", "project_id", "repo_root", "schema_version", + "database", "dolt_mode", "min_client_wire_revision", "project_id", + "repo_root", "schema_version", "wire_revision", } // TestContextResponseAllowlist pins that field set from both sides: the diff --git a/internal/httpapi/testdata/pretoken_parameters.json b/internal/httpapi/testdata/pretoken_parameters.json new file mode 100644 index 000000000..06fc860e5 --- /dev/null +++ b/internal/httpapi/testdata/pretoken_parameters.json @@ -0,0 +1,182 @@ +{ + "addComment": [ + "id" + ], + "claimIssue": [ + "id" + ], + "claimNextIssue": [ + "assignee", + "exclude_label", + "exclude_type", + "has_metadata_key", + "include_deferred", + "include_ephemeral", + "label", + "label_any", + "label_pattern", + "label_regex", + "metadata_field", + "parent", + "priority", + "sort", + "type", + "unassigned" + ], + "closeIssue": [ + "id" + ], + "compareAndSetMetadata": [ + "id" + ], + "countDependencyEdges": [ + "direction", + "issue_id", + "status", + "type" + ], + "countIssues": [ + "assignee", + "closed_after", + "closed_before", + "created_after", + "created_before", + "desc_contains", + "empty_description", + "group_by", + "has_metadata_key", + "id", + "include_ephemeral", + "include_infra", + "label", + "label_any", + "metadata_field", + "no_assignee", + "no_labels", + "notes_contains", + "priority", + "priority_max", + "priority_min", + "status", + "title", + "title_contains", + "type", + "updated_after", + "updated_before" + ], + "countReadyWork": [ + "assignee", + "exclude_label", + "exclude_type", + "has_metadata_key", + "include_deferred", + "include_ephemeral", + "label", + "label_any", + "label_pattern", + "label_regex", + "metadata_field", + "parent", + "priority", + "type", + "unassigned" + ], + "forgetMemory": [ + "key" + ], + "getDependencyTree": [ + "direction", + "max_depth", + "root_id", + "status" + ], + "getIssue": [ + "brief_deps", + "id", + "include_comments", + "include_dependents" + ], + "getMemory": [ + "key" + ], + "getStats": [ + "assignee", + "skip_blocked" + ], + "listBlockingAnnotations": [ + "issue_id" + ], + "listDependencies": [ + "issue_id", + "type" + ], + "listEvents": [ + "limit", + "since" + ], + "listIssues": [ + "all", + "assignee", + "brief", + "created_after", + "created_before", + "cursor", + "exclude_label", + "has_metadata_key", + "include_ephemeral", + "include_gates", + "include_infra", + "include_templates", + "label", + "label_any", + "metadata_field", + "parent", + "status", + "type" + ], + "listMemories": [ + "search" + ], + "listReadyWork": [ + "assignee", + "brief", + "exclude_label", + "exclude_type", + "has_metadata_key", + "include_deferred", + "include_ephemeral", + "label", + "label_any", + "label_pattern", + "label_regex", + "metadata_field", + "parent", + "priority", + "sort", + "type", + "unassigned" + ], + "listRelatedIssues": [ + "id", + "type" + ], + "queryIssues": [ + "all", + "q", + "reverse", + "sort" + ], + "releaseIssue": [ + "id" + ], + "reopenIssue": [ + "id" + ], + "updateIssue": [ + "id" + ], + "watchEvents": [ + "Last-Event-ID", + "since" + ] +} diff --git a/internal/httpapi/wire_revision.go b/internal/httpapi/wire_revision.go new file mode 100644 index 000000000..303374b47 --- /dev/null +++ b/internal/httpapi/wire_revision.go @@ -0,0 +1,72 @@ +package httpapi + +import ( + "net/http" + "strconv" +) + +// WireRevisionHeader names the optional per-request wire-floor declaration. A +// client that knows the revision it was compiled against puts that revision +// here; checkWireRevision refuses the request when it names a revision below +// MinClientWireRevision, before the client can act on a body shaped for a +// revision it has already said it cannot decode. The header is optional and +// absent by default: an older client that never sends it is served exactly as +// before, which is what keeps this additive rather than a new precondition. +const WireRevisionHeader = "Bd-Wire-Revision" + +// CurrentWireRevision is ContextResponse.wire_revision: the wire-shape counter +// for THIS build. See the `wire_revision` property in openapi.v0.yaml for the +// full revision history and what bumps it. It increases only alongside a +// deliberate, non-additive wire change AND a matching update to the +// internal/httpapi/wireshape golden digest — TestWireShapeDigest fails closed +// the moment the two disagree. +const CurrentWireRevision = 2 + +// MinClientWireRevision is ContextResponse.min_client_wire_revision: the +// oldest client-declared revision this build still answers correctly. Raising +// it is the deliberate act of dropping support for clients built against an +// older wire shape; it must never exceed CurrentWireRevision. +const MinClientWireRevision = 0 + +// checkWireRevision enforces the optional per-request wire-floor declaration. +// A client that knows the revision it was compiled against may assert it in +// the Bd-Wire-Revision header; if that revision is below s.minClientWireRevision, +// the request is refused before the server spends any work building a body the +// caller has already said it cannot decode. It returns nil when the request +// may proceed, or the 400 to write. +// +// s.minClientWireRevision starts at the package constant MinClientWireRevision +// and is a Server field — rather than a direct read of that constant — only so +// a test can raise the floor; production never overrides it. +// +// Three paths return nil. An ABSENT header is the backward-compatible one: an +// older client never sends it, so enforcement triggers only when the header +// arrives and adds no precondition to any request already in the field. A +// non-negative integer at or above s.minClientWireRevision is in range. A value +// that fails to parse as a non-negative integer is NOT this check's refusal — +// it is a malformed header, reported as `invalid_value` rather than +// `wire_revision_unsupported`, which is reserved for a revision this server +// has read and understood to be too old. +// +// The refusal is recorded on the request line like every other middleware +// refusal, so a client persistently declaring a dropped revision is +// attributable down to the local process. +func (s *Server) checkWireRevision(r *http.Request) *Result { + raw := r.Header.Get(WireRevisionHeader) + if raw == "" { + return nil + } + got, err := strconv.Atoi(raw) + if err != nil || got < 0 { + requestInfo(r.Context()).refuse(raw) + res := InvalidArgument(WireRevisionHeader, ReasonInvalidValue, + "the "+WireRevisionHeader+" header must be a non-negative integer") + return &res + } + if got >= s.minClientWireRevision { + return nil + } + requestInfo(r.Context()).refuse(raw) + res := WireRevisionUnsupported(got, s.minClientWireRevision) + return &res +} diff --git a/internal/httpapi/wire_revision_test.go b/internal/httpapi/wire_revision_test.go new file mode 100644 index 000000000..d5681a3fd --- /dev/null +++ b/internal/httpapi/wire_revision_test.go @@ -0,0 +1,222 @@ +package httpapi + +import ( + "net/http" + "slices" + "strconv" + "strings" + "testing" +) + +// These are pure, like the rest of the package's edge tests: the whole +// request lifecycle runs over a real listener against a fake provider, so the +// Bd-Wire-Revision gate — where it sits in route(), what it refuses, and +// that it reaches the identity handshake too — is covered on every pull +// request by the unconditional Go test job. + +// wireRevisioned drives a request carrying (or omitting) the Bd-Wire-Revision +// header. An empty raw value sends no header at all, which is the +// backward-compatible path. +func (ts *testServer) wireRevisioned(t *testing.T, method, path, raw string) *http.Response { + t.Helper() + req, err := http.NewRequest(method, ts.base+path, nil) + if err != nil { + t.Fatalf("new %s %s: %v", method, path, err) + } + if raw != "" { + req.Header.Set(WireRevisionHeader, raw) + } + resp, err := ts.client.Do(req) + if err != nil { + t.Fatalf("%s %s: %v", method, path, err) + } + t.Cleanup(func() { _ = resp.Body.Close() }) + return resp +} + +func assertWireRevisionUnsupported(t *testing.T, resp *http.Response, wantMin int) { + t.Helper() + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + if got := resp.Header.Get("Content-Type"); got != "application/problem+json; charset=utf-8" { + t.Errorf("Content-Type = %q, want problem+json", got) + } + body := decodeBody(t, resp) + if body["code"] != string(CodeInvalidArgument) { + t.Errorf("code = %v, want %s", body["code"], CodeInvalidArgument) + } + if body["param"] != WireRevisionHeader { + t.Errorf("param = %v, want %s", body["param"], WireRevisionHeader) + } + if body["reason"] != string(ReasonWireRevisionUnsupported) { + t.Errorf("reason = %v, want %s", body["reason"], ReasonWireRevisionUnsupported) + } + // The one member that scopes the disclosure: it carries this server's own + // floor, and it is present ONLY on this refusal. + min, ok := body["min_wire_revision"].(float64) + if !ok || int(min) != wantMin { + t.Errorf("min_wire_revision = %#v, want %d", body["min_wire_revision"], wantMin) + } +} + +// wireRevisionReadServer is newReadServer's provider wiring (reads_test.go), +// needed here because /v0/beads/ready 500s against the bare default +// fakeProvider{} newTestServer(t, Config{}) otherwise falls back to — that +// fake answers the claim path only, not a read. +func wireRevisionReadServer(t *testing.T, tune ...func(*Server)) *testServer { + t.Helper() + cfg := Config{Provider: &fakeProvider{ + issues: &fakeIssues{}, + readIssues: &recordingIssues{}, + readConfig: emptyConfig{}, + }} + return newTestServer(t, cfg, tune...) +} + +// TestWireRevisionAbsentIsTodaysBehavior is the backward-compatibility +// contract: a client that sends no Bd-Wire-Revision header is served exactly +// as before, on both an ordinary route and the identity handshake. +func TestWireRevisionAbsentIsTodaysBehavior(t *testing.T) { + ts := wireRevisionReadServer(t) + + for _, path := range []string{"/v0/beads/ready", "/v0/beads/context"} { + resp := ts.wireRevisioned(t, http.MethodGet, path, "") + if resp.StatusCode != http.StatusOK { + t.Errorf("GET %s with no header = %d, want 200: %s", path, resp.StatusCode, readAll(t, resp)) + } + } +} + +// TestWireRevisionAtOrAboveFloorServes: a declared revision at or above +// MinClientWireRevision is in range and the request runs exactly as an +// undeclared one does. +func TestWireRevisionAtOrAboveFloorServes(t *testing.T) { + ts := wireRevisionReadServer(t) + + resp := ts.wireRevisioned(t, http.MethodGet, "/v0/beads/ready", strconv.Itoa(MinClientWireRevision)) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d, want 200 for a floor-equal revision: %s", resp.StatusCode, readAll(t, resp)) + } +} + +// TestWireRevisionBelowFloorRefusesEveryEnforcedRoute mirrors +// TestProjectStampMismatchRefusesEveryEnforcedRoute: a declared revision below +// the server's floor is refused on an ordinary read AND on the identity +// handshake — the one route Bd-Project-Id is exempt on but this header is +// deliberately not, since the handshake is where catching the mismatch is +// cheapest. +// +// The floor is raised to 3 via the tune hook (mirroring semTimeout et al in +// newTestServer's tune pattern) rather than read at its production default of +// 0: a non-negative integer strictly below a floor of 0 does not exist, so +// MinClientWireRevision itself could never exercise this path. +func TestWireRevisionBelowFloorRefusesEveryEnforcedRoute(t *testing.T) { + const floor = 3 + ts := wireRevisionReadServer(t, func(s *Server) { s.minClientWireRevision = floor }) + below := strconv.Itoa(floor - 1) + + for _, path := range []string{"/v0/beads/ready", "/v0/beads/context"} { + t.Run(path, func(t *testing.T) { + resp := ts.wireRevisioned(t, http.MethodGet, path, below) + assertWireRevisionUnsupported(t, resp, floor) + }) + } + + // Attributable, like every other middleware refusal. + line := findLogLine(t, ts.stderr.String(), "op="+OpListReadyWork) + for _, want := range []string{"status=400", "code=invalid_argument", "refused=" + logValue(below)} { + if !strings.Contains(line, want) { + t.Errorf("the refused request's log line is missing %q:\n%s", want, line) + } + } +} + +// TestWireRevisionMalformedIsInvalidValueNotUnsupported: a header that does +// not parse as a non-negative integer is a different client mistake than a +// revision this server has read and understood to be too old, and must not +// be confused with one by carrying min_wire_revision. +func TestWireRevisionMalformedIsInvalidValueNotUnsupported(t *testing.T) { + ts := wireRevisionReadServer(t) + + for _, raw := range []string{"not-a-number", "-1", "1.5", ""} { + if raw == "" { + continue // the empty string is the absent-header case, tested elsewhere + } + t.Run(raw, func(t *testing.T) { + resp := ts.wireRevisioned(t, http.MethodGet, "/v0/beads/ready", raw) + if resp.StatusCode != http.StatusBadRequest { + t.Fatalf("status = %d, want 400: %s", resp.StatusCode, readAll(t, resp)) + } + body := decodeBody(t, resp) + if body["reason"] != string(ReasonInvalidValue) { + t.Errorf("reason = %v, want %s", body["reason"], ReasonInvalidValue) + } + if _, present := body["min_wire_revision"]; present { + t.Errorf("a malformed header disclosed min_wire_revision: %v", body["min_wire_revision"]) + } + }) + } +} + +// TestHealthAnswersDespiteABelowFloorRevision: liveness is exempt. A kubelet +// probe carries no Bd-Wire-Revision of its own, and must not be refused for a +// header it never had a reason to send. +func TestHealthAnswersDespiteABelowFloorRevision(t *testing.T) { + ts := newTestServer(t, Config{}) + + resp := ts.wireRevisioned(t, http.MethodGet, "/healthz", strconv.Itoa(MinClientWireRevision-1)) + if resp.StatusCode != http.StatusOK { + t.Errorf("GET /healthz with a below-floor revision = %d, want 200 (this route is exempt)", resp.StatusCode) + } +} + +// TestContextAdvertisesWireRevision: the handshake carries both members every +// client needs to decide whether it can safely proceed. +func TestContextAdvertisesWireRevision(t *testing.T) { + ts := newTestServer(t, Config{}) + + body := decodeBody(t, ts.get(t, "/v0/beads/context")) + wire, ok := body["wire_revision"].(float64) + if !ok || int(wire) != CurrentWireRevision { + t.Errorf("wire_revision = %#v, want %d", body["wire_revision"], CurrentWireRevision) + } + min, ok := body["min_client_wire_revision"].(float64) + if !ok || int(min) != MinClientWireRevision { + t.Errorf("min_client_wire_revision = %#v, want %d", body["min_client_wire_revision"], MinClientWireRevision) + } +} + +// TestWireRevisionExemptRoutesAreExactlyHealth pins the exempt column by +// enumeration: exactly {OpHealth} skips the Bd-Wire-Revision floor check. +// Unlike projectExempt, the identity handshake is deliberately NOT on this +// list — see wireRevisionExempt's doc comment for why the two diverge. +func TestWireRevisionExemptRoutesAreExactlyHealth(t *testing.T) { + var exempt []string + for _, rt := range routeTable { + if rt.wireRevisionExempt { + exempt = append(exempt, rt.op) + } + } + slices.Sort(exempt) + want := []string{OpHealth} + if !slices.Equal(exempt, want) { + t.Errorf("wireRevisionExempt routes = %v, want exactly {OpHealth}", exempt) + } +} + +// TestCapabilitiesAdvertiseIssuesListSort: a client learns this server +// accepts GET /v0/beads/issues' sort parameter from the capability list, +// never from the version string. +func TestCapabilitiesAdvertiseIssuesListSort(t *testing.T) { + ts := newTestServer(t, Config{}) + + caps, _ := decodeBody(t, ts.get(t, "/v0/beads/context"))["capabilities"].([]any) + var got []string + for _, c := range caps { + got = append(got, c.(string)) + } + if !slices.Contains(got, CapIssuesListSort) { + t.Errorf("capabilities %v do not advertise %q", got, CapIssuesListSort) + } +} diff --git a/internal/httpapi/wireshape/cmd/gendigest/main.go b/internal/httpapi/wireshape/cmd/gendigest/main.go new file mode 100644 index 000000000..f93499cd1 --- /dev/null +++ b/internal/httpapi/wireshape/cmd/gendigest/main.go @@ -0,0 +1,49 @@ +// Command gendigest regenerates internal/httpapi/wireshape/testdata/golden.json +// from the embedded OpenAPI document and the Go CurrentWireRevision constant. +// +// Run it after a deliberate, revision-bumped wire-shape change — never to make +// a failing TestWireShapeDigest pass on an accidental one. A diff this command +// produces that only adds entries is additive and needs no revision bump; a +// diff that changes or removes an entry without a CurrentWireRevision bump is +// the exact drift the test exists to catch, not something to paper over by +// running this. +package main + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "runtime" + + "github.com/steveyegge/beads/internal/httpapi" + "github.com/steveyegge/beads/internal/httpapi/wireshape" +) + +func main() { + digest, err := wireshape.Compute(httpapi.CurrentWireRevision) + if err != nil { + fmt.Fprintln(os.Stderr, "gendigest:", err) + os.Exit(1) + } + blob, err := json.MarshalIndent(digest, "", " ") + if err != nil { + fmt.Fprintln(os.Stderr, "gendigest:", err) + os.Exit(1) + } + blob = append(blob, '\n') + + // Resolve relative to this source file so the command works from any cwd, + // the same way `go generate` directives in this repo do. + _, thisFile, _, ok := runtime.Caller(0) + if !ok { + fmt.Fprintln(os.Stderr, "gendigest: could not resolve own source path") + os.Exit(1) + } + out := filepath.Join(filepath.Dir(thisFile), "..", "..", "testdata", "golden.json") + if err := os.WriteFile(out, blob, 0o600); err != nil { + fmt.Fprintln(os.Stderr, "gendigest:", err) + os.Exit(1) + } + fmt.Println("wrote", out) +} diff --git a/internal/httpapi/wireshape/doc.go b/internal/httpapi/wireshape/doc.go new file mode 100644 index 000000000..a7fb078bc --- /dev/null +++ b/internal/httpapi/wireshape/doc.go @@ -0,0 +1,21 @@ +// Package wireshape is the CI drift gate for ContextResponse.wire_revision. +// +// It computes a digest of every member reachable from any response schema in +// internal/httpapi/spec/openapi.v0.yaml — walking $refs through +// components.schemas and components.responses — recording, per member, the +// (schema, member, type, format, enum, required) tuple the document promises. +// TestWireShapeDigest compares that digest against the committed golden in +// testdata/golden.json and fails on any difference: a member added is fine (an +// additive change never bumps wire_revision), but a member's type, format, +// enum vocabulary, or required-ness changing — or a member disappearing — is +// exactly the class of change internal/httpapi/wire_revision.go's +// CurrentWireRevision exists to gate, and the golden's own +// "wire_revision" field is compared too, so a shape change recorded against +// the OLD revision number still fails. +// +// Regenerate the golden after a deliberate, revision-bumped change with: +// +// go run ./internal/httpapi/wireshape/cmd/gendigest +// +// Never hand-edit testdata/golden.json. +package wireshape diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json new file mode 100644 index 000000000..432dda1f5 --- /dev/null +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -0,0 +1,3050 @@ +{ + "wire_revision": 2, + "entries": [ + { + "schema": "AddDependenciesResponse", + "member": "added", + "type": "array", + "required": true + }, + { + "schema": "AnchorEdgeCount", + "member": "count", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "AnchorEdgeCount", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "AnchorEdgeCount", + "member": "missing", + "type": "boolean", + "required": true + }, + { + "schema": "ApplyBatchResponse", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "ApplyBatchResponse", + "member": "keys", + "type": "object", + "required": true + }, + { + "schema": "ApplyItemResult", + "member": "changed", + "type": "boolean", + "required": true + }, + { + "schema": "ApplyItemResult", + "member": "depends_on_id", + "type": "string", + "required": false + }, + { + "schema": "ApplyItemResult", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "ApplyItemResult", + "member": "kind", + "type": "string", + "enum": [ + "close", + "create", + "dep_add", + "update" + ], + "required": true + }, + { + "schema": "ApplyItemResult", + "member": "revision", + "type": "string", + "required": true + }, + { + "schema": "BatchCloseResponse", + "member": "outcomes", + "type": "array", + "required": true + }, + { + "schema": "BatchCreateResponse", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "Blocker", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "Blocker", + "member": "kind", + "type": "string", + "required": true + }, + { + "schema": "Blocker", + "member": "type", + "type": "string", + "required": false + }, + { + "schema": "BlockingAnnotations", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "BondRef", + "member": "bond_point", + "type": "string", + "required": false + }, + { + "schema": "BondRef", + "member": "bond_type", + "type": "string", + "required": true + }, + { + "schema": "BondRef", + "member": "source_id", + "type": "string", + "required": true + }, + { + "schema": "ClaimNextResponse", + "member": "claimed", + "type": "object", + "required": false + }, + { + "schema": "ClaimResponse", + "member": "already_claimed", + "type": "boolean", + "required": true + }, + { + "schema": "ClaimResponse", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "CloseIssueResponse", + "member": "already_closed", + "type": "boolean", + "required": true + }, + { + "schema": "CloseIssueResponse", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "CloseIssueResponse", + "member": "open_children", + "type": "integer", + "required": true + }, + { + "schema": "CloseIssueResponse", + "member": "revision", + "type": "string", + "required": true + }, + { + "schema": "CloseOutcome", + "member": "already_closed", + "type": "boolean", + "required": false + }, + { + "schema": "CloseOutcome", + "member": "blockers", + "type": "array", + "required": false + }, + { + "schema": "CloseOutcome", + "member": "code", + "type": "string", + "required": false + }, + { + "schema": "CloseOutcome", + "member": "detail", + "type": "string", + "required": false + }, + { + "schema": "CloseOutcome", + "member": "issue", + "type": "object", + "required": false + }, + { + "schema": "CloseOutcome", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "CloseOutcome", + "member": "open_children", + "type": "integer", + "required": false + }, + { + "schema": "Comment", + "member": "author", + "type": "string", + "required": true + }, + { + "schema": "Comment", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "Comment", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "Comment", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "Comment", + "member": "text", + "type": "string", + "required": true + }, + { + "schema": "CompareAndSetMetadataResponse", + "member": "current", + "type": "", + "required": false + }, + { + "schema": "CompareAndSetMetadataResponse", + "member": "swapped", + "type": "boolean", + "required": true + }, + { + "schema": "ContextResponse", + "member": "api_version", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "backend", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "bd_version", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "beads_dir", + "type": "string", + "required": false + }, + { + "schema": "ContextResponse", + "member": "capabilities", + "type": "array", + "required": true + }, + { + "schema": "ContextResponse", + "member": "database", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "dolt_mode", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "min_client_wire_revision", + "type": "integer", + "required": true + }, + { + "schema": "ContextResponse", + "member": "project_id", + "type": "string", + "required": true + }, + { + "schema": "ContextResponse", + "member": "repo_root", + "type": "string", + "required": false + }, + { + "schema": "ContextResponse", + "member": "schema_version", + "type": "integer", + "required": true + }, + { + "schema": "ContextResponse", + "member": "wire_revision", + "type": "integer", + "required": true + }, + { + "schema": "Cycle", + "member": "members", + "type": "array", + "required": true + }, + { + "schema": "Cycle", + "member": "partial", + "type": "boolean", + "required": true + }, + { + "schema": "CycleMember", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "CycleMember", + "member": "issue", + "type": "object", + "required": false + }, + { + "schema": "CyclesPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "CyclesPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "deleted", + "type": "integer", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "dependencies", + "type": "integer", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "dry_run", + "type": "boolean", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "events", + "type": "integer", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "labels", + "type": "integer", + "required": true + }, + { + "schema": "DeleteIssuesResult", + "member": "orphaned", + "type": "array", + "required": false + }, + { + "schema": "DeleteIssuesResult", + "member": "references_updated", + "type": "integer", + "required": true + }, + { + "schema": "Dependency", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "Dependency", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "Dependency", + "member": "depends_on_id", + "type": "string", + "required": true + }, + { + "schema": "Dependency", + "member": "id", + "type": "string", + "required": false + }, + { + "schema": "Dependency", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "Dependency", + "member": "metadata", + "type": "string", + "required": false + }, + { + "schema": "Dependency", + "member": "thread_id", + "type": "string", + "required": false + }, + { + "schema": "Dependency", + "member": "type", + "type": "string", + "required": true + }, + { + "schema": "DependencyEdge", + "member": "depends_on_id", + "type": "string", + "required": true + }, + { + "schema": "DependencyEdge", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "DependencyEdge", + "member": "type", + "type": "string", + "required": true + }, + { + "schema": "DependencyEdges", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "DependencyEdges", + "member": "missing", + "type": "array", + "required": true + }, + { + "schema": "DependencyTreePage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "DependencyTreePage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "EdgeCounts", + "member": "anchors", + "type": "array", + "required": true + }, + { + "schema": "EventRecord", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "EventRecord", + "member": "comment", + "type": "object", + "required": false + }, + { + "schema": "EventRecord", + "member": "dep", + "type": "object", + "required": false + }, + { + "schema": "EventRecord", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "EventRecord", + "member": "issue_id", + "type": "string", + "required": true + }, + { + "schema": "EventRecord", + "member": "op", + "type": "string", + "required": true + }, + { + "schema": "EventRecord", + "member": "seq", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "EventRecord", + "member": "ts", + "type": "string", + "required": true + }, + { + "schema": "EventsPage", + "member": "head", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "EventsPage", + "member": "records", + "type": "array", + "required": true + }, + { + "schema": "Health", + "member": "status", + "type": "string", + "enum": [ + "ok" + ], + "required": true + }, + { + "schema": "Issue", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "await_id", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "await_type", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "bonded_from", + "type": "array", + "required": false + }, + { + "schema": "Issue", + "member": "close_reason", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "closed_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "closed_by_session", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "comments", + "type": "array", + "required": false + }, + { + "schema": "Issue", + "member": "compacted_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "compacted_at_commit", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "compaction_level", + "type": "integer", + "required": false + }, + { + "schema": "Issue", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "Issue", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "dependencies", + "type": "array", + "required": false + }, + { + "schema": "Issue", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "Issue", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "Issue", + "member": "event_kind", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "heartbeat_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "Issue", + "member": "is_blocked", + "type": "boolean", + "required": false + }, + { + "schema": "Issue", + "member": "is_template", + "type": "boolean", + "required": false + }, + { + "schema": "Issue", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "labels", + "type": "array", + "required": false + }, + { + "schema": "Issue", + "member": "lease_expires_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "lease_granted_node", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "Issue", + "member": "mol_type", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "Issue", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "original_size", + "type": "integer", + "required": false + }, + { + "schema": "Issue", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "payload", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "pinned", + "type": "boolean", + "required": false + }, + { + "schema": "Issue", + "member": "priority", + "type": "integer", + "required": true + }, + { + "schema": "Issue", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "source_formula", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "source_location", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "source_system", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "spec_id", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "started_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "Issue", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "storage_class", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "target", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "timeout", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "Issue", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "Issue", + "member": "updated_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "Issue", + "member": "waiters", + "type": "array", + "required": false + }, + { + "schema": "Issue", + "member": "wisp_type", + "type": "string", + "required": false + }, + { + "schema": "Issue", + "member": "work_type", + "type": "string", + "required": false + }, + { + "schema": "IssueBlocking", + "member": "blocked_by", + "type": "array", + "required": true + }, + { + "schema": "IssueBlocking", + "member": "blocks", + "type": "array", + "required": true + }, + { + "schema": "IssueBlocking", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "IssueBlocking", + "member": "parent", + "type": "string", + "required": false + }, + { + "schema": "IssueCount", + "member": "groups", + "type": "object", + "required": false + }, + { + "schema": "IssueCount", + "member": "total", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "IssueDetails", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "await_id", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "await_type", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "bonded_from", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "close_reason", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "closed_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "closed_by_session", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "comment_count", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "comments", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "comments_omitted", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "compacted_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "compacted_at_commit", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "compaction_level", + "type": "integer", + "required": false + }, + { + "schema": "IssueDetails", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueDetails", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "dependencies", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "dependency_count", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "dependent_count", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "dependents", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "epic_closeable", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "epic_closed_children", + "type": "integer", + "required": false + }, + { + "schema": "IssueDetails", + "member": "epic_total_children", + "type": "integer", + "required": false + }, + { + "schema": "IssueDetails", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "IssueDetails", + "member": "event_kind", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "heartbeat_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "IssueDetails", + "member": "is_blocked", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "is_template", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "labels", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "lease_expires_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "lease_granted_node", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "IssueDetails", + "member": "mol_type", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "original_size", + "type": "integer", + "required": false + }, + { + "schema": "IssueDetails", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "parent", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "payload", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "pinned", + "type": "boolean", + "required": false + }, + { + "schema": "IssueDetails", + "member": "priority", + "type": "integer", + "required": true + }, + { + "schema": "IssueDetails", + "member": "revision", + "type": "string", + "required": true + }, + { + "schema": "IssueDetails", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "source_formula", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "source_location", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "source_system", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "spec_id", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "started_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueDetails", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "storage_class", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "target", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "timeout", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "IssueDetails", + "member": "unresolvable_dependencies", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "unresolvable_dependents", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueDetails", + "member": "updated_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueDetails", + "member": "waiters", + "type": "array", + "required": false + }, + { + "schema": "IssueDetails", + "member": "wisp_type", + "type": "string", + "required": false + }, + { + "schema": "IssueDetails", + "member": "work_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "await_id", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "await_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "bonded_from", + "type": "array", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "close_reason", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "closed_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "closed_by_session", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "comment_count", + "type": "integer", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "comments", + "type": "array", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "comments_omitted", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "compacted_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "compacted_at_commit", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "compaction_level", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "dependencies", + "type": "array", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "dependency_count", + "type": "integer", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "dependent_count", + "type": "integer", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "event_kind", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "heartbeat_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "is_blocked", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "is_template", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "labels", + "type": "array", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "lease_expires_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "lease_granted_node", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "mol_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "original_size", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "parent", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "payload", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "pinned", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "priority", + "type": "integer", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "source_formula", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "source_location", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "source_system", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "spec_id", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "started_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "storage_class", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "target", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "timeout", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "updated_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueWithCounts", + "member": "waiters", + "type": "array", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "wisp_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithCounts", + "member": "work_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "await_id", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "await_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "bonded_from", + "type": "array", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "close_reason", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "closed_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "closed_by_session", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "comments", + "type": "array", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "compacted_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "compacted_at_commit", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "compaction_level", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "dependencies", + "type": "array", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "dependency_type", + "type": "string", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "event_kind", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "heartbeat_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "is_blocked", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "is_template", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "labels", + "type": "array", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "lease_expires_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "lease_granted_node", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "mol_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "original_size", + "type": "integer", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "payload", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "pinned", + "type": "boolean", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "priority", + "type": "integer", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "source_formula", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "source_location", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "source_system", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "spec_id", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "started_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "storage_class", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "target", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "timeout", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "updated_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "waiters", + "type": "array", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "wisp_type", + "type": "string", + "required": false + }, + { + "schema": "IssueWithDependencyMetadata", + "member": "work_type", + "type": "string", + "required": false + }, + { + "schema": "IssuesPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "IssuesPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "IssuesPage", + "member": "next_cursor", + "type": "string", + "required": false + }, + { + "schema": "MemoriesPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "MemoriesPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "MemoriesPage", + "member": "next_cursor", + "type": "string", + "required": false + }, + { + "schema": "Memory", + "member": "key", + "type": "string", + "required": true + }, + { + "schema": "Memory", + "member": "value", + "type": "string", + "required": true + }, + { + "schema": "Problem", + "member": "actual_assignee", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "actual_status", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "actual_version", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "blocker_id", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "blocker_is_ancestor", + "type": "boolean", + "required": false + }, + { + "schema": "Problem", + "member": "blockers", + "type": "array", + "required": false + }, + { + "schema": "Problem", + "member": "code", + "type": "string", + "required": true + }, + { + "schema": "Problem", + "member": "declared_later", + "type": "boolean", + "required": false + }, + { + "schema": "Problem", + "member": "detail", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "existing_type", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "expected_assignee", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "expected_status", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "floor", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "Problem", + "member": "head", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "Problem", + "member": "issue_id", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "issue_status", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "item_index", + "type": "integer", + "required": false + }, + { + "schema": "Problem", + "member": "item_issue_id", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "item_key", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "item_kind", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "min_wire_revision", + "type": "integer", + "required": false + }, + { + "schema": "Problem", + "member": "open_children", + "type": "integer", + "required": false + }, + { + "schema": "Problem", + "member": "param", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "reason", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "request_id", + "type": "string", + "required": true + }, + { + "schema": "Problem", + "member": "requested_type", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "server_project_id", + "type": "string", + "required": false + }, + { + "schema": "Problem", + "member": "since", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "Problem", + "member": "status", + "type": "integer", + "required": true + }, + { + "schema": "Problem", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "Problem", + "member": "type", + "type": "string", + "format": "uri", + "required": false + }, + { + "schema": "QueryPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "QueryPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "ReadyCount", + "member": "total", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "ReadyPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "ReadyPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "RelatedIssues", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "ReleaseIssueResponse", + "member": "changed", + "type": "boolean", + "required": true + }, + { + "schema": "ReleaseIssueResponse", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "ReleaseIssueResponse", + "member": "revision", + "type": "string", + "required": true + }, + { + "schema": "RememberedMemory", + "member": "key", + "type": "string", + "required": true + }, + { + "schema": "RememberedMemory", + "member": "replaced", + "type": "boolean", + "required": true + }, + { + "schema": "RememberedMemory", + "member": "value", + "type": "string", + "required": true + }, + { + "schema": "RemoveDependencyResponse", + "member": "removed", + "type": "boolean", + "required": true + }, + { + "schema": "RemovedSetting", + "member": "key", + "type": "string", + "required": true + }, + { + "schema": "ReopenIssueResponse", + "member": "already_open", + "type": "boolean", + "required": true + }, + { + "schema": "ReopenIssueResponse", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "ReopenIssueResponse", + "member": "revision", + "type": "string", + "required": true + }, + { + "schema": "Setting", + "member": "key", + "type": "string", + "required": true + }, + { + "schema": "Setting", + "member": "redacted", + "type": "boolean", + "required": true + }, + { + "schema": "Setting", + "member": "value", + "type": "string", + "required": false + }, + { + "schema": "SettingsPage", + "member": "has_more", + "type": "boolean", + "required": true + }, + { + "schema": "SettingsPage", + "member": "items", + "type": "array", + "required": true + }, + { + "schema": "SettingsPage", + "member": "next_cursor", + "type": "string", + "required": false + }, + { + "schema": "Statistics", + "member": "average_lead_time_hours", + "type": "number", + "required": true + }, + { + "schema": "Statistics", + "member": "blocked_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "closed_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "deferred_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "epics_eligible_for_closure", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "gate_issues", + "type": "integer", + "required": false + }, + { + "schema": "Statistics", + "member": "in_progress_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "open_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "pinned_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "ready_issues", + "type": "integer", + "required": true + }, + { + "schema": "Statistics", + "member": "template_issues", + "type": "integer", + "required": false + }, + { + "schema": "Statistics", + "member": "total_issues", + "type": "integer", + "required": true + }, + { + "schema": "StatsResponse", + "member": "blocked_count_skipped", + "type": "boolean", + "required": true + }, + { + "schema": "StatsResponse", + "member": "summary", + "type": "object", + "required": true + }, + { + "schema": "SweepResult", + "member": "dependencies", + "type": "integer", + "required": true + }, + { + "schema": "SweepResult", + "member": "dry_run", + "type": "boolean", + "required": true + }, + { + "schema": "SweepResult", + "member": "events", + "type": "integer", + "required": true + }, + { + "schema": "SweepResult", + "member": "labels", + "type": "integer", + "required": true + }, + { + "schema": "SweepResult", + "member": "referenced_ids", + "type": "array", + "required": false + }, + { + "schema": "SweepResult", + "member": "skipped", + "type": "object", + "required": true + }, + { + "schema": "SweepResult", + "member": "swept", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "closed_at_or_after_cutoff", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "not_closed", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "pinned", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "referenced", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "unknown_closed_at", + "type": "integer", + "required": true + }, + { + "schema": "SweepSkips", + "member": "unreadable", + "type": "integer", + "required": true + }, + { + "schema": "TreeNode", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "await_id", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "await_type", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "bonded_from", + "type": "array", + "required": false + }, + { + "schema": "TreeNode", + "member": "close_reason", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "closed_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "closed_by_session", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "comments", + "type": "array", + "required": false + }, + { + "schema": "TreeNode", + "member": "compacted_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "compacted_at_commit", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "compaction_level", + "type": "integer", + "required": false + }, + { + "schema": "TreeNode", + "member": "created_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "TreeNode", + "member": "created_by", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "dependencies", + "type": "array", + "required": false + }, + { + "schema": "TreeNode", + "member": "depth", + "type": "integer", + "required": true + }, + { + "schema": "TreeNode", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "edge_from_parent", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "TreeNode", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "TreeNode", + "member": "event_kind", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "heartbeat_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "TreeNode", + "member": "is_blocked", + "type": "boolean", + "required": false + }, + { + "schema": "TreeNode", + "member": "is_template", + "type": "boolean", + "required": false + }, + { + "schema": "TreeNode", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "labels", + "type": "array", + "required": false + }, + { + "schema": "TreeNode", + "member": "lease_expires_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "lease_granted_node", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "TreeNode", + "member": "mol_type", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "TreeNode", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "original_size", + "type": "integer", + "required": false + }, + { + "schema": "TreeNode", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "parent_id", + "type": "string", + "required": true + }, + { + "schema": "TreeNode", + "member": "payload", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "pinned", + "type": "boolean", + "required": false + }, + { + "schema": "TreeNode", + "member": "priority", + "type": "integer", + "required": true + }, + { + "schema": "TreeNode", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "source_formula", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "source_location", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "source_system", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "spec_id", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "started_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "TreeNode", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "storage_class", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "target", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "timeout", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "TreeNode", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "TreeNode", + "member": "truncated", + "type": "boolean", + "required": true + }, + { + "schema": "TreeNode", + "member": "updated_at", + "type": "string", + "format": "date-time", + "required": true + }, + { + "schema": "TreeNode", + "member": "waiters", + "type": "array", + "required": false + }, + { + "schema": "TreeNode", + "member": "wisp_type", + "type": "string", + "required": false + }, + { + "schema": "TreeNode", + "member": "work_type", + "type": "string", + "required": false + }, + { + "schema": "UpdateIssueResponse", + "member": "changed", + "type": "boolean", + "required": true + }, + { + "schema": "UpdateIssueResponse", + "member": "issue", + "type": "object", + "required": true + }, + { + "schema": "UpdateIssueResponse", + "member": "revision", + "type": "string", + "required": true + } + ] +} diff --git a/internal/httpapi/wireshape/wireshape.go b/internal/httpapi/wireshape/wireshape.go new file mode 100644 index 000000000..e50fc7b7a --- /dev/null +++ b/internal/httpapi/wireshape/wireshape.go @@ -0,0 +1,285 @@ +package wireshape + +import ( + "fmt" + "sort" + "strings" + + "gopkg.in/yaml.v3" + + "github.com/steveyegge/beads/internal/httpapi/spec" +) + +// Entry is one member of one schema reachable from an operation's response. +// Schema is the component's name (or, for a member whose own value is an +// inline object or array with no $ref of its own, a synthetic dotted path +// rooted at the nearest named schema) so that two members with the same name +// on different schemas are never confused with one another. +type Entry struct { + Schema string `json:"schema"` + Member string `json:"member"` + Type string `json:"type"` + Format string `json:"format,omitempty"` + Enum []string `json:"enum,omitempty"` + Required bool `json:"required"` +} + +// Digest is the golden document: the wire revision it was computed against, +// plus every member entry, sorted for a stable diff. +type Digest struct { + WireRevision int `json:"wire_revision"` + Entries []Entry `json:"entries"` +} + +var httpVerbs = map[string]bool{ + "get": true, "put": true, "post": true, "delete": true, "patch": true, + "head": true, "options": true, "trace": true, +} + +// Compute walks the embedded OpenAPI document and builds the digest for the +// current revision. wireRevision is passed in rather than read from a +// constant here so this package carries no dependency on internal/httpapi +// (which would otherwise be a cycle: internal/httpapi imports nothing from +// here, and this package must stay that way to run from the standalone +// gendigest command without pulling in the server). +func Compute(wireRevision int) (Digest, error) { + var doc map[string]any + if err := yaml.Unmarshal(spec.OpenAPIV0(), &doc); err != nil { + return Digest{}, fmt.Errorf("parse embedded openapi document: %w", err) + } + + c := &collector{doc: doc, visited: map[string]bool{}, byKey: map[string]Entry{}} + + paths, _ := doc["paths"].(map[string]any) + for _, item := range paths { + methods, ok := item.(map[string]any) + if !ok { + continue + } + for method, raw := range methods { + if !httpVerbs[strings.ToLower(method)] { + continue + } + op, ok := raw.(map[string]any) + if !ok { + continue + } + responses, _ := op["responses"].(map[string]any) + for _, rawResp := range responses { + respMap, ok := rawResp.(map[string]any) + if !ok { + continue + } + respMap = c.resolveAny(respMap) + content, _ := respMap["content"].(map[string]any) + for _, rawMedia := range content { + media, ok := rawMedia.(map[string]any) + if !ok { + continue + } + schemaNode, ok := media["schema"].(map[string]any) + if !ok { + continue + } + c.walk(schemaNode, "") + } + } + } + } + + entries := make([]Entry, 0, len(c.byKey)) + for _, e := range c.byKey { + entries = append(entries, e) + } + sort.Slice(entries, func(i, j int) bool { + if entries[i].Schema != entries[j].Schema { + return entries[i].Schema < entries[j].Schema + } + return entries[i].Member < entries[j].Member + }) + return Digest{WireRevision: wireRevision, Entries: entries}, nil +} + +type collector struct { + doc map[string]any + visited map[string]bool + byKey map[string]Entry // "schema\x00member" -> Entry +} + +// lookup resolves one local $ref to the node it names. It returns nil for +// anything this document does not use (a remote ref, or a path with no node). +func lookup(doc map[string]any, ref string) map[string]any { + rest := strings.TrimPrefix(ref, "#/") + var cur any = doc + for _, part := range strings.Split(rest, "/") { + m, ok := cur.(map[string]any) + if !ok { + return nil + } + cur = m[part] + } + m, _ := cur.(map[string]any) + return m +} + +// schemaRefName reports the component schema name a $ref names, or "" when +// the ref points somewhere else (components.responses, for instance). +func schemaRefName(ref string) string { + const prefix = "#/components/schemas/" + name, ok := strings.CutPrefix(ref, prefix) + if !ok { + return "" + } + return name +} + +// resolveAny follows a $ref chain to its concrete node, however many hops it +// takes (a response $ref lands on a response object; it is never itself a +// further $ref in this document, but the walk tolerates it either way). +func (c *collector) resolveAny(node map[string]any) map[string]any { + seen := 0 + for { + ref, ok := node["$ref"].(string) + if !ok { + return node + } + next := lookup(c.doc, ref) + if next == nil { + return node + } + node = next + seen++ + if seen > 32 { + // Defensive only: this document has no ref cycle, and 32 hops is + // far beyond anything a hand-written spec would ever chain. + return node + } + } +} + +func toStringSlice(v any) []string { + raw, ok := v.([]any) + if !ok { + return nil + } + out := make([]string, 0, len(raw)) + for _, e := range raw { + out = append(out, fmt.Sprint(e)) + } + sort.Strings(out) + return out +} + +func asString(v any) string { + s, _ := v.(string) + return s +} + +// walk records every property of the schema node reaches (following $refs and +// flattening allOf), then recurses into each property's own object or array +// shape. label names an anonymous (non-$ref) node for entries recorded under +// it; a $ref always overrides it with the component's own name. +// +// Every NAMED schema is walked at most once (the visited guard), which is +// what makes a cycle between named schemas (and there is at least the +// potential for one as this document grows) a no-op rather than a stack +// overflow, and what makes the digest the same whichever operation reaches a +// shared schema first. +func (c *collector) walk(node map[string]any, label string) { + name := label + for { + ref, ok := node["$ref"].(string) + if !ok { + break + } + if n := schemaRefName(ref); n != "" { + name = n + } + next := lookup(c.doc, ref) + if next == nil { + return + } + node = next + } + + if allOf, ok := node["allOf"].([]any); ok { + mergedProps := map[string]any{} + var mergedRequired []any + for _, rawSub := range allOf { + subMap, ok := rawSub.(map[string]any) + if !ok { + continue + } + sub := c.resolveAny(subMap) + if props, ok := sub["properties"].(map[string]any); ok { + for k, v := range props { + mergedProps[k] = v + } + } + if req, ok := sub["required"].([]any); ok { + mergedRequired = append(mergedRequired, req...) + } + } + node = map[string]any{ + "type": "object", + "properties": mergedProps, + "required": mergedRequired, + } + } + + if name != "" { + if c.visited[name] { + return + } + c.visited[name] = true + } + + if props, ok := node["properties"].(map[string]any); ok { + required := map[string]bool{} + for _, r := range toStringSlice(node["required"]) { + required[r] = true + } + members := make([]string, 0, len(props)) + for m := range props { + members = append(members, m) + } + sort.Strings(members) + for _, member := range members { + propRaw := props[member] + propMap, ok := propRaw.(map[string]any) + if !ok { + continue + } + resolved := c.resolveAny(propMap) + entry := Entry{ + Schema: name, + Member: member, + Type: asString(resolved["type"]), + Format: asString(resolved["format"]), + Enum: toStringSlice(resolved["enum"]), + Required: required[member], + } + c.byKey[entry.Schema+"\x00"+entry.Member] = entry + + switch entry.Type { + case "object": + c.walk(propMap, name+"."+member) + case "array": + if items, ok := resolved["items"].(map[string]any); ok { + c.walk(items, name+"."+member+"[]") + } + } + } + return + } + + if asString(node["type"]) == "array" { + if items, ok := node["items"].(map[string]any); ok { + itemLabel := name + if itemLabel == "" { + itemLabel = label + } + c.walk(items, itemLabel+"[]") + } + } +} diff --git a/internal/httpapi/wireshape/wireshape_test.go b/internal/httpapi/wireshape/wireshape_test.go new file mode 100644 index 000000000..1d8245621 --- /dev/null +++ b/internal/httpapi/wireshape/wireshape_test.go @@ -0,0 +1,104 @@ +package wireshape_test + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "testing" + + "github.com/steveyegge/beads/internal/httpapi" + "github.com/steveyegge/beads/internal/httpapi/wireshape" +) + +// TestWireShapeDigest is the drift gate ContextResponse.wire_revision +// documents: it fails the moment the shape of any EXISTING response member +// changes without CurrentWireRevision increasing to match. +// +// It is deliberately two separate comparisons rather than one struct-equal, +// because the two ways this can go red call for different instructions. A +// changed or removed entry at the SAME wire_revision is drift nobody signed +// off on — regenerate only after bumping CurrentWireRevision (see +// internal/httpapi/wire_revision.go) and the `wire_revision` property's +// revision table in openapi.v0.yaml, never before. A changed wire_revision +// with an unchanged entry set (or only additions) means this golden is simply +// stale — regenerate it with: +// +// go run ./internal/httpapi/wireshape/cmd/gendigest +func TestWireShapeDigest(t *testing.T) { + golden := loadGolden(t) + got, err := wireshape.Compute(httpapi.CurrentWireRevision) + if err != nil { + t.Fatalf("compute digest: %v", err) + } + + if len(got.Entries) < 20 { + // The document describes dozens of response members; a near-empty + // digest means the walk found nothing, which would make every + // assertion below pass vacuously. + t.Fatalf("digest has %d entries, want the full walk", len(got.Entries)) + } + + wantByKey := map[string]wireshape.Entry{} + for _, e := range golden.Entries { + wantByKey[e.Schema+"\x00"+e.Member] = e + } + gotByKey := map[string]wireshape.Entry{} + for _, e := range got.Entries { + gotByKey[e.Schema+"\x00"+e.Member] = e + } + + var changed, removed, added []string + for key, want := range wantByKey { + got, ok := gotByKey[key] + if !ok { + removed = append(removed, key) + continue + } + if !reflect.DeepEqual(want, got) { + changed = append(changed, key) + } + } + for key := range gotByKey { + if _, ok := wantByKey[key]; !ok { + added = append(added, key) + } + } + + if len(changed) > 0 || len(removed) > 0 { + if got.WireRevision <= golden.WireRevision { + t.Errorf("response member shape changed without a wire_revision bump: changed=%v removed=%v\n"+ + "bump CurrentWireRevision (internal/httpapi/wire_revision.go) and the revision table "+ + "in openapi.v0.yaml's `wire_revision` property, THEN regenerate the golden with "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest`", changed, removed) + } else { + t.Errorf("response member shape changed (changed=%v removed=%v) and wire_revision moved "+ + "%d -> %d, but the golden was not regenerated: run "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", + changed, removed, golden.WireRevision, got.WireRevision) + } + } else if len(added) > 0 && got.WireRevision == golden.WireRevision { + t.Errorf("response members were added (%v) but the golden was not regenerated: "+ + "this is additive and needs no wire_revision bump, but still run "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", added) + } + + if got.WireRevision != golden.WireRevision && len(changed) == 0 && len(removed) == 0 && len(added) == 0 { + t.Errorf("CurrentWireRevision is %d but the golden still says %d, with no shape change to justify "+ + "either: regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`", + got.WireRevision, golden.WireRevision) + } +} + +func loadGolden(t *testing.T) wireshape.Digest { + t.Helper() + blob, err := os.ReadFile(filepath.Join("testdata", "golden.json")) + if err != nil { + t.Fatalf("read golden: %v", err) + } + var d wireshape.Digest + if err := json.Unmarshal(blob, &d); err != nil { + t.Fatalf("decode golden: %v", err) + } + return d +} From f62986731115f99334bbd75883b252b14793a971 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:42 +0000 Subject: [PATCH 02/16] fix(httpapi): rename issues.listSort to issues.list.sort, harden token rule Opus review of 12c5144ca (HIGH-1, MEDIUM-5): - Rename the capability token issues.listSort -> issues.list.sort to match bd-enterprise's client (httpstore/wire/handshake.go:121, DESIGN.txt S4e), and widen capabilityToken to accept an optional third segment. - untokenedParameters() now requires the extracted token to be one Capabilities() actually serves, not just backtick-shaped text. This catches real false positives that existed in the spec already (workapi.DefaultReadyLimit, status.custom) which only shape-matched the old regex; they're added to the pretoken baseline as genuinely pre-existing untokened parameters. - Extend the rule to request body members (new testdata/pretoken_request_body_members.json baseline) and exempt brand-new operations via a frozen testdata/pretoken_operations.json snapshot. - Add TestPretokenBaselinesNeverGrow pinning all three baseline sizes so a future change can't hand-add an exemption instead of a token. - Update spec prose for the widened rule scope. go test ./internal/httpapi/... and the reviewer's pmut.py all pass as expected: plain/bogus modes go RED, reuse (real-but-unrelated token) stays green, matching the review's literal ask ("a token must be one advertised by Capabilities()"). Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 2 +- engdocs/design/bd-serve-v0.md | 2 +- internal/httpapi/apigen/types.gen.go | 6 +- internal/httpapi/routes.go | 2 +- internal/httpapi/server_test.go | 2 +- internal/httpapi/spec/openapi.v0.yaml | 42 +- internal/httpapi/spec_parity_test.go | 372 +++++++++++++++--- .../httpapi/testdata/pretoken_operations.json | 43 ++ .../httpapi/testdata/pretoken_parameters.json | 13 + .../pretoken_request_body_members.json | 123 ++++++ 10 files changed, 539 insertions(+), 68 deletions(-) create mode 100644 internal/httpapi/testdata/pretoken_operations.json create mode 100644 internal/httpapi/testdata/pretoken_request_body_members.json diff --git a/CHANGELOG.md b/CHANGELOG.md index c03a88781..0a7f1a8cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,7 +37,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `internal/httpapi/wire_revision.go` and the `wire_revision` property in `internal/httpapi/spec/openapi.v0.yaml` for the full revision history. - `GET /v0/beads/issues`'s `sort` parameter now advertises the - `issues.listSort` capability token, the same way an operation itself + `issues.list.sort` capability token, the same way an operation itself does, and a CI rule (`TestNewParameterOnExistingOperationHasABehaviorToken`) requires every new parameter on an existing operation to carry one. diff --git a/engdocs/design/bd-serve-v0.md b/engdocs/design/bd-serve-v0.md index b0bd0204e..66584d184 100644 --- a/engdocs/design/bd-serve-v0.md +++ b/engdocs/design/bd-serve-v0.md @@ -595,7 +595,7 @@ and the `openapi.v0.yaml` history entry it belongs beside. `resource.verb` token for a new OPERATION; nothing required one for a new PARAMETER on an EXISTING operation, so a client had no single place to learn that, say, `GET /v0/beads/issues`'s `sort` parameter had started accepting a -value. `issues.listSort` is the first deliberate use of the same convention +value. `issues.list.sort` is the first deliberate use of the same convention one level down: the token lives in the parameter's own description, exactly as `capabilities` members carry theirs, enforced by `TestNewParameterOnExistingOperationHasABehaviorToken` against a frozen diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 573d43fcd..a5981b360 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1010,7 +1010,7 @@ type ContextResponse struct { // OPTIONAL, and absent means only that this server does not disclose its filesystem layout — never that it has no workspace. A client MUST NOT require it, MUST NOT treat absence as an error, and has no use for the value beyond display: it is a path on the SERVER's filesystem, which the client cannot open. Identify the workspace by `project_id` and `database`, which are required. BeadsDir *string `json:"beads_dir,omitempty"` - // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.listSort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. + // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. // // THIS LIST IS BUILD-LEVEL, NOT WORKSPACE-LEVEL. It says which operations this binary serves, and for every entry but two that is the whole answer. `events.list` and `events.watch` are the exceptions: the durable events journal is a per-workspace setting that is OFF by default, so a server that advertises them may still refuse every request to both with 409 `events_journal_disabled` — correctly, because the operations exist and the workspace has no journal. A consumer of either MUST treat the capability as "this server speaks it" and the 409 as "not on this workspace", and must not read the capability as a promise that records will arrive. Capabilities []string `json:"capabilities"` @@ -1057,7 +1057,7 @@ type ContextResponse struct { // // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member's (schema, member, type, format, enum, required) fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass. // - // Every request PARAMETER (query, path or header — not a request body member) added to an EXISTING operation after this document must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against a frozen baseline (`internal/httpapi/testdata/pretoken_parameters.json`) of every parameter this document already had the day the rule was written. A parameter on that baseline is grandfathered and never needs one; any parameter not on it — present now or added later — must carry a token today. The baseline is written once and never regenerated: only a pre-existing, untokened parameter ever belongs on it, and a newly tokened parameter satisfies the rule without needing an entry at all. `issues.listSort` is the first deliberate use of the rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. + // Every request PARAMETER (query, path or header) and every request BODY MEMBER, added to an EXISTING operation after this document, must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against two frozen baselines (`internal/httpapi/testdata/pretoken_parameters.json` for parameters, `pretoken_request_body_members.json` for body members) of every parameter and body member this document already had the day each rule was written. A member on its baseline is grandfathered and never needs a token; any member not on it — present now or added later, on an operation that already existed when the relevant baseline was frozen — must carry one today. A BRAND-NEW operation (one absent from the third frozen file, `pretoken_operations.json`) is exempt entirely: its own review decides its capability story, not a baseline written before it existed. The token itself must also be one `Capabilities()` actually serves — a backticked string merely shaped like a token proves nothing on its own. All three baselines are written once and never regenerated: only a pre-existing, untokened member or operation ever belongs on them, a newly tokened member satisfies the rule without needing an entry at all, and `TestPretokenBaselinesNeverGrow` pins their sizes so nothing can hand-add a new exemption instead of a token. `issues.list.sort` is the first deliberate use of the parameter rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. WireRevision int `json:"wire_revision"` } @@ -2230,7 +2230,7 @@ type ListIssuesParams struct { // // THE VOCABULARY IS CLOSED, and deliberately smaller than the nine values `bd list --sort` and `GET /v0/beads/issues:query` take. Each value here is a cursor contract, not a display preference: it needs a keyset predicate and a key proven total. See the operation description for why the other seven have neither. // - // A value outside the enum is a 400 `invalid_argument` with `param: "sort"` and `reason: "invalid_value"`. A server that predates this parameter answers `param: "sort"` with `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in `created` order and sorting client-side. This server also advertises the `issues.listSort` token in `ContextResponse.capabilities` for exactly that check, so a client with capabilities cached from the handshake never has to pay for the round trip an old server's `unknown_parameter` answer costs. + // A value outside the enum is a 400 `invalid_argument` with `param: "sort"` and `reason: "invalid_value"`. A server that predates this parameter answers `param: "sort"` with `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in `created` order and sorting client-side. This server also advertises the `issues.list.sort` token in `ContextResponse.capabilities` for exactly that check, so a client with capabilities cached from the handshake never has to pay for the round trip an old server's `unknown_parameter` answer costs. Sort *ListIssuesParamsSort `form:"sort,omitempty" json:"sort,omitempty"` // Cursor Opaque keyset position, taken verbatim from a previous response's `next_cursor`. Clients MUST NOT construct, parse or mutate it: its encoding is server-private and versioned, and an undecodable or unknown-version value is refused with 400 `invalid_cursor`. The recovery for that refusal is normative: restart paging with no `cursor` at all — the position cannot be salvaged, and re-sending the same value cannot succeed. diff --git a/internal/httpapi/routes.go b/internal/httpapi/routes.go index a56723172..cb7a84b76 100644 --- a/internal/httpapi/routes.go +++ b/internal/httpapi/routes.go @@ -74,7 +74,7 @@ const CapBatchApplyLarge = "issues.batchApplyLarge" // `sort` predates that rule and would have failed it, so it is the parity fix // — ent already advertises the equivalent token — that the rule's own // baseline is built from. -const CapIssuesListSort = "issues.listSort" +const CapIssuesListSort = "issues.list.sort" // customMethodTarget splits the custom method off the segment the router // matched, and reports the row that claims it. diff --git a/internal/httpapi/server_test.go b/internal/httpapi/server_test.go index 1b1dff570..0b5799395 100644 --- a/internal/httpapi/server_test.go +++ b/internal/httpapi/server_test.go @@ -635,7 +635,7 @@ func TestCapabilitiesAdvertiseEveryImplementedOperation(t *testing.T) { // them; project.enforce and issues.batchApplyLarge are advertised in the // same list. Both are spelled literally here to keep this an independent // oracle rather than a second call to the code under test. - want = append(want, "project.enforce", "issues.batchApplyLarge", "issues.listSort") + want = append(want, "project.enforce", "issues.batchApplyLarge", "issues.list.sort") slices.Sort(want) if !slices.Equal(got, want) { t.Errorf("capabilities = %v, want %v", got, want) diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index ca0cb8bef..e42fdbc5c 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -1313,7 +1313,7 @@ paths: `reason: "unknown_parameter"` instead, which is the per-parameter capability probe a client can dispatch on to fall back to paging in `created` order and sorting client-side. This server also - advertises the `issues.listSort` token in + advertises the `issues.list.sort` token in `ContextResponse.capabilities` for exactly that check, so a client with capabilities cached from the handshake never has to pay for the round trip an old server's `unknown_parameter` answer costs. @@ -7507,7 +7507,7 @@ components: server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an - older server would refuse anyway, and `issues.listSort`, which + older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`. The @@ -7576,23 +7576,33 @@ components: never frozen in place) but needs no revision bump to pass. - Every request PARAMETER (query, path or header — not a request - body member) added to an EXISTING operation after this document + Every request PARAMETER (query, path or header) and every request + BODY MEMBER, added to an EXISTING operation after this document, must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it - against a frozen baseline - (`internal/httpapi/testdata/pretoken_parameters.json`) of every - parameter this document already had the day the rule was written. - A parameter on that baseline is grandfathered and never needs one; - any parameter not on it — present now or added later — must carry - a token today. The baseline is written once and never - regenerated: only a pre-existing, untokened parameter ever - belongs on it, and a newly tokened parameter satisfies the rule - without needing an entry at all. `issues.listSort` is the first - deliberate use of the rule; it also closes the one gap the rule - was written to catch, since `sort` itself shipped (#5666) with no - token at all. + against two frozen baselines + (`internal/httpapi/testdata/pretoken_parameters.json` for + parameters, `pretoken_request_body_members.json` for body + members) of every parameter and body member this document already + had the day each rule was written. A member on its baseline is + grandfathered and never needs a token; any member not on it — + present now or added later, on an operation that already existed + when the relevant baseline was frozen — must carry one today. A + BRAND-NEW operation (one absent from the third frozen file, + `pretoken_operations.json`) is exempt entirely: its own review + decides its capability story, not a baseline written before it + existed. The token itself must also be one `Capabilities()` + actually serves — a backticked string merely shaped like a token + proves nothing on its own. All three baselines are written once + and never regenerated: only a pre-existing, untokened member or + operation ever belongs on them, a newly tokened member satisfies + the rule without needing an entry at all, and + `TestPretokenBaselinesNeverGrow` pins their sizes so nothing can + hand-add a new exemption instead of a token. `issues.list.sort` is + the first deliberate use of the parameter rule; it also closes the + one gap the rule was written to catch, since `sort` itself shipped + (#5666) with no token at all. min_client_wire_revision: type: integer description: >- diff --git a/internal/httpapi/spec_parity_test.go b/internal/httpapi/spec_parity_test.go index 075894866..b299f2b02 100644 --- a/internal/httpapi/spec_parity_test.go +++ b/internal/httpapi/spec_parity_test.go @@ -421,11 +421,13 @@ func TestSpecDefaultsMatchSharedConstants(t *testing.T) { } // capabilityToken matches one backticked capability token in the -// `capabilities` description. The shape — a lowercase resource, a dot, a -// lowerCamel verb — is what every token on this surface is, and it is narrow -// enough that no other backticked word in that paragraph can be mistaken for -// one. -var capabilityToken = regexp.MustCompile("`([a-z]+\\.[a-zA-Z]+)`") +// `capabilities` description. The shape is a lowercase resource, a dot, a +// lowerCamel verb, and OPTIONALLY a second dot and a third lowerCamel segment +// (`issues.list.sort`, matching the bd-enterprise client's +// httpstore/wire/handshake.go:121 and DESIGN.txt §4e) — narrow enough that no +// other backticked word in that paragraph can be mistaken for one, but wide +// enough for a sub-capability one level below an operation's own token. +var capabilityToken = regexp.MustCompile("`([a-z]+\\.[a-zA-Z]+(?:\\.[a-zA-Z]+)?)`") // TestSpecCapabilityVocabularyMatchesTheRouteTable is the gate for the ONE // piece of per-operation plumbing that had no gate. @@ -472,17 +474,57 @@ func TestSpecCapabilityVocabularyMatchesTheRouteTable(t *testing.T) { } } -// untokenedParameters reports "operationId.paramName" for every parameter in -// ops whose own description carries no capabilityToken and whose -// (operationId, paramName) pair is not in baseline. doc resolves the $refs a -// parameter may point at (components.parameters); baseline is keyed by -// operationId to the list of its parameter names exempt from the rule. +// hasValidCapabilityToken reports whether desc carries at least one +// backticked token that is BOTH shaped like a capability (capabilityToken) +// AND a member of validTokens. A backticked string that merely looks like a +// token — `metadata.json`, say — proves nothing: anyone can put backticks +// around two words. Checking membership against the real advertised set is +// what makes the token verifiable rather than decorative (review HIGH, +// "a token must be one advertised by Capabilities(), not any backticked a.b +// text"). +func hasValidCapabilityToken(desc string, validTokens map[string]bool) bool { + for _, m := range capabilityToken.FindAllStringSubmatch(desc, -1) { + if validTokens[m[1]] { + return true + } + } + return false +} + +// untokenedParameters reports "operationId.paramName" (query/path/header +// parameters) and "operationId body.memberName" (request body members, one +// level deep) for every undocumented dispatch surface in ops: one whose own +// description carries no token that is a member of validTokens, and whose +// (operationId, name) pair is not in baseline. +// +// Two further gates narrow this to a true ratchet rather than a sweep that +// would also fire on surface this rule did not exist to judge: +// +// - existingOps restricts the rule to operations that existed when the rule +// was frozen (testdata/pretoken_operations.json). A parameter or body +// member on a BRAND-NEW operation is never "missing" a token under this +// rule — a new operation's own review is where its capability story gets +// decided, not a baseline file written before it existed. +// - validTokens restricts what counts as a token at all: shape is not +// enough (hasValidCapabilityToken). +// +// doc resolves the $refs a parameter or request body may point at. baseline +// is keyed by operationId to the list of its parameter names exempt from the +// rule; bodyBaseline is the same shape for "body.member" names. // // Factored out of TestNewParameterOnExistingOperationHasABehaviorToken so // TestUntokenedParameterRuleFires can falsify it against a fixture with no // dependency on the real document: the rule fires on its own fabricated // miss, not just on whatever this spec currently happens to contain. -func untokenedParameters(t *testing.T, doc map[string]any, ops map[string]specOp, baseline map[string][]string) []string { +func untokenedParameters( + t *testing.T, + doc map[string]any, + ops map[string]specOp, + baseline map[string][]string, + bodyBaseline map[string][]string, + existingOps map[string]bool, + validTokens map[string]bool, +) []string { t.Helper() grandfathered := map[string]bool{} for opID, params := range baseline { @@ -490,28 +532,46 @@ func untokenedParameters(t *testing.T, doc map[string]any, ops map[string]specOp grandfathered[opID+"\x00"+p] = true } } + bodyGrandfathered := map[string]bool{} + for opID, members := range bodyBaseline { + for _, m := range members { + bodyGrandfathered[opID+"\x00"+m] = true + } + } + var missing []string for opID, so := range ops { - raw, ok := so.op["parameters"].([]any) - if !ok { - continue + if existingOps != nil && !existingOps[opID] { + continue // brand-new operation: its own review decides its tokens. } - for _, r := range raw { - pm, ok := r.(map[string]any) - if !ok { - t.Fatalf("%s: parameter is %T, want a mapping", opID, r) + + if raw, ok := so.op["parameters"].([]any); ok { + for _, r := range raw { + pm, ok := r.(map[string]any) + if !ok { + t.Fatalf("%s: parameter is %T, want a mapping", opID, r) + } + pm = resolveRef(t, doc, pm) + name, _ := pm["name"].(string) + if name == "" { + t.Fatalf("%s: parameter with no name", opID) + } + if grandfathered[opID+"\x00"+name] { + continue + } + desc, _ := pm["description"].(string) + if !hasValidCapabilityToken(desc, validTokens) { + missing = append(missing, opID+"."+name) + } } - pm = resolveRef(t, doc, pm) - name, _ := pm["name"].(string) - if name == "" { - t.Fatalf("%s: parameter with no name", opID) - } - if grandfathered[opID+"\x00"+name] { + } + + for member, desc := range requestBodyMemberDescriptions(t, doc, so) { + if bodyGrandfathered[opID+"\x00"+member] { continue } - desc, _ := pm["description"].(string) - if len(capabilityToken.FindAllString(desc, -1)) == 0 { - missing = append(missing, opID+"."+name) + if !hasValidCapabilityToken(desc, validTokens) { + missing = append(missing, opID+" body."+member) } } } @@ -519,17 +579,109 @@ func untokenedParameters(t *testing.T, doc map[string]any, ops map[string]specOp return missing } +// requestBodyMemberDescriptions returns, for so's request body (if any), each +// top-level member name mapped to its own description. Every request body in +// this document is `required: true` with a single application/json media +// type whose schema is a direct $ref to a named object schema with no allOf — +// simple enough that one level of property-walking covers all of them; a +// future body shaped any other way will simply report no members here rather +// than silently miscounting, which TestRequestBodyMembersCoversEveryBody (if +// that ever becomes necessary) would be the place to pin. +func requestBodyMemberDescriptions(t *testing.T, doc map[string]any, so specOp) map[string]string { + t.Helper() + rb, ok := so.op["requestBody"].(map[string]any) + if !ok { + return nil + } + content, ok := rb["content"].(map[string]any) + if !ok { + return nil + } + media, ok := content["application/json"].(map[string]any) + if !ok { + return nil + } + schemaRef, ok := media["schema"].(map[string]any) + if !ok { + return nil + } + schema := resolveRef(t, doc, schemaRef) + props, ok := schema["properties"].(map[string]any) + if !ok { + return nil + } + out := map[string]string{} + for member, raw := range props { + pm, ok := raw.(map[string]any) + if !ok { + continue + } + pm = resolveRef(t, doc, pm) + desc, _ := pm["description"].(string) + out[member] = desc + } + return out +} + func loadPretokenBaseline(t *testing.T) map[string][]string { t.Helper() - blob, err := os.ReadFile(filepath.Join("testdata", "pretoken_parameters.json")) + return loadJSONStringMap(t, "pretoken_parameters.json") +} + +// loadPretokenBodyBaseline is loadPretokenBaseline's sibling for request body +// members (review MEDIUM, "extend the rule to request-body members on +// existing operations"): same write-once-grandfather shape, keyed by +// operationId to the "body.member" names that already existed, untokened, +// the day body-member coverage was added. +func loadPretokenBodyBaseline(t *testing.T) map[string][]string { + t.Helper() + return loadJSONStringMap(t, "pretoken_request_body_members.json") +} + +func loadJSONStringMap(t *testing.T, filename string) map[string][]string { + t.Helper() + blob, err := os.ReadFile(filepath.Join("testdata", filename)) if err != nil { - t.Fatalf("read pretoken baseline: %v", err) + t.Fatalf("read %s: %v", filename, err) } - var baseline map[string][]string - if err := json.Unmarshal(blob, &baseline); err != nil { - t.Fatalf("decode %s: %v", "testdata/pretoken_parameters.json", err) + var out map[string][]string + if err := json.Unmarshal(blob, &out); err != nil { + t.Fatalf("decode testdata/%s: %v", filename, err) } - return baseline + return out +} + +// loadPretokenOperations is the review's "don't fire on brand-new +// operations" gate: the set of operationIds that existed the day the +// untokened-parameter rule (and its body-member extension) was frozen. An +// operation not in this file is new, and gets no say from either baseline — +// its own code review decides its capability story. +func loadPretokenOperations(t *testing.T) map[string]bool { + t.Helper() + blob, err := os.ReadFile(filepath.Join("testdata", "pretoken_operations.json")) + if err != nil { + t.Fatalf("read pretoken operations: %v", err) + } + var list []string + if err := json.Unmarshal(blob, &list); err != nil { + t.Fatalf("decode testdata/pretoken_operations.json: %v", err) + } + out := make(map[string]bool, len(list)) + for _, id := range list { + out[id] = true + } + return out +} + +// servedCapabilities is validTokens for the real document: exactly what +// Capabilities() advertises right now, which is also what +// TestSpecCapabilityVocabularyMatchesTheRouteTable pins the spec prose to. +func servedCapabilities() map[string]bool { + out := map[string]bool{} + for _, token := range Capabilities() { + out[token] = true + } + return out } // TestNewParameterOnExistingOperationHasABehaviorToken is the lint rule S0 @@ -537,7 +689,7 @@ func loadPretokenBaseline(t *testing.T) map[string][]string { // token in its own description is undocumented dispatch surface — a client // has no way to learn THIS server accepts it without sending it and risking // `unknown_parameter`, which is exactly what happened to `sort` (#5666) -// before `issues.listSort` closed the gap. +// before `issues.list.sort` closed the gap. // // The baseline (testdata/pretoken_parameters.json) is a WRITE-ONCE grandfather // list: every parameter this document already had the day the rule was @@ -545,26 +697,72 @@ func loadPretokenBaseline(t *testing.T) map[string][]string { // parameter on it is exempt permanently, and nothing here ever regenerates // the file. Any parameter not on it — present today or added by any later // change — must carry a token, which is what makes this a ratchet rather than -// a one-time sweep: `issues.listSort` is already the parity fix the rule's +// a one-time sweep: `issues.list.sort` is already the parity fix the rule's // own baseline is built from, not an exemption to it. func TestNewParameterOnExistingOperationHasABehaviorToken(t *testing.T) { doc := loadSpec(t) ops := specOps(t, doc) baseline := loadPretokenBaseline(t) + bodyBaseline := loadPretokenBodyBaseline(t) + existingOps := loadPretokenOperations(t) + valid := servedCapabilities() - if missing := untokenedParameters(t, doc, ops, baseline); len(missing) > 0 { - t.Errorf("parameter(s) on an existing operation carry no capability token and are not on the "+ - "frozen baseline (internal/httpapi/testdata/pretoken_parameters.json): %v\n"+ - "add a `resource.verb` token to the parameter's own description, the way `sort` on "+ - "GET /v0/beads/issues advertises `issues.listSort`", missing) + if missing := untokenedParameters(t, doc, ops, baseline, bodyBaseline, existingOps, valid); len(missing) > 0 { + t.Errorf("parameter(s) or request body member(s) on an existing operation carry no capability token "+ + "that Capabilities() actually serves, and are not on the frozen baseline "+ + "(internal/httpapi/testdata/pretoken_parameters.json / pretoken_request_body_members.json): %v\n"+ + "add a `resource.verb` token to the member's own description, the way `sort` on "+ + "GET /v0/beads/issues advertises `issues.list.sort`", missing) + } +} + +// TestPretokenBaselinesNeverGrow pins the three frozen testdata files at +// their current size (review MEDIUM, "the baseline must never grow"): +// nothing here ever regenerates them, so the only way their counts change is +// someone hand-editing a new exemption in, which is exactly what this guards +// against. A legitimate new parameter or body member must carry its own +// token, not an entry on these files; a legitimate new OPERATION is added to +// pretoken_operations.json never — new operations are simply never members of +// it, by construction, since nothing regenerates it either. +func TestPretokenBaselinesNeverGrow(t *testing.T) { + const ( + wantParamEntries = 137 + wantBodyEntries = 85 + wantOps = 41 + ) + baseline := loadPretokenBaseline(t) + paramEntries := 0 + for _, params := range baseline { + paramEntries += len(params) + } + if paramEntries != wantParamEntries { + t.Errorf("testdata/pretoken_parameters.json has %d grandfathered parameters, want exactly %d "+ + "(frozen the day the rule was written); if this moved, something added a new exemption instead "+ + "of a token", paramEntries, wantParamEntries) + } + + bodyBaseline := loadPretokenBodyBaseline(t) + bodyEntries := 0 + for _, members := range bodyBaseline { + bodyEntries += len(members) + } + if bodyEntries != wantBodyEntries { + t.Errorf("testdata/pretoken_request_body_members.json has %d grandfathered body members, want exactly %d", + bodyEntries, wantBodyEntries) + } + + existingOps := loadPretokenOperations(t) + if len(existingOps) != wantOps { + t.Errorf("testdata/pretoken_operations.json has %d operations, want exactly %d "+ + "(frozen the day the new-operation exemption was added)", len(existingOps), wantOps) } } // TestUntokenedParameterRuleFires is the rule's own falsification, required // because TestNewParameterOnExistingOperationHasABehaviorToken is permanently // green against a spec written to satisfy it: a synthetic operation proves -// the CHECK itself still distinguishes the three cases it exists to tell -// apart, independent of anything the real document currently says. +// the CHECK itself still distinguishes the cases it exists to tell apart, +// independent of anything the real document currently says. func TestUntokenedParameterRuleFires(t *testing.T) { doc := map[string]any{} // no $refs in the fixtures below; resolveRef never consults it newOp := func(desc string) specOp { @@ -578,20 +776,104 @@ func TestUntokenedParameterRuleFires(t *testing.T) { }, } } + noBody := map[string][]string{} + allExisting := map[string]bool{"syntheticOp": true} + validTokens := map[string]bool{"synthetic.widget": true, "issues.list": true} ops := map[string]specOp{"syntheticOp": newOp("plain prose, no token at all")} - if got := untokenedParameters(t, doc, ops, map[string][]string{}); !slices.Equal(got, []string{"syntheticOp.widget"}) { + if got := untokenedParameters(t, doc, ops, noBody, noBody, allExisting, validTokens); !slices.Equal(got, []string{"syntheticOp.widget"}) { t.Fatalf("untokened, non-grandfathered parameter: got %v, want [syntheticOp.widget]", got) } - if got := untokenedParameters(t, doc, ops, map[string][]string{"syntheticOp": {"widget"}}); len(got) != 0 { + if got := untokenedParameters(t, doc, ops, map[string][]string{"syntheticOp": {"widget"}}, noBody, allExisting, validTokens); len(got) != 0 { t.Fatalf("grandfathered parameter still reported: got %v, want none", got) } ops["syntheticOp"] = newOp("carries the `synthetic.widget` token") - if got := untokenedParameters(t, doc, ops, map[string][]string{}); len(got) != 0 { + if got := untokenedParameters(t, doc, ops, noBody, noBody, allExisting, validTokens); len(got) != 0 { t.Fatalf("tokened parameter still reported: got %v, want none", got) } + + // A backticked string shaped like a token but not one Capabilities() + // actually serves must still fire: shape alone is not proof (review HIGH). + ops["syntheticOp"] = newOp("see `metadata.json` for details") + if got := untokenedParameters(t, doc, ops, noBody, noBody, allExisting, validTokens); !slices.Equal(got, []string{"syntheticOp.widget"}) { + t.Fatalf("bogus (unserved) token: got %v, want [syntheticOp.widget]", got) + } + + // A REAL token, just not one this parameter actually advertises, passes: + // Capabilities() has no per-parameter binding to check against, so + // membership in the served set is the full extent of this rule (review + // HIGH says "a token must be one advertised by Capabilities()", not that + // it must be the specific token for this parameter). + ops["syntheticOp"] = newOp("reuses an unrelated real token (`issues.list`)") + if got := untokenedParameters(t, doc, ops, noBody, noBody, allExisting, validTokens); len(got) != 0 { + t.Fatalf("real-but-unrelated token: got %v, want none", got) + } + + // A brand-new operation (absent from the existingOps baseline) is exempt + // even with no token at all (review MEDIUM, "don't fire on brand-new + // operations"). syntheticOp still carries its real-but-unrelated token + // from the previous case, so it reports nothing either. + ops["brandNewOp"] = newOp("plain prose, no token at all") + if got := untokenedParameters(t, doc, ops, noBody, noBody, allExisting, validTokens); len(got) != 0 { + t.Fatalf("brand-new operation must not be reported: got %v", got) + } + // And a nil existingOps (the behavior with no freeze gate at all) does + // catch it, so the gate is additive, not a silent replacement for the + // baseline check. + if got := untokenedParameters(t, doc, ops, noBody, noBody, nil, validTokens); !slices.Equal(got, []string{"brandNewOp.widget"}) { + t.Fatalf("nil existingOps must not exempt anything: got %v", got) + } +} + +// TestUntokenedRequestBodyMemberRuleFires is TestUntokenedParameterRuleFires' +// sibling for request body members (review MEDIUM, "extend the rule to +// request-body members on existing operations"). +func TestUntokenedRequestBodyMemberRuleFires(t *testing.T) { + doc := map[string]any{ + "components": map[string]any{ + "schemas": map[string]any{ + "SyntheticRequest": map[string]any{ + "type": "object", + "properties": map[string]any{ + "widget": map[string]any{"type": "string", "description": "plain prose, no token at all"}, + }, + }, + }, + }, + } + so := specOp{ + method: "POST", + path: "/synthetic", + op: map[string]any{ + "requestBody": map[string]any{ + "content": map[string]any{ + "application/json": map[string]any{ + "schema": map[string]any{"$ref": "#/components/schemas/SyntheticRequest"}, + }, + }, + }, + }, + } + ops := map[string]specOp{"syntheticOp": so} + noParams := map[string][]string{} + allExisting := map[string]bool{"syntheticOp": true} + validTokens := map[string]bool{"synthetic.widget": true} + + if got := untokenedParameters(t, doc, ops, noParams, noParams, allExisting, validTokens); !slices.Equal(got, []string{"syntheticOp body.widget"}) { + t.Fatalf("untokened, non-grandfathered body member: got %v, want [syntheticOp body.widget]", got) + } + + if got := untokenedParameters(t, doc, ops, noParams, map[string][]string{"syntheticOp": {"widget"}}, allExisting, validTokens); len(got) != 0 { + t.Fatalf("grandfathered body member still reported: got %v, want none", got) + } + + props := doc["components"].(map[string]any)["schemas"].(map[string]any)["SyntheticRequest"].(map[string]any)["properties"].(map[string]any) + props["widget"] = map[string]any{"type": "string", "description": "carries the `synthetic.widget` token"} + if got := untokenedParameters(t, doc, ops, noParams, noParams, allExisting, validTokens); len(got) != 0 { + t.Fatalf("tokened body member still reported: got %v, want none", got) + } } func specParam(t *testing.T, so specOp, name string) map[string]any { diff --git a/internal/httpapi/testdata/pretoken_operations.json b/internal/httpapi/testdata/pretoken_operations.json new file mode 100644 index 000000000..5ce49890a --- /dev/null +++ b/internal/httpapi/testdata/pretoken_operations.json @@ -0,0 +1,43 @@ +[ + "addComment", + "addDependencies", + "applyBatch", + "batchCloseIssues", + "batchCreateIssues", + "claimIssue", + "claimNextIssue", + "closeIssue", + "compareAndSetMetadata", + "countDependencyEdges", + "countIssues", + "countReadyWork", + "createIssue", + "deleteIssues", + "forgetMemory", + "getContext", + "getDependencyTree", + "getIssue", + "getMemory", + "getSetting", + "getStats", + "health", + "listBlockingAnnotations", + "listDependencies", + "listDependencyCycles", + "listEvents", + "listIssues", + "listMemories", + "listReadyWork", + "listRelatedIssues", + "listSettings", + "queryIssues", + "releaseIssue", + "rememberMemory", + "removeDependency", + "reopenIssue", + "setSetting", + "sweepIssues", + "unsetSetting", + "updateIssue", + "watchEvents" +] \ No newline at end of file diff --git a/internal/httpapi/testdata/pretoken_parameters.json b/internal/httpapi/testdata/pretoken_parameters.json index 06fc860e5..b39bc4ed7 100644 --- a/internal/httpapi/testdata/pretoken_parameters.json +++ b/internal/httpapi/testdata/pretoken_parameters.json @@ -99,6 +99,9 @@ "getMemory": [ "key" ], + "getSetting": [ + "key" + ], "getStats": [ "assignee", "skip_blocked" @@ -129,6 +132,7 @@ "include_templates", "label", "label_any", + "limit", "metadata_field", "parent", "status", @@ -149,6 +153,7 @@ "label_any", "label_pattern", "label_regex", + "limit", "metadata_field", "parent", "priority", @@ -157,11 +162,13 @@ "unassigned" ], "listRelatedIssues": [ + "direction", "id", "type" ], "queryIssues": [ "all", + "limit", "q", "reverse", "sort" @@ -172,6 +179,12 @@ "reopenIssue": [ "id" ], + "setSetting": [ + "key" + ], + "unsetSetting": [ + "key" + ], "updateIssue": [ "id" ], diff --git a/internal/httpapi/testdata/pretoken_request_body_members.json b/internal/httpapi/testdata/pretoken_request_body_members.json new file mode 100644 index 000000000..3eb1a95cd --- /dev/null +++ b/internal/httpapi/testdata/pretoken_request_body_members.json @@ -0,0 +1,123 @@ +{ + "addComment": [ + "author", + "text" + ], + "addDependencies": [ + "actor", + "edges" + ], + "applyBatch": [ + "actor", + "force_id_prefix", + "items", + "provenance", + "skip_per_edge_cycle_check" + ], + "batchCloseIssues": [ + "actor", + "force", + "items", + "session" + ], + "batchCreateIssues": [ + "actor", + "items" + ], + "claimIssue": [ + "actor" + ], + "claimNextIssue": [ + "actor" + ], + "closeIssue": [ + "actor", + "expected_version", + "force", + "reason", + "session" + ], + "compareAndSetMetadata": [ + "actor", + "expected", + "key", + "value" + ], + "createIssue": [ + "acceptance_criteria", + "actor", + "assignee", + "defer_until", + "dependencies", + "description", + "design", + "due_at", + "ephemeral", + "estimated_minutes", + "external_ref", + "force_id_prefix", + "id", + "inherit_labels_from_parent", + "issue_type", + "labels", + "metadata", + "no_history", + "notes", + "owner", + "parent_id", + "priority", + "sender", + "status", + "title", + "waits_for" + ], + "deleteIssues": [ + "actor", + "cascade", + "dry_run", + "expected_version", + "force", + "ids" + ], + "releaseIssue": [ + "actor", + "expected_assignee", + "force" + ], + "rememberMemory": [ + "content", + "key" + ], + "removeDependency": [ + "actor", + "depends_on_id", + "issue_id" + ], + "reopenIssue": [ + "actor", + "expected_version", + "reason" + ], + "setSetting": [ + "value" + ], + "sweepIssues": [ + "actor", + "closed_before", + "dry_run", + "pattern", + "protect_referenced", + "tier" + ], + "updateIssue": [ + "actor", + "claim", + "expected_assignee", + "expected_status", + "expected_version", + "force_assignee_transfer", + "force_close_policy", + "force_notes_overwrite", + "patch" + ] +} \ No newline at end of file From f9e1ceda52c1d6350681398b4b08ccac54d34261 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:43 +0000 Subject: [PATCH 03/16] fix(wireshape): close four digest-coverage gaps, add gendigest write guard Opus review of 12c5144ca (HIGH-2, MEDIUM-4). The digest only tracked Type/Format/Enum/Required on direct properties, missing: - nullable: true (never read at all) - array item shape (Issue.labels items string->integer was invisible; only object-shaped items got recursed into) - additionalProperties value shape (IssueCount.groups) - request bodies entirely (Compute() only walked responses, so DeleteIssuesRequest.expected_version string->integer, the other half of #6053, was invisible) Also added defensive oneOf flattening alongside the existing allOf merge (dormant today -- grep confirms this spec uses oneOf nowhere -- but no longer silently invisible if that changes). Extracted the golden-vs-computed comparison into wireshape.Compare(), shared by TestWireShapeDigest and the new wireshape.SafeToWrite(), which gendigest now calls before writing: it refuses to overwrite golden.json with changed/removed entries unless the candidate's wire_revision is strictly greater than the existing golden's. Pure additions always write. TestSafeToWrite exercises both guard directions. golden.json is regenerated. This single regen is a one-time exception to the new guard, done by writing the digest directly rather than through gendigest: confirmed via a throwaway diff (not checked in) that every one of the 57 "changed" entries differs only in the NEW fields the wider walk now populates (ItemType/ItemFormat/Nullable/AdditionalProps*) -- every pre-existing Type/Format/Required/Enum value is byte-identical. Nothing on the wire changed; the measurement got more precise. Bumping CurrentWireRevision for this would have written a false "the protocol changed here" entry into the revision history table. Going forward, any REAL change the guard now reports uses the normal bump-then-regenerate path, same as before. Verified against the reviewer's mut.py: nested_type, nullable, nullable_issue, array_items, addlprops, request_body and format all go RED on TestWireShapeDigest, each reverted after confirming. Co-Authored-By: Claude Sonnet 5 --- .../httpapi/wireshape/cmd/gendigest/main.go | 33 +- .../httpapi/wireshape/testdata/golden.json | 1376 ++++++++++++++++- internal/httpapi/wireshape/wireshape.go | 269 +++- internal/httpapi/wireshape/wireshape_test.go | 101 +- 4 files changed, 1648 insertions(+), 131 deletions(-) diff --git a/internal/httpapi/wireshape/cmd/gendigest/main.go b/internal/httpapi/wireshape/cmd/gendigest/main.go index f93499cd1..80c2dd089 100644 --- a/internal/httpapi/wireshape/cmd/gendigest/main.go +++ b/internal/httpapi/wireshape/cmd/gendigest/main.go @@ -26,12 +26,6 @@ func main() { fmt.Fprintln(os.Stderr, "gendigest:", err) os.Exit(1) } - blob, err := json.MarshalIndent(digest, "", " ") - if err != nil { - fmt.Fprintln(os.Stderr, "gendigest:", err) - os.Exit(1) - } - blob = append(blob, '\n') // Resolve relative to this source file so the command works from any cwd, // the same way `go generate` directives in this repo do. @@ -41,6 +35,33 @@ func main() { os.Exit(1) } out := filepath.Join(filepath.Dir(thisFile), "..", "..", "testdata", "golden.json") + + // Refuse to overwrite a golden that would silently launder a failing + // TestWireShapeDigest: a changed or removed entry must come with a + // CurrentWireRevision bump ABOVE what's already committed, never from + // just re-running this command. A purely additive diff, or no existing + // golden at all (first run), always writes. + // #nosec G304 -- out is this command's own source-relative output path, + // never request- or argv-influenced; reading it back is the write guard. + if existing, err := os.ReadFile(out); err == nil { + var golden wireshape.Digest + if err := json.Unmarshal(existing, &golden); err != nil { + fmt.Fprintln(os.Stderr, "gendigest: decode existing golden:", err) + os.Exit(1) + } + if ok, reason := wireshape.SafeToWrite(golden, digest); !ok { + fmt.Fprintln(os.Stderr, "gendigest:", reason) + os.Exit(1) + } + } + + blob, err := json.MarshalIndent(digest, "", " ") + if err != nil { + fmt.Fprintln(os.Stderr, "gendigest:", err) + os.Exit(1) + } + blob = append(blob, '\n') + if err := os.WriteFile(out, blob, 0o600); err != nil { fmt.Fprintln(os.Stderr, "gendigest:", err) os.Exit(1) diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index 432dda1f5..942647f7f 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -1,11 +1,37 @@ { "wire_revision": 2, "entries": [ + { + "schema": "AddCommentRequest", + "member": "author", + "type": "string", + "required": true + }, + { + "schema": "AddCommentRequest", + "member": "text", + "type": "string", + "required": true + }, + { + "schema": "AddDependenciesRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "AddDependenciesRequest", + "member": "edges", + "type": "array", + "required": true, + "item_type": "object" + }, { "schema": "AddDependenciesResponse", "member": "added", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "AnchorEdgeCount", @@ -26,18 +52,279 @@ "type": "boolean", "required": true }, + { + "schema": "ApplyBatchRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "ApplyBatchRequest", + "member": "force_id_prefix", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyBatchRequest", + "member": "items", + "type": "array", + "required": true, + "item_type": "object" + }, + { + "schema": "ApplyBatchRequest", + "member": "provenance", + "type": "string", + "required": false + }, + { + "schema": "ApplyBatchRequest", + "member": "skip_per_edge_cycle_check", + "type": "boolean", + "required": false + }, { "schema": "ApplyBatchResponse", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "ApplyBatchResponse", "member": "keys", "type": "object", + "required": true, + "additional_props_type": "string" + }, + { + "schema": "ApplyCloseItem", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "ApplyCloseItem", + "member": "force", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyCloseItem", + "member": "reason", + "type": "string", + "required": false + }, + { + "schema": "ApplyCloseItem", + "member": "session", + "type": "string", + "required": false + }, + { + "schema": "ApplyCloseItem", + "member": "target", + "type": "object", "required": true }, + { + "schema": "ApplyCreateItem", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "id", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "key", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "ApplyCreateItem", + "member": "metadata", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "ApplyCreateItem", + "member": "metadata_refs", + "type": "object", + "required": false, + "additional_props_type": "object" + }, + { + "schema": "ApplyCreateItem", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "priority", + "type": "integer", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "ApplyCreateItem", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "ApplyDepAddItem", + "member": "metadata", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "ApplyDepAddItem", + "member": "source", + "type": "object", + "required": true + }, + { + "schema": "ApplyDepAddItem", + "member": "target", + "type": "object", + "required": true + }, + { + "schema": "ApplyDepAddItem", + "member": "type", + "type": "string", + "required": true + }, + { + "schema": "ApplyItem", + "member": "close", + "type": "object", + "required": false + }, + { + "schema": "ApplyItem", + "member": "create", + "type": "object", + "required": false + }, + { + "schema": "ApplyItem", + "member": "dep_add", + "type": "object", + "required": false + }, + { + "schema": "ApplyItem", + "member": "kind", + "type": "string", + "enum": [ + "close", + "create", + "dep_add", + "update" + ], + "required": true + }, + { + "schema": "ApplyItem", + "member": "update", + "type": "object", + "required": false + }, { "schema": "ApplyItemResult", "member": "changed", @@ -74,17 +361,341 @@ "type": "string", "required": true }, + { + "schema": "ApplyLabelPatch", + "member": "add", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "ApplyLabelPatch", + "member": "remove", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "ApplyLabelPatch", + "member": "replace", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "ApplyMetadataPatch", + "member": "merge", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "ApplyMetadataPatch", + "member": "replace", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "ApplyMetadataPatch", + "member": "set", + "type": "object", + "required": false + }, + { + "schema": "ApplyMetadataPatch", + "member": "unset", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "ApplyPatchBody", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "append_notes", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false, + "nullable": true + }, + { + "schema": "ApplyPatchBody", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false, + "nullable": true + }, + { + "schema": "ApplyPatchBody", + "member": "estimated_minutes", + "type": "integer", + "required": false, + "nullable": true + }, + { + "schema": "ApplyPatchBody", + "member": "external_ref", + "type": "string", + "required": false, + "nullable": true + }, + { + "schema": "ApplyPatchBody", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "labels", + "type": "object", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "priority", + "type": "integer", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "ApplyPatchBody", + "member": "title", + "type": "string", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "expected_assignee", + "type": "string", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "expected_status", + "type": "string", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "force_assignee_transfer", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "force_close_policy", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "force_notes_overwrite", + "type": "boolean", + "required": false + }, + { + "schema": "ApplyUpdateItem", + "member": "patch", + "type": "object", + "required": true + }, + { + "schema": "ApplyUpdateItem", + "member": "target", + "type": "object", + "required": true + }, + { + "schema": "BatchCloseItem", + "member": "id", + "type": "string", + "required": true + }, + { + "schema": "BatchCloseItem", + "member": "reason", + "type": "string", + "required": false + }, + { + "schema": "BatchCloseRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "BatchCloseRequest", + "member": "force", + "type": "boolean", + "required": false + }, + { + "schema": "BatchCloseRequest", + "member": "items", + "type": "array", + "required": true, + "item_type": "object" + }, + { + "schema": "BatchCloseRequest", + "member": "session", + "type": "string", + "required": false + }, { "schema": "BatchCloseResponse", "member": "outcomes", "type": "array", + "required": true, + "item_type": "object" + }, + { + "schema": "BatchCreateDependency", + "member": "target_id", + "type": "string", "required": true }, + { + "schema": "BatchCreateDependency", + "member": "type", + "type": "string", + "required": true + }, + { + "schema": "BatchCreateItem", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "dependencies", + "type": "array", + "required": false, + "item_type": "object" + }, + { + "schema": "BatchCreateItem", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "BatchCreateItem", + "member": "priority", + "type": "integer", + "required": false + }, + { + "schema": "BatchCreateItem", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "BatchCreateRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "BatchCreateRequest", + "member": "items", + "type": "array", + "required": true, + "item_type": "object" + }, { "schema": "BatchCreateResponse", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "Blocker", @@ -108,7 +719,8 @@ "schema": "BlockingAnnotations", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "BondRef", @@ -128,12 +740,24 @@ "type": "string", "required": true }, + { + "schema": "ClaimNextRequest", + "member": "actor", + "type": "string", + "required": true + }, { "schema": "ClaimNextResponse", "member": "claimed", "type": "object", "required": false }, + { + "schema": "ClaimRequest", + "member": "actor", + "type": "string", + "required": true + }, { "schema": "ClaimResponse", "member": "already_claimed", @@ -146,6 +770,36 @@ "type": "object", "required": true }, + { + "schema": "CloseIssueRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "CloseIssueRequest", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "CloseIssueRequest", + "member": "force", + "type": "boolean", + "required": false + }, + { + "schema": "CloseIssueRequest", + "member": "reason", + "type": "string", + "required": false + }, + { + "schema": "CloseIssueRequest", + "member": "session", + "type": "string", + "required": false + }, { "schema": "CloseIssueResponse", "member": "already_closed", @@ -180,7 +834,8 @@ "schema": "CloseOutcome", "member": "blockers", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "CloseOutcome", @@ -243,11 +898,38 @@ "type": "string", "required": true }, + { + "schema": "CompareAndSetMetadataRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "CompareAndSetMetadataRequest", + "member": "expected", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "CompareAndSetMetadataRequest", + "member": "key", + "type": "string", + "required": true + }, + { + "schema": "CompareAndSetMetadataRequest", + "member": "value", + "type": "", + "required": false, + "nullable": true + }, { "schema": "CompareAndSetMetadataResponse", "member": "current", "type": "", - "required": false + "required": false, + "nullable": true }, { "schema": "CompareAndSetMetadataResponse", @@ -283,7 +965,8 @@ "schema": "ContextResponse", "member": "capabilities", "type": "array", - "required": true + "required": true, + "item_type": "string" }, { "schema": "ContextResponse", @@ -327,11 +1010,210 @@ "type": "integer", "required": true }, + { + "schema": "CreateIssueDependency", + "member": "metadata", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "CreateIssueDependency", + "member": "reverse", + "type": "boolean", + "required": false + }, + { + "schema": "CreateIssueDependency", + "member": "target_id", + "type": "string", + "required": true + }, + { + "schema": "CreateIssueDependency", + "member": "type", + "type": "string", + "required": true + }, + { + "schema": "CreateIssueRequest", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "CreateIssueRequest", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "dependencies", + "type": "array", + "required": false, + "item_type": "object" + }, + { + "schema": "CreateIssueRequest", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "ephemeral", + "type": "boolean", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "estimated_minutes", + "type": "integer", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "external_ref", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "force_id_prefix", + "type": "boolean", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "id", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "inherit_labels_from_parent", + "type": "boolean", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "CreateIssueRequest", + "member": "metadata", + "type": "", + "required": false, + "nullable": true + }, + { + "schema": "CreateIssueRequest", + "member": "no_history", + "type": "boolean", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "owner", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "parent_id", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "priority", + "type": "integer", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "sender", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueRequest", + "member": "title", + "type": "string", + "required": true + }, + { + "schema": "CreateIssueRequest", + "member": "waits_for", + "type": "object", + "required": false + }, + { + "schema": "CreateIssueWaitsFor", + "member": "gate", + "type": "string", + "required": false + }, + { + "schema": "CreateIssueWaitsFor", + "member": "spawner_id", + "type": "string", + "required": true + }, { "schema": "Cycle", "member": "members", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "Cycle", @@ -361,7 +1243,45 @@ "schema": "CyclesPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" + }, + { + "schema": "DeleteIssuesRequest", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "DeleteIssuesRequest", + "member": "cascade", + "type": "boolean", + "required": false + }, + { + "schema": "DeleteIssuesRequest", + "member": "dry_run", + "type": "boolean", + "required": false + }, + { + "schema": "DeleteIssuesRequest", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "DeleteIssuesRequest", + "member": "force", + "type": "boolean", + "required": false + }, + { + "schema": "DeleteIssuesRequest", + "member": "ids", + "type": "array", + "required": true, + "item_type": "string" }, { "schema": "DeleteIssuesResult", @@ -397,7 +1317,8 @@ "schema": "DeleteIssuesResult", "member": "orphaned", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "DeleteIssuesResult", @@ -476,13 +1397,15 @@ "schema": "DependencyEdges", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "DependencyEdges", "member": "missing", "type": "array", - "required": true + "required": true, + "item_type": "string" }, { "schema": "DependencyTreePage", @@ -494,13 +1417,15 @@ "schema": "DependencyTreePage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "EdgeCounts", "member": "anchors", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "EventRecord", @@ -524,7 +1449,8 @@ "schema": "EventRecord", "member": "issue", "type": "object", - "required": true + "required": true, + "nullable": true }, { "schema": "EventRecord", @@ -562,7 +1488,8 @@ "schema": "EventsPage", "member": "records", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "Health", @@ -607,7 +1534,8 @@ "schema": "Issue", "member": "bonded_from", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "Issue", @@ -632,7 +1560,8 @@ "schema": "Issue", "member": "comments", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "Issue", @@ -677,7 +1606,8 @@ "schema": "Issue", "member": "dependencies", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "Issue", @@ -757,7 +1687,8 @@ "schema": "Issue", "member": "labels", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "Issue", @@ -905,7 +1836,8 @@ "schema": "Issue", "member": "waiters", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "Issue", @@ -923,13 +1855,15 @@ "schema": "IssueBlocking", "member": "blocked_by", "type": "array", - "required": true + "required": true, + "item_type": "string" }, { "schema": "IssueBlocking", "member": "blocks", "type": "array", - "required": true + "required": true, + "item_type": "string" }, { "schema": "IssueBlocking", @@ -947,7 +1881,8 @@ "schema": "IssueCount", "member": "groups", "type": "object", - "required": false + "required": false, + "additional_props_type": "integer" }, { "schema": "IssueCount", @@ -990,7 +1925,8 @@ "schema": "IssueDetails", "member": "bonded_from", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueDetails", @@ -1022,7 +1958,8 @@ "schema": "IssueDetails", "member": "comments", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueDetails", @@ -1073,7 +2010,8 @@ "schema": "IssueDetails", "member": "dependencies", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueDetails", @@ -1093,7 +2031,8 @@ "schema": "IssueDetails", "member": "dependents", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueDetails", @@ -1191,7 +2130,8 @@ "schema": "IssueDetails", "member": "labels", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueDetails", @@ -1365,7 +2305,8 @@ "schema": "IssueDetails", "member": "waiters", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueDetails", @@ -1379,6 +2320,129 @@ "type": "string", "required": false }, + { + "schema": "IssuePatchBody", + "member": "acceptance_criteria", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "add_labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "IssuePatchBody", + "member": "append_notes", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "assignee", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "defer_until", + "type": "string", + "format": "date-time", + "required": false, + "nullable": true + }, + { + "schema": "IssuePatchBody", + "member": "description", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "design", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "due_at", + "type": "string", + "format": "date-time", + "required": false, + "nullable": true + }, + { + "schema": "IssuePatchBody", + "member": "estimated_minutes", + "type": "integer", + "required": false, + "nullable": true + }, + { + "schema": "IssuePatchBody", + "member": "external_ref", + "type": "string", + "required": false, + "nullable": true + }, + { + "schema": "IssuePatchBody", + "member": "issue_type", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "IssuePatchBody", + "member": "metadata", + "type": "object", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "notes", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "parent_id", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "priority", + "type": "integer", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "remove_labels", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "IssuePatchBody", + "member": "status", + "type": "string", + "required": false + }, + { + "schema": "IssuePatchBody", + "member": "title", + "type": "string", + "required": false + }, { "schema": "IssueWithCounts", "member": "acceptance_criteria", @@ -1413,7 +2477,8 @@ "schema": "IssueWithCounts", "member": "bonded_from", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithCounts", @@ -1444,7 +2509,8 @@ "schema": "IssueWithCounts", "member": "comments", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithCounts", @@ -1495,7 +2561,8 @@ "schema": "IssueWithCounts", "member": "dependencies", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithCounts", @@ -1587,7 +2654,8 @@ "schema": "IssueWithCounts", "member": "labels", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueWithCounts", @@ -1741,7 +2809,8 @@ "schema": "IssueWithCounts", "member": "waiters", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueWithCounts", @@ -1789,7 +2858,8 @@ "schema": "IssueWithDependencyMetadata", "member": "bonded_from", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithDependencyMetadata", @@ -1814,7 +2884,8 @@ "schema": "IssueWithDependencyMetadata", "member": "comments", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithDependencyMetadata", @@ -1859,7 +2930,8 @@ "schema": "IssueWithDependencyMetadata", "member": "dependencies", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "IssueWithDependencyMetadata", @@ -1945,7 +3017,8 @@ "schema": "IssueWithDependencyMetadata", "member": "labels", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueWithDependencyMetadata", @@ -2093,7 +3166,8 @@ "schema": "IssueWithDependencyMetadata", "member": "waiters", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "IssueWithDependencyMetadata", @@ -2117,7 +3191,8 @@ "schema": "IssuesPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "IssuesPage", @@ -2135,7 +3210,8 @@ "schema": "MemoriesPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "MemoriesPage", @@ -2195,7 +3271,8 @@ "schema": "Problem", "member": "blockers", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "Problem", @@ -2367,7 +3444,8 @@ "schema": "QueryPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "ReadyCount", @@ -2386,14 +3464,46 @@ "schema": "ReadyPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" + }, + { + "schema": "Ref", + "member": "id", + "type": "string", + "required": false + }, + { + "schema": "Ref", + "member": "key", + "type": "string", + "required": false }, { "schema": "RelatedIssues", "member": "items", "type": "array", + "required": true, + "item_type": "object" + }, + { + "schema": "ReleaseIssueRequest", + "member": "actor", + "type": "string", "required": true }, + { + "schema": "ReleaseIssueRequest", + "member": "expected_assignee", + "type": "string", + "required": false + }, + { + "schema": "ReleaseIssueRequest", + "member": "force", + "type": "boolean", + "required": false + }, { "schema": "ReleaseIssueResponse", "member": "changed", @@ -2412,6 +3522,18 @@ "type": "string", "required": true }, + { + "schema": "RememberRequest", + "member": "content", + "type": "string", + "required": true + }, + { + "schema": "RememberRequest", + "member": "key", + "type": "string", + "required": false + }, { "schema": "RememberedMemory", "member": "key", @@ -2430,6 +3552,24 @@ "type": "string", "required": true }, + { + "schema": "RemoveDependencyRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "RemoveDependencyRequest", + "member": "depends_on_id", + "type": "string", + "required": true + }, + { + "schema": "RemoveDependencyRequest", + "member": "issue_id", + "type": "string", + "required": true + }, { "schema": "RemoveDependencyResponse", "member": "removed", @@ -2442,6 +3582,24 @@ "type": "string", "required": true }, + { + "schema": "ReopenIssueRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "ReopenIssueRequest", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "ReopenIssueRequest", + "member": "reason", + "type": "string", + "required": false + }, { "schema": "ReopenIssueResponse", "member": "already_open", @@ -2460,6 +3618,12 @@ "type": "string", "required": true }, + { + "schema": "SetSettingRequest", + "member": "value", + "type": "string", + "required": true + }, { "schema": "Setting", "member": "key", @@ -2488,7 +3652,8 @@ "schema": "SettingsPage", "member": "items", "type": "array", - "required": true + "required": true, + "item_type": "object" }, { "schema": "SettingsPage", @@ -2506,7 +3671,8 @@ "schema": "Statistics", "member": "blocked_issues", "type": "integer", - "required": true + "required": true, + "nullable": true }, { "schema": "Statistics", @@ -2554,7 +3720,8 @@ "schema": "Statistics", "member": "ready_issues", "type": "integer", - "required": true + "required": true, + "nullable": true }, { "schema": "Statistics", @@ -2580,6 +3747,47 @@ "type": "object", "required": true }, + { + "schema": "SweepRequest", + "member": "actor", + "type": "string", + "required": false + }, + { + "schema": "SweepRequest", + "member": "closed_before", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "SweepRequest", + "member": "dry_run", + "type": "boolean", + "required": false + }, + { + "schema": "SweepRequest", + "member": "pattern", + "type": "string", + "required": false + }, + { + "schema": "SweepRequest", + "member": "protect_referenced", + "type": "boolean", + "required": false + }, + { + "schema": "SweepRequest", + "member": "tier", + "type": "string", + "enum": [ + "durable", + "ephemeral" + ], + "required": true + }, { "schema": "SweepResult", "member": "dependencies", @@ -2608,7 +3816,8 @@ "schema": "SweepResult", "member": "referenced_ids", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "SweepResult", @@ -2692,7 +3901,8 @@ "schema": "TreeNode", "member": "bonded_from", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "TreeNode", @@ -2717,7 +3927,8 @@ "schema": "TreeNode", "member": "comments", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "TreeNode", @@ -2762,7 +3973,8 @@ "schema": "TreeNode", "member": "dependencies", "type": "array", - "required": false + "required": false, + "item_type": "object" }, { "schema": "TreeNode", @@ -2854,7 +4066,8 @@ "schema": "TreeNode", "member": "labels", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "TreeNode", @@ -3014,7 +4227,8 @@ "schema": "TreeNode", "member": "waiters", "type": "array", - "required": false + "required": false, + "item_type": "string" }, { "schema": "TreeNode", @@ -3028,6 +4242,60 @@ "type": "string", "required": false }, + { + "schema": "UpdateIssueRequest", + "member": "actor", + "type": "string", + "required": true + }, + { + "schema": "UpdateIssueRequest", + "member": "claim", + "type": "boolean", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "expected_assignee", + "type": "string", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "expected_status", + "type": "string", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "expected_version", + "type": "string", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "force_assignee_transfer", + "type": "boolean", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "force_close_policy", + "type": "boolean", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "force_notes_overwrite", + "type": "boolean", + "required": false + }, + { + "schema": "UpdateIssueRequest", + "member": "patch", + "type": "object", + "required": true + }, { "schema": "UpdateIssueResponse", "member": "changed", diff --git a/internal/httpapi/wireshape/wireshape.go b/internal/httpapi/wireshape/wireshape.go index e50fc7b7a..1d137aee0 100644 --- a/internal/httpapi/wireshape/wireshape.go +++ b/internal/httpapi/wireshape/wireshape.go @@ -2,6 +2,7 @@ package wireshape import ( "fmt" + "reflect" "sort" "strings" @@ -10,11 +11,22 @@ import ( "github.com/steveyegge/beads/internal/httpapi/spec" ) -// Entry is one member of one schema reachable from an operation's response. -// Schema is the component's name (or, for a member whose own value is an -// inline object or array with no $ref of its own, a synthetic dotted path -// rooted at the nearest named schema) so that two members with the same name -// on different schemas are never confused with one another. +// Entry is one member of one schema reachable from an operation's response OR +// request body. Schema is the component's name (or, for a member whose own +// value is an inline object or array with no $ref of its own, a synthetic +// dotted path rooted at the nearest named schema) so that two members with +// the same name on different schemas are never confused with one another. +// +// The Item* and AdditionalProps* fields exist because a property's own +// Type/Format/Enum/Nullable describe the CONTAINER, not what it contains: a +// `type: array` property is always `Type: "array"` whether its items are +// strings or integers, and a `type: object` property with no `properties` of +// its own (a map, via `additionalProperties`) is always `Type: "object"` +// whether its values are strings or integers. Without these fields, changing +// Issue.labels' item type from string to integer, or IssueCount.groups' +// value type from integer to string, is invisible to this digest even though +// it is exactly the non-additive wire change CurrentWireRevision exists to +// gate — the other half of #6053 alongside the request-body coverage below. type Entry struct { Schema string `json:"schema"` Member string `json:"member"` @@ -22,6 +34,30 @@ type Entry struct { Format string `json:"format,omitempty"` Enum []string `json:"enum,omitempty"` Required bool `json:"required"` + // Nullable records `nullable: true` on this member's own schema node. A + // response member moving from always-present-when-returned to + // sometimes-null is exactly as breaking to a client as the type itself + // changing, and before this field nothing here could see it. + Nullable bool `json:"nullable,omitempty"` + + // ItemType, ItemFormat, ItemEnum and ItemNullable describe a `type: array` + // member's own items node. They are set whenever Type == "array" and the + // items schema resolves to something, regardless of whether the items are + // a scalar (only these fields describe it) or an object with its own + // named members (which ALSO get walked and recorded as their own + // Entry-ies under schema+"."+member+"[]", same as before). + ItemType string `json:"item_type,omitempty"` + ItemFormat string `json:"item_format,omitempty"` + ItemEnum []string `json:"item_enum,omitempty"` + ItemNullable bool `json:"item_nullable,omitempty"` + + // AdditionalPropsType and AdditionalPropsFormat describe a `type: object` + // member's `additionalProperties` schema — the map-value shape of a + // member like IssueCount.groups, which has no `properties` of its own. + // Set only when additionalProperties is a schema (not the boolean `true` + // or `false` some operations use for "no constraint" / "sealed"). + AdditionalPropsType string `json:"additional_props_type,omitempty"` + AdditionalPropsFormat string `json:"additional_props_format,omitempty"` } // Digest is the golden document: the wire revision it was computed against, @@ -31,6 +67,75 @@ type Digest struct { Entries []Entry `json:"entries"` } +// CompareResult is the outcome of diffing two digests by "schema\x00member" +// key: Changed holds keys present in both whose Entry differs, Removed holds +// keys only want has, and Added holds keys only got has. +type CompareResult struct { + Changed []string + Removed []string + Added []string +} + +// Compare diffs want (typically the committed golden) against got (typically +// a fresh Compute()). It is the one comparison TestWireShapeDigest and +// gendigest's SafeToWrite guard both need, extracted here so the write guard +// and the test can never disagree about what counts as a change. +func Compare(want, got Digest) CompareResult { + wantByKey := map[string]Entry{} + for _, e := range want.Entries { + wantByKey[e.Schema+"\x00"+e.Member] = e + } + gotByKey := map[string]Entry{} + for _, e := range got.Entries { + gotByKey[e.Schema+"\x00"+e.Member] = e + } + + var result CompareResult + for key, w := range wantByKey { + g, ok := gotByKey[key] + if !ok { + result.Removed = append(result.Removed, key) + continue + } + if !reflect.DeepEqual(w, g) { + result.Changed = append(result.Changed, key) + } + } + for key := range gotByKey { + if _, ok := wantByKey[key]; !ok { + result.Added = append(result.Added, key) + } + } + sort.Strings(result.Changed) + sort.Strings(result.Removed) + sort.Strings(result.Added) + return result +} + +// SafeToWrite is gendigest's write guard (review MEDIUM: "refuse to write +// changed or removed entries unless CurrentWireRevision is higher than the +// golden's recorded revision. Additive entries are allowed."). A pure +// addition is always safe regardless of revision; a changed or removed entry +// is only safe when candidate's WireRevision is strictly greater than +// golden's — the exact bump TestWireShapeDigest itself requires to go green, +// so gendigest can never be used to silently launder a failing test instead +// of fixing it. +func SafeToWrite(golden, candidate Digest) (bool, string) { + cmp := Compare(golden, candidate) + if len(cmp.Changed) == 0 && len(cmp.Removed) == 0 { + return true, "" + } + if candidate.WireRevision > golden.WireRevision { + return true, "" + } + return false, fmt.Sprintf( + "refusing to write: %d changed and %d removed entr(ies) but wire_revision %d is not greater than "+ + "the existing golden's %d (changed=%v removed=%v) — bump CurrentWireRevision "+ + "(internal/httpapi/wire_revision.go) and the revision table in openapi.v0.yaml's "+ + "wire_revision property FIRST, then regenerate", + len(cmp.Changed), len(cmp.Removed), candidate.WireRevision, golden.WireRevision, cmp.Changed, cmp.Removed) +} + var httpVerbs = map[string]bool{ "get": true, "put": true, "post": true, "delete": true, "patch": true, "head": true, "options": true, "trace": true, @@ -42,6 +147,12 @@ var httpVerbs = map[string]bool{ // (which would otherwise be a cycle: internal/httpapi imports nothing from // here, and this package must stay that way to run from the standalone // gendigest command without pulling in the server). +// +// Both RESPONSE and REQUEST BODY schemas are walked. A request-body-only +// shape change — DeleteIssuesRequest.expected_version moving from string to +// integer, say — is exactly as much an undocumented break to an existing +// client as a response change, and the other half of #6053 was invisible +// here until request bodies were added alongside responses. func Compute(wireRevision int) (Digest, error) { var doc map[string]any if err := yaml.Unmarshal(spec.OpenAPIV0(), &doc); err != nil { @@ -64,6 +175,7 @@ func Compute(wireRevision int) (Digest, error) { if !ok { continue } + responses, _ := op["responses"].(map[string]any) for _, rawResp := range responses { respMap, ok := rawResp.(map[string]any) @@ -71,18 +183,12 @@ func Compute(wireRevision int) (Digest, error) { continue } respMap = c.resolveAny(respMap) - content, _ := respMap["content"].(map[string]any) - for _, rawMedia := range content { - media, ok := rawMedia.(map[string]any) - if !ok { - continue - } - schemaNode, ok := media["schema"].(map[string]any) - if !ok { - continue - } - c.walk(schemaNode, "") - } + c.walkContent(respMap) + } + + if rb, ok := op["requestBody"].(map[string]any); ok { + rb = c.resolveAny(rb) + c.walkContent(rb) } } } @@ -100,6 +206,25 @@ func Compute(wireRevision int) (Digest, error) { return Digest{WireRevision: wireRevision, Entries: entries}, nil } +// walkContent walks every media type's schema under a resolved response or +// requestBody node's `content` map. Shared by Compute's response and request +// body loops: a response object and a resolved requestBody object are both +// shaped `{content: {mediaType: {schema: ...}}}`. +func (c *collector) walkContent(node map[string]any) { + content, _ := node["content"].(map[string]any) + for _, rawMedia := range content { + media, ok := rawMedia.(map[string]any) + if !ok { + continue + } + schemaNode, ok := media["schema"].(map[string]any) + if !ok { + continue + } + c.walk(schemaNode, "") + } +} + type collector struct { doc map[string]any visited map[string]bool @@ -175,10 +300,16 @@ func asString(v any) string { return s } +func asBool(v any) bool { + b, _ := v.(bool) + return b +} + // walk records every property of the schema node reaches (following $refs and -// flattening allOf), then recurses into each property's own object or array -// shape. label names an anonymous (non-$ref) node for entries recorded under -// it; a $ref always overrides it with the component's own name. +// flattening allOf and oneOf), then recurses into each property's own object +// or array shape. label names an anonymous (non-$ref) node for entries +// recorded under it; a $ref always overrides it with the component's own +// name. // // Every NAMED schema is walked at most once (the visited guard), which is // what makes a cycle between named schemas (and there is at least the @@ -202,29 +333,16 @@ func (c *collector) walk(node map[string]any, label string) { node = next } - if allOf, ok := node["allOf"].([]any); ok { - mergedProps := map[string]any{} - var mergedRequired []any - for _, rawSub := range allOf { - subMap, ok := rawSub.(map[string]any) - if !ok { - continue - } - sub := c.resolveAny(subMap) - if props, ok := sub["properties"].(map[string]any); ok { - for k, v := range props { - mergedProps[k] = v - } - } - if req, ok := sub["required"].([]any); ok { - mergedRequired = append(mergedRequired, req...) - } - } - node = map[string]any{ - "type": "object", - "properties": mergedProps, - "required": mergedRequired, - } + if merged, ok := c.mergeCombinator(node, "allOf"); ok { + node = merged + } else if merged, ok := c.mergeCombinator(node, "oneOf"); ok { + // oneOf is a union, not an intersection — merging its branches' + // properties together is not semantically a "oneOf" anymore. This + // document uses oneOf nowhere today (confirmed by grep), so this + // branch is defensive only: a best-effort shape so a future oneOf is + // SEEN by the digest rather than silently invisible, not a claim + // that the merge is the right model for real union validation. + node = merged } if name != "" { @@ -258,15 +376,43 @@ func (c *collector) walk(node map[string]any, label string) { Format: asString(resolved["format"]), Enum: toStringSlice(resolved["enum"]), Required: required[member], + Nullable: asBool(resolved["nullable"]), } + + var itemsNode, addlPropsNode map[string]any + switch entry.Type { + case "array": + if items, ok := resolved["items"].(map[string]any); ok { + resolvedItems := c.resolveAny(items) + entry.ItemType = asString(resolvedItems["type"]) + entry.ItemFormat = asString(resolvedItems["format"]) + entry.ItemEnum = toStringSlice(resolvedItems["enum"]) + entry.ItemNullable = asBool(resolvedItems["nullable"]) + itemsNode = items + } + case "object": + if ap, ok := resolved["additionalProperties"].(map[string]any); ok { + resolvedAP := c.resolveAny(ap) + entry.AdditionalPropsType = asString(resolvedAP["type"]) + entry.AdditionalPropsFormat = asString(resolvedAP["format"]) + addlPropsNode = ap + } + } + c.byKey[entry.Schema+"\x00"+entry.Member] = entry switch entry.Type { case "object": c.walk(propMap, name+"."+member) + if addlPropsNode != nil { + // A map value that is itself a named/object schema gets + // its own members walked too, under a label distinct + // from an array's "[]" so the two can never collide. + c.walk(addlPropsNode, name+"."+member+"{}") + } case "array": - if items, ok := resolved["items"].(map[string]any); ok { - c.walk(items, name+"."+member+"[]") + if itemsNode != nil { + c.walk(itemsNode, name+"."+member+"[]") } } } @@ -283,3 +429,36 @@ func (c *collector) walk(node map[string]any, label string) { } } } + +// mergeCombinator flattens node[key] (allOf or oneOf: a list of subschemas) +// into a synthetic object node with merged properties and required lists, the +// way the pre-existing allOf handling always worked. It reports ok == false +// when node has no such key, leaving node untouched. +func (c *collector) mergeCombinator(node map[string]any, key string) (map[string]any, bool) { + list, ok := node[key].([]any) + if !ok { + return nil, false + } + mergedProps := map[string]any{} + var mergedRequired []any + for _, rawSub := range list { + subMap, ok := rawSub.(map[string]any) + if !ok { + continue + } + sub := c.resolveAny(subMap) + if props, ok := sub["properties"].(map[string]any); ok { + for k, v := range props { + mergedProps[k] = v + } + } + if req, ok := sub["required"].([]any); ok { + mergedRequired = append(mergedRequired, req...) + } + } + return map[string]any{ + "type": "object", + "properties": mergedProps, + "required": mergedRequired, + }, true +} diff --git a/internal/httpapi/wireshape/wireshape_test.go b/internal/httpapi/wireshape/wireshape_test.go index 1d8245621..6df3d7d23 100644 --- a/internal/httpapi/wireshape/wireshape_test.go +++ b/internal/httpapi/wireshape/wireshape_test.go @@ -4,7 +4,6 @@ import ( "encoding/json" "os" "path/filepath" - "reflect" "testing" "github.com/steveyegge/beads/internal/httpapi" @@ -39,31 +38,8 @@ func TestWireShapeDigest(t *testing.T) { t.Fatalf("digest has %d entries, want the full walk", len(got.Entries)) } - wantByKey := map[string]wireshape.Entry{} - for _, e := range golden.Entries { - wantByKey[e.Schema+"\x00"+e.Member] = e - } - gotByKey := map[string]wireshape.Entry{} - for _, e := range got.Entries { - gotByKey[e.Schema+"\x00"+e.Member] = e - } - - var changed, removed, added []string - for key, want := range wantByKey { - got, ok := gotByKey[key] - if !ok { - removed = append(removed, key) - continue - } - if !reflect.DeepEqual(want, got) { - changed = append(changed, key) - } - } - for key := range gotByKey { - if _, ok := wantByKey[key]; !ok { - added = append(added, key) - } - } + cmp := wireshape.Compare(golden, got) + changed, removed, added := cmp.Changed, cmp.Removed, cmp.Added if len(changed) > 0 || len(removed) > 0 { if got.WireRevision <= golden.WireRevision { @@ -90,6 +66,79 @@ func TestWireShapeDigest(t *testing.T) { } } +// TestSafeToWrite is the gendigest write guard's own falsification (review +// MEDIUM: "refuse to write changed or removed entries unless +// CurrentWireRevision is higher than the golden's recorded revision"). +func TestSafeToWrite(t *testing.T) { + base := wireshape.Digest{ + WireRevision: 2, + Entries: []wireshape.Entry{ + {Schema: "Widget", Member: "name", Type: "string", Required: true}, + }, + } + + t.Run("pure addition at the same revision is always safe", func(t *testing.T) { + candidate := wireshape.Digest{ + WireRevision: 2, + Entries: append(append([]wireshape.Entry{}, base.Entries...), + wireshape.Entry{Schema: "Widget", Member: "color", Type: "string"}), + } + if ok, reason := wireshape.SafeToWrite(base, candidate); !ok { + t.Fatalf("pure addition refused: %s", reason) + } + }) + + t.Run("identical digest is always safe", func(t *testing.T) { + if ok, reason := wireshape.SafeToWrite(base, base); !ok { + t.Fatalf("identical digest refused: %s", reason) + } + }) + + t.Run("changed entry at the same revision is refused", func(t *testing.T) { + candidate := wireshape.Digest{ + WireRevision: 2, + Entries: []wireshape.Entry{{Schema: "Widget", Member: "name", Type: "integer", Required: true}}, + } + if ok, _ := wireshape.SafeToWrite(base, candidate); ok { + t.Fatal("changed entry at the same revision was allowed") + } + }) + + t.Run("removed entry at the same revision is refused", func(t *testing.T) { + candidate := wireshape.Digest{WireRevision: 2, Entries: nil} + if ok, _ := wireshape.SafeToWrite(base, candidate); ok { + t.Fatal("removed entry at the same revision was allowed") + } + }) + + t.Run("changed entry with a lower revision is refused", func(t *testing.T) { + candidate := wireshape.Digest{ + WireRevision: 1, + Entries: []wireshape.Entry{{Schema: "Widget", Member: "name", Type: "integer", Required: true}}, + } + if ok, _ := wireshape.SafeToWrite(base, candidate); ok { + t.Fatal("changed entry with a LOWER revision was allowed") + } + }) + + t.Run("changed entry with a higher revision is safe", func(t *testing.T) { + candidate := wireshape.Digest{ + WireRevision: 3, + Entries: []wireshape.Entry{{Schema: "Widget", Member: "name", Type: "integer", Required: true}}, + } + if ok, reason := wireshape.SafeToWrite(base, candidate); !ok { + t.Fatalf("changed entry with a higher revision refused: %s", reason) + } + }) + + t.Run("removed entry with a higher revision is safe", func(t *testing.T) { + candidate := wireshape.Digest{WireRevision: 3, Entries: nil} + if ok, reason := wireshape.SafeToWrite(base, candidate); !ok { + t.Fatalf("removed entry with a higher revision refused: %s", reason) + } + }) +} + func loadGolden(t *testing.T) wireshape.Digest { t.Helper() blob, err := os.ReadFile(filepath.Join("testdata", "golden.json")) From e8a4edbbdcd3595f61042e283bca18383b64f431 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:43 +0000 Subject: [PATCH 04/16] fix(httpapi): make bd_version wire_revision inference advisory Opus review of 12c5144ca (HIGH-3). The spec told a client inferring wire_revision from bd_version to use the release cutoff >= 1.3.0, and said nothing about what happens when that inference is wrong -- which it demonstrably is for real builds already in the field: - main builds up to commit b3ef65c85 report bd_version 1.2.2 (below any 1.3.0 cutoff) while already sending the string-typed revision/ expected_version shape (#6053). - 1.3.0-rc.N pre-release builds sort BELOW the release cutoff 1.3.0 under plain string comparison despite already carrying the string shape. - The bd-enterprise compatibility line at d3ab32773462 reports bd_version 1.1.0 while also sending strings. Moved the cutoff to >= 1.3.0-rc.1 (compared as semver-with-prerelease, which at least captures the rc builds), documented both named exceptions inline, and made the whole inference explicitly ADVISORY: a client must accept revision/expected_version as either a string or a number whenever wire_revision is absent (or probe directly) rather than let the inferred value choose a decoder, and must never refuse solely because the inferred revision is 0. Mirrored into engdocs/design/bd-serve-v0.md. This is spec/doc prose only -- there is no server-side inference code to change, since this server always sends wire_revision directly; the inference is guidance for clients reading a pre-signal server's response (this server, past builds, or an unrelated implementation). Co-Authored-By: Claude Sonnet 5 --- engdocs/design/bd-serve-v0.md | 16 +++++++-- internal/httpapi/apigen/types.gen.go | 14 +++++++- internal/httpapi/spec/openapi.v0.yaml | 47 +++++++++++++++++++++++---- 3 files changed, 66 insertions(+), 11 deletions(-) diff --git a/engdocs/design/bd-serve-v0.md b/engdocs/design/bd-serve-v0.md index 66584d184..de5551079 100644 --- a/engdocs/design/bd-serve-v0.md +++ b/engdocs/design/bd-serve-v0.md @@ -547,9 +547,19 @@ property's description in `openapi.v0.yaml` — `0` for every build before this field existed (an integer `revision`, pre-#6053), `1` for the brief string-typed `revision` (#6053), `2` for the first build carrying `wire_revision` itself. A client talking to a build too old to answer this -field at all infers the value from `bd_version`, which is always present; the -mapping is documented beside the property rather than assumed, because -`bd_version` alone does not sort the same way a counter does. +field at all may use `bd_version` as an ADVISORY hint, never proof: compare it +as semver with pre-release identifiers and treat `>= 1.3.0-rc.1` (not the +release cutoff `1.3.0` — a `1.3.0-rc.N` build already carries the string shape +and sorts below `1.3.0`) as a hint toward `1`, else a hint toward `0`. Two +known builds defeat even that corrected hint: `main` up to commit +`b3ef65c85` reports `bd_version: "1.2.2"` while already sending strings, and +the bd-enterprise compatibility line at `d3ab32773462` reports `"1.1.0"` while +also sending strings. A client therefore MUST accept `revision` and +`expected_version` as either a string or a number whenever `wire_revision` is +absent (or probe the shape directly) rather than let the inferred value +choose a decoder, and MUST NEVER refuse solely because the inferred revision +is `0` — see the `wire_revision` property's description for the exact +language. `min_client_wire_revision` is the other half: the oldest revision a build still answers correctly. A client that knows the revision it was compiled diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index a5981b360..7c5758539 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1053,7 +1053,19 @@ type ContextResponse struct { // `wire_revision` and `min_client_wire_revision` at all (this one). // // - // A client talking to a server that omits this member entirely is talking to a pre-signal server and INFERS the revision rather than reading one: `1` when `bd_version` is `>= 1.3.0`, else `0`. A client that does not know how to speak the revision it reads or infers must refuse to proceed rather than guess at the shape. + // A client talking to a server that omits this member entirely is talking to a pre-signal server. `bd_version` is a HINT for that case, never proof: compare it as semver WITH PRE-RELEASE identifiers, and treat `>= 1.3.0-rc.1` — not the release cutoff `1.3.0` itself — as a hint toward `1`, anything below as a hint toward `0`. The release cutoff is wrong on its own because a `1.3.0-rc.N` pre-release build already carries the `1` shape and sorts BELOW `1.3.0`. + // + // Known exceptions make even that corrected hint unreliable, which is why it is advisory rather than authoritative: + // + // * Builds from `main` up to commit `b3ef65c85` report + // `bd_version: "1.2.2"` — below the `1.3.0-rc.1` hint cutoff — + // while already sending the string-typed shape (`1`, not `0`). + // * The bd-enterprise compatibility line at commit `d3ab32773462` + // reports `bd_version: "1.1.0"` while also sending the + // string-typed shape. + // + // + // Because builds like these exist, a client MUST NOT trust the inferred revision to pick a decoder. Instead it MUST do ONE of: accept `revision` and `expected_version` as EITHER a JSON string OR a JSON number whenever `wire_revision` is absent, or probe the actual shape directly (for instance by inspecting the JSON type of a `revision` value already in hand). A client MUST NEVER refuse a request solely because the inferred revision is `0` — `0` here means only "this is an unreliable hint," never a confirmed fact to gate decoding on by itself. A client that still cannot cope with either shape after probing must refuse to proceed rather than guess, exactly as it would for a `wire_revision` it read directly and does not understand. // // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member's (schema, member, type, format, enum, required) fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass. // diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index e42fdbc5c..4c9d65298 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -220,9 +220,13 @@ info: behavior: a non-additive change to an EXISTING member's type, meaning, presence or name bumps `wire_revision`; a brand-new member or operation does not. A client that reads no `wire_revision` at all is talking to a - server that predates it and infers one from `bd_version` (`>= 1.3.0` - means `1`, else `0`); see `ContextResponse.wire_revision` for the full - table and the `Bd-Wire-Revision` request header. + server that predates it and may use `bd_version` as an ADVISORY hint + (`>= 1.3.0-rc.1` hints `1`, else hints `0`) — never as proof: see + `ContextResponse.wire_revision` for the known builds where the hint is + wrong, why a client must accept `revision`/`expected_version` as either + a string or a number whenever `wire_revision` is absent, and why an + inferred `0` must never by itself be grounds to refuse a request. See + also the `Bd-Wire-Revision` request header. * Clients MUST default-branch on unknown `code` and `reason` values within a status class: both vocabularies grow additively, so an exhaustive switch breaks on the first addition. @@ -7557,10 +7561,39 @@ components: A client talking to a server that omits this member entirely is - talking to a pre-signal server and INFERS the revision rather than - reading one: `1` when `bd_version` is `>= 1.3.0`, else `0`. A - client that does not know how to speak the revision it reads or - infers must refuse to proceed rather than guess at the shape. + talking to a pre-signal server. `bd_version` is a HINT for that + case, never proof: compare it as semver WITH PRE-RELEASE + identifiers, and treat `>= 1.3.0-rc.1` — not the release cutoff + `1.3.0` itself — as a hint toward `1`, anything below as a hint + toward `0`. The release cutoff is wrong on its own because a + `1.3.0-rc.N` pre-release build already carries the `1` shape and + sorts BELOW `1.3.0`. + + + Known exceptions make even that corrected hint unreliable, which + is why it is advisory rather than authoritative: + + + * Builds from `main` up to commit `b3ef65c85` report + `bd_version: "1.2.2"` — below the `1.3.0-rc.1` hint cutoff — + while already sending the string-typed shape (`1`, not `0`). + * The bd-enterprise compatibility line at commit `d3ab32773462` + reports `bd_version: "1.1.0"` while also sending the + string-typed shape. + + + Because builds like these exist, a client MUST NOT trust the + inferred revision to pick a decoder. Instead it MUST do ONE of: + accept `revision` and `expected_version` as EITHER a JSON string + OR a JSON number whenever `wire_revision` is absent, or probe the + actual shape directly (for instance by inspecting the JSON type of + a `revision` value already in hand). A client MUST NEVER refuse a + request solely because the inferred revision is `0` — `0` here + means only "this is an unreliable hint," never a confirmed fact to + gate decoding on by itself. A client that still cannot cope with + either shape after probing must refuse to proceed rather than + guess, exactly as it would for a `wire_revision` it read directly + and does not understand. This document's own drift gate is `TestWireShapeDigest` From dbdbf153f6421a0404a54910d7e704d8d8ac0b9e Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:44 +0000 Subject: [PATCH 05/16] fix(httpapi): LOW-7/8/9/10 review follow-ups for wire-revision signal LOW-7: WireRevisionUnsupported now also sets Problem.wire_revision (this server's CurrentWireRevision) and Problem.bd_version on the wire_revision_unsupported refusal, alongside the existing min_wire_revision, so a client logging the refusal has this server's full self-description without a second /v0/beads/context call. LOW-8: gazelle + make bazel-sync add the BUILD.bazel entries the prior commits were missing: wire_revision.go/wire_revision_test.go in internal/httpapi's srcs, new BUILD.bazel files for wireshape and wireshape/cmd/gendigest, and a testdata glob on httpapi's go_test so the pretoken_*.json baselines are visible under `bazel test`. Verified with `bazel build //internal/httpapi/...` and `bazel test //internal/httpapi/... --test_env=BEADS_TEST_SKIP=dolt` (both green) and `python3 tools/bazel/go_srcs.py --check` (clean). LOW-9: narrowed CHANGELOG.md and engdocs/design/bd-serve-v0.md to describe the digest's actual final coverage (format/enum/required/ nullable, array items, additionalProperties, request bodies, $ref/ allOf/oneOf recursion, the SafeToWrite revision-bump guard) and the token rule's actual final shape (three pinned baselines, request-body members, Capabilities()-membership validation, new-operation exemption) instead of the original, narrower claims. LOW-10: Bd-Wire-Revision now parses via a strict decimal-only parseStrictNonNegativeInt (no leading '+', no leading zero except a bare "0") instead of strconv.Atoi alone, so two different header bytes can no longer name the same revision. New malformed-header test cases: "+1", "01", "007". Regenerated apigen/types.gen.go (make api-gen) for the new Problem fields and wireshape/testdata/golden.json (go run .../wireshape/cmd/gendigest) for the two new additive Problem members; the latter wrote cleanly through SafeToWrite itself, since both are pure additions. go test ./... and bazel test //internal/httpapi/... both green. Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 43 +++++++---- engdocs/design/bd-serve-v0.md | 72 ++++++++++++------- internal/httpapi/BUILD.bazel | 4 +- internal/httpapi/apigen/types.gen.go | 8 +++ internal/httpapi/problem.go | 24 ++++--- internal/httpapi/spec/openapi.v0.yaml | 35 +++++++++ internal/httpapi/wire_revision.go | 33 ++++++++- internal/httpapi/wire_revision_test.go | 15 +++- internal/httpapi/wireshape/BUILD.bazel | 25 +++++++ .../wireshape/cmd/gendigest/BUILD.bazel | 18 +++++ .../httpapi/wireshape/testdata/golden.json | 12 ++++ 11 files changed, 235 insertions(+), 54 deletions(-) create mode 100644 internal/httpapi/wireshape/BUILD.bazel create mode 100644 internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a7f1a8cb..ede534081 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,24 +28,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 non-additive wire-shape counter for this build) and `min_client_wire_revision` (the oldest revision this build still answers correctly). A client may declare the revision it was built for on any - request via the optional `Bd-Wire-Revision` header; a declared revision - below `min_client_wire_revision` is refused with - `400 invalid_argument`/`reason: "wire_revision_unsupported"` and a - `min_wire_revision` field naming the floor. The header is absent by - default and an absent header is served exactly as before — this is - additive, not a new precondition on existing clients. See - `internal/httpapi/wire_revision.go` and the `wire_revision` property in - `internal/httpapi/spec/openapi.v0.yaml` for the full revision history. + request via the optional `Bd-Wire-Revision` header (a plain non-negative + decimal — no leading `+` or leading zero); a declared revision below + `min_client_wire_revision` is refused with `400 invalid_argument`/`reason: + "wire_revision_unsupported"` and `min_wire_revision`, `wire_revision`, and + `bd_version` fields naming this server's floor, current revision, and + release string. The header is absent by default and an absent header is + served exactly as before — this is additive, not a new precondition on + existing clients. A client with no `wire_revision` signal to read yet may + infer one from `bd_version` only as an ADVISORY hint (string-or-number, + never a refusal on an inferred `0`) — see the `wire_revision` property's + inference notes in `internal/httpapi/spec/openapi.v0.yaml` for the cutoff + and known-exception builds. See `internal/httpapi/wire_revision.go` and + that same property for the full revision history. - `GET /v0/beads/issues`'s `sort` parameter now advertises the `issues.list.sort` capability token, the same way an operation itself does, and a CI rule (`TestNewParameterOnExistingOperationHasABehaviorToken`) requires every - new parameter on an existing operation to carry one. -- A CI golden digest (`internal/httpapi/wireshape`) pins the JSON name and - type of every response member across the whole HTTP spec and fails if any - changes without `wire_revision` bumping to match; regenerate it with - `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, - revision-bumped change. + new parameter on an existing operation, or new request body member on an + existing operation, to carry a token that `Capabilities()` actually + advertises (not just backticked `a.b` text shaped like one). Three frozen + baselines (`internal/httpapi/testdata/pretoken_*.json`) grandfather what + predates the rule and are pinned never to grow + (`TestPretokenBaselinesNeverGrow`); a brand-new operation is exempt until + its next change. +- A CI golden digest (`internal/httpapi/wireshape`) pins the JSON name, + type, format, enum, required-ness, and nullability of every response AND + request-body member across the whole HTTP spec — including array item + shape and `additionalProperties` value shape — recursing through `$ref`, + `allOf`, and `oneOf`, and fails if any changes without `wire_revision` + bumping to match; `go run ./internal/httpapi/wireshape/cmd/gendigest` + refuses to write a changed or removed entry unless `wire_revision` has + moved past what the existing golden recorded (purely additive entries + always write). ### Changed diff --git a/engdocs/design/bd-serve-v0.md b/engdocs/design/bd-serve-v0.md index de5551079..727a967c8 100644 --- a/engdocs/design/bd-serve-v0.md +++ b/engdocs/design/bd-serve-v0.md @@ -589,31 +589,49 @@ had a reason to send would make `/healthz` lie about the process being alive. **The golden digest.** A revision bump is a promise a human makes; nothing enforced that the promise was kept, or that it was even necessary. CI pins one against the other with a golden digest, `internal/httpapi/wireshape`: -`wireshape.Compute` walks every response's every content-type schema across -the whole spec — not only `application/json`, so `application/problem+json` -and therefore `Problem` itself is covered too — recording the JSON name and -type of every member it finds, keyed by schema and member name. -`TestWireShapeDigest` fails one of two ways: a changed or removed entry at an -UNCHANGED `wire_revision` is drift nobody signed off on; a changed -`wire_revision` with an unchanged entry set means the golden is simply stale. -Either way the fix is the same explicit command, -`go run ./internal/httpapi/wireshape/cmd/gendigest` — never run to make a -failing test pass on an accidental shape change, only after the revision bump -and the `openapi.v0.yaml` history entry it belongs beside. +`wireshape.Compute` walks every response's AND every request body's every +content-type schema across the whole spec — not only `application/json`, so +`application/problem+json` and therefore `Problem` itself is covered too — +recursing through `$ref`, `allOf`, and `oneOf`, and recording, keyed by schema +and member name, each member's JSON name, type, format, enum values, +required-ness, and nullability, plus the scalar shape of array items and of +`additionalProperties` map values. `TestWireShapeDigest` fails one of two +ways: a changed or removed entry at an UNCHANGED `wire_revision` is drift +nobody signed off on; a changed `wire_revision` with an unchanged entry set +means the golden is simply stale. Either way the fix is the same explicit +command, `go run ./internal/httpapi/wireshape/cmd/gendigest` — never run to +make a failing test pass on an accidental shape change, only after the +revision bump and the `openapi.v0.yaml` history entry it belongs beside; the +command itself refuses to write a changed or removed entry unless +`wire_revision` has moved past what the existing golden recorded, so running +it cannot silently launder drift into the baseline. A purely additive diff — +new members, with no existing entry changed or removed — always writes, +revision bump or not. -**The new-parameter token rule.** `capabilities` already required a -`resource.verb` token for a new OPERATION; nothing required one for a new -PARAMETER on an EXISTING operation, so a client had no single place to learn -that, say, `GET /v0/beads/issues`'s `sort` parameter had started accepting a -value. `issues.list.sort` is the first deliberate use of the same convention -one level down: the token lives in the parameter's own description, exactly as -`capabilities` members carry theirs, enforced by -`TestNewParameterOnExistingOperationHasABehaviorToken` against a frozen -baseline (`internal/httpapi/testdata/pretoken_parameters.json`) of every -parameter the document already had the day the rule was written — written -once and never regenerated, since only a pre-existing, untokened parameter -ever belongs on it. `TestUntokenedParameterRuleFires` proves the checker -itself still distinguishes tokened, grandfathered and untokened parameters, -against a synthetic fixture rather than the real spec, so the rule's own logic -is covered independently of whether the current document happens to exercise -every branch. +**The new-parameter and new-request-body-member token rule.** `capabilities` +already required a `resource.verb` token for a new OPERATION; nothing +required one for a new PARAMETER, or new JSON request body member, on an +EXISTING operation, so a client had no single place to learn that, say, `GET +/v0/beads/issues`'s `sort` parameter had started accepting a value. +`issues.list.sort` is the first deliberate use of the same convention one +level down: the token lives in the parameter's (or body member's) own +description, exactly as `capabilities` members carry theirs, enforced by +`TestNewParameterOnExistingOperationHasABehaviorToken` against three frozen +baselines (`internal/httpapi/testdata/pretoken_parameters.json`, +`pretoken_request_body_members.json`, `pretoken_operations.json`) describing +every parameter, body member, and operation the document already had the day +the rule was written — written once and pinned never to grow +(`TestPretokenBaselinesNeverGrow`), since only something pre-existing and +untokened ever belongs on them, and a brand-new operation is exempt from the +rule entirely until its next change. A backticked `a.b`(`.c`)-shaped span in a +description is not enough on its own: the rule also checks the token against +`Capabilities()`'s own served list, so Go identifiers or example values that +merely look like a token (`workapi.DefaultReadyLimit`, `status.custom`) do not +satisfy it. `TestUntokenedParameterRuleFires` and +`TestUntokenedRequestBodyMemberRuleFires` prove the checker itself still +distinguishes tokened, grandfathered, and untokened parameters and body +members — including a real-but-unrelated token, which correctly passes, since +`Capabilities()` has no per-parameter binding to check a token's relevance +against, only its existence — against synthetic fixtures rather than the real +spec, so the rule's own logic is covered independently of whether the current +document happens to exercise every branch. diff --git a/internal/httpapi/BUILD.bazel b/internal/httpapi/BUILD.bazel index c0e0be8f4..091375c48 100644 --- a/internal/httpapi/BUILD.bazel +++ b/internal/httpapi/BUILD.bazel @@ -39,6 +39,7 @@ go_library( "sweep.go", "tree.go", "update.go", + "wire_revision.go", ], importpath = "github.com/steveyegge/beads/internal/httpapi", visibility = ["//:__subpackages__"], @@ -103,12 +104,13 @@ go_test( "tree_test.go", "update_test.go", "wire_bijection_test.go", + "wire_revision_test.go", ], data = [ "claim.go", "//cmd/bd:list.go", "//cmd/bd:ready.go", - ], + ] + glob(["testdata/**"]), embed = [":httpapi"], deps = [ "//internal/config", diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 7c5758539..6927d64f8 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1053,6 +1053,8 @@ type ContextResponse struct { // `wire_revision` and `min_client_wire_revision` at all (this one). // // + // PRESENCE, not just value: a generated client whose JSON decoder zero-values a missing integer field cannot tell, by looking at a decoded `0` alone, whether the server SENT `0` or sent nothing. This is never actually ambiguous in practice, because `0` and `1` are PERMANENTLY RETIRED values — they describe servers that predate this field or carried the brief interim string shape, and no server that implements this field (`CurrentWireRevision` starts at `2` and only increases) will ever legitimately send a literal `0` or `1`. A decoded `0` can therefore only mean "this server omitted the field," and a client inferring from `bd_version` per the paragraph below is doing exactly the right thing in that case, never guessing at a value the field could have meant on its own. A client that wants presence as a first-class fact rather than an inference anyway should decode `wire_revision` into a nullable type, or check for the JSON key's presence directly, instead of relying on this retirement guarantee. + // // A client talking to a server that omits this member entirely is talking to a pre-signal server. `bd_version` is a HINT for that case, never proof: compare it as semver WITH PRE-RELEASE identifiers, and treat `>= 1.3.0-rc.1` — not the release cutoff `1.3.0` itself — as a hint toward `1`, anything below as a hint toward `0`. The release cutoff is wrong on its own because a `1.3.0-rc.N` pre-release build already carries the `1` shape and sorts BELOW `1.3.0`. // // Known exceptions make even that corrected hint unreliable, which is why it is advisory rather than authoritative: @@ -1534,6 +1536,9 @@ type Problem struct { // IT IS OPTIONAL ON EVERY OPERATION BUT THE CLAIM. `POST /v0/beads/issues/{id}:claim` always carries it, because its conflict path reads the row it lost to. `PATCH /v0/beads/issues/{id}` and `POST /v0/beads/issues:batchApply` carry it only when the refusing transaction reported a holder, and `POST /v0/beads/issues/{id}:release` never does — the ownership fence refuses without naming anyone. An absent member means "this refusal could not name the holder", never "nobody holds it"; re-read the row. Assignee *string `json:"assignee,omitempty"` + // BdVersion With `invalid_argument` / `reason: "wire_revision_unsupported"` ONLY: this server's own `ContextResponse.bd_version`, for a client that logs or reports the refusal and wants the release string alongside the two wire-revision numbers rather than a second request to fetch it. It is set on that refusal and on no other. + BdVersion *string `json:"bd_version,omitempty"` + // BlockerId With `dependency_cycle`, hierarchy refusal only: the ancestor or descendant the edge named as blocker. See `issue_id`. BlockerId *string `json:"blocker_id,omitempty"` @@ -1642,6 +1647,9 @@ type Problem struct { // Type RFC 9457 problem type. This server never emits it, so `about:blank` is implied. A deployment that hosts problem documentation MAY supply it: one stable URI per status+code pair, dereferencing to documentation for that pair. It restates identity that `code` already carries, so a client MUST NOT dispatch on it and a server MUST NOT use it to subdivide a code. Type *string `json:"type,omitempty"` + + // WireRevision With `invalid_argument` / `reason: "wire_revision_unsupported"` ONLY: this server's own current `ContextResponse.wire_revision`, alongside `min_wire_revision`, so a client logging the refusal has both ends of the supported range without a second request to `GET /v0/beads/context`. It is set on that refusal and on no other. + WireRevision *int `json:"wire_revision,omitempty"` } // QueryPage A page of query results. It is `ReadyPage`'s shape rather than `IssuesPage`'s, and the missing member is the point: a page of this operation carries no `next_cursor`, because a cursor is a keyset position in a database order and a predicate query's matching set is assembled outside the database. diff --git a/internal/httpapi/problem.go b/internal/httpapi/problem.go index 7f500db04..eeee88f25 100644 --- a/internal/httpapi/problem.go +++ b/internal/httpapi/problem.go @@ -1361,19 +1361,27 @@ func ProjectMismatch(got, own string) Result { // extension member for the same reason ProjectMismatch discloses // `server_project_id`: so a client that already knows it declared a revision // can tell exactly how far behind it is without re-deriving the number from -// `bd_version`. +// `bd_version`. current and bdVersion are this server's own +// ContextResponse.wire_revision and ContextResponse.bd_version, disclosed in +// the `wire_revision` and `bd_version` extension members so a client logging +// or reporting the refusal has the server's full version story without a +// second request to GET /v0/beads/context (review: "include the server's +// current wire_revision and bd_version in the refusal problem body"). // -// This is the ONLY refusal on the surface that sets `min_wire_revision`, and — -// unlike ProjectMismatch, which is exempt on the identity handshake because -// that is where a client LEARNS the id it must stamp with — it is raised on -// GET /v0/beads/context too: a client that already knows the revision it was -// built for gains nothing from being served a body it has already said it -// cannot decode, and loses the chance to fail before acting on it. -func WireRevisionUnsupported(got, min int) Result { +// This is the ONLY refusal on the surface that sets `min_wire_revision`, +// `wire_revision` and `bd_version`, and — unlike ProjectMismatch, which is +// exempt on the identity handshake because that is where a client LEARNS the +// id it must stamp with — it is raised on GET /v0/beads/context too: a client +// that already knows the revision it was built for gains nothing from being +// served a body it has already said it cannot decode, and loses the chance to +// fail before acting on it. +func WireRevisionUnsupported(got, min, current int, bdVersion string) Result { res := InvalidArgument(WireRevisionHeader, ReasonWireRevisionUnsupported, "the "+WireRevisionHeader+" header names revision "+strconv.Itoa(got)+ ", which is below the "+strconv.Itoa(min)+" this server still answers correctly") res.Problem.MinWireRevision = &min + res.Problem.WireRevision = ¤t + res.Problem.BdVersion = &bdVersion return res } diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index 4c9d65298..6dbaa711b 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -7560,6 +7560,24 @@ components: `wire_revision` and `min_client_wire_revision` at all (this one). + PRESENCE, not just value: a generated client whose JSON decoder + zero-values a missing integer field cannot tell, by looking at a + decoded `0` alone, whether the server SENT `0` or sent nothing. + This is never actually ambiguous in practice, because `0` and `1` + are PERMANENTLY RETIRED values — they describe servers that predate + this field or carried the brief interim string shape, and no + server that implements this field (`CurrentWireRevision` starts at + `2` and only increases) will ever legitimately send a literal `0` + or `1`. A decoded `0` can therefore only mean "this server omitted + the field," and a client inferring from `bd_version` per the + paragraph below is doing exactly the right thing in that case, + never guessing at a value the field could have meant on its own. A + client that wants presence as a first-class fact rather than an + inference anyway should decode `wire_revision` into a nullable + type, or check for the JSON key's presence directly, instead of + relying on this retirement guarantee. + + A client talking to a server that omits this member entirely is talking to a pre-signal server. `bd_version` is a HINT for that case, never proof: compare it as semver WITH PRE-RELEASE @@ -11626,3 +11644,20 @@ components: so a client that declared a `Bd-Wire-Revision` can log exactly how far behind it is without re-deriving the number from `bd_version`. It is set on that refusal and on no other. + wire_revision: + type: integer + description: >- + With `invalid_argument` / `reason: "wire_revision_unsupported"` + ONLY: this server's own current `ContextResponse.wire_revision`, + alongside `min_wire_revision`, so a client logging the refusal has + both ends of the supported range without a second request to + `GET /v0/beads/context`. It is set on that refusal and on no + other. + bd_version: + type: string + description: >- + With `invalid_argument` / `reason: "wire_revision_unsupported"` + ONLY: this server's own `ContextResponse.bd_version`, for a client + that logs or reports the refusal and wants the release string + alongside the two wire-revision numbers rather than a second + request to fetch it. It is set on that refusal and on no other. diff --git a/internal/httpapi/wire_revision.go b/internal/httpapi/wire_revision.go index 303374b47..981eb3a19 100644 --- a/internal/httpapi/wire_revision.go +++ b/internal/httpapi/wire_revision.go @@ -3,6 +3,7 @@ package httpapi import ( "net/http" "strconv" + "strings" ) // WireRevisionHeader names the optional per-request wire-floor declaration. A @@ -56,8 +57,8 @@ func (s *Server) checkWireRevision(r *http.Request) *Result { if raw == "" { return nil } - got, err := strconv.Atoi(raw) - if err != nil || got < 0 { + got, ok := parseStrictNonNegativeInt(raw) + if !ok { requestInfo(r.Context()).refuse(raw) res := InvalidArgument(WireRevisionHeader, ReasonInvalidValue, "the "+WireRevisionHeader+" header must be a non-negative integer") @@ -67,6 +68,32 @@ func (s *Server) checkWireRevision(r *http.Request) *Result { return nil } requestInfo(r.Context()).refuse(raw) - res := WireRevisionUnsupported(got, s.minClientWireRevision) + res := WireRevisionUnsupported(got, s.minClientWireRevision, CurrentWireRevision, s.ctxBody.BdVersion) return &res } + +// parseStrictNonNegativeInt accepts exactly the plain-decimal shape this +// header is documented to carry: one or more ASCII digits, no sign, and no +// leading zero unless the whole value is the single digit "0". strconv.Atoi +// alone is looser than that — it also accepts a leading "+", and it accepts +// leading zeros like "007" — both of which would let two different header +// bytes name the same revision, which is a wire ambiguity this header exists +// to avoid, not a client convenience to extend to it. +func parseStrictNonNegativeInt(raw string) (int, bool) { + if raw == "" || strings.ContainsAny(raw, "+-") { + return 0, false + } + if len(raw) > 1 && raw[0] == '0' { + return 0, false + } + for _, c := range raw { + if c < '0' || c > '9' { + return 0, false + } + } + got, err := strconv.Atoi(raw) + if err != nil { + return 0, false + } + return got, true +} diff --git a/internal/httpapi/wire_revision_test.go b/internal/httpapi/wire_revision_test.go index d5681a3fd..537a568f2 100644 --- a/internal/httpapi/wire_revision_test.go +++ b/internal/httpapi/wire_revision_test.go @@ -58,6 +58,19 @@ func assertWireRevisionUnsupported(t *testing.T, resp *http.Response, wantMin in if !ok || int(min) != wantMin { t.Errorf("min_wire_revision = %#v, want %d", body["min_wire_revision"], wantMin) } + // wire_revision and bd_version ride alongside min_wire_revision on this + // same refusal, so a client logging it can report what this server + // currently is without a second /v0/beads/context round trip. + wire, ok := body["wire_revision"].(float64) + if !ok || int(wire) != CurrentWireRevision { + t.Errorf("wire_revision = %#v, want %d", body["wire_revision"], CurrentWireRevision) + } + // bd_version mirrors this server's own ContextResponse.bd_version, which a + // bare test fixture with no version wired in reports as "" — a string, not + // an absence, is what's being pinned here. + if _, ok := body["bd_version"].(string); !ok { + t.Errorf("bd_version = %#v, want a string", body["bd_version"]) + } } // wireRevisionReadServer is newReadServer's provider wiring (reads_test.go), @@ -139,7 +152,7 @@ func TestWireRevisionBelowFloorRefusesEveryEnforcedRoute(t *testing.T) { func TestWireRevisionMalformedIsInvalidValueNotUnsupported(t *testing.T) { ts := wireRevisionReadServer(t) - for _, raw := range []string{"not-a-number", "-1", "1.5", ""} { + for _, raw := range []string{"not-a-number", "-1", "1.5", "+1", "01", "007", ""} { if raw == "" { continue // the empty string is the absent-header case, tested elsewhere } diff --git a/internal/httpapi/wireshape/BUILD.bazel b/internal/httpapi/wireshape/BUILD.bazel new file mode 100644 index 000000000..2f9f078f1 --- /dev/null +++ b/internal/httpapi/wireshape/BUILD.bazel @@ -0,0 +1,25 @@ +load("@rules_go//go:def.bzl", "go_library", "go_test") + +go_library( + name = "wireshape", + srcs = [ + "doc.go", + "wireshape.go", + ], + importpath = "github.com/steveyegge/beads/internal/httpapi/wireshape", + visibility = ["//:__subpackages__"], + deps = [ + "//internal/httpapi/spec", + "@in_gopkg_yaml_v3//:yaml_v3", + ], +) + +go_test( + name = "wireshape_test", + srcs = ["wireshape_test.go"], + data = glob(["testdata/**"]), + deps = [ + ":wireshape", + "//internal/httpapi", + ], +) diff --git a/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel b/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel new file mode 100644 index 000000000..b377bb9b0 --- /dev/null +++ b/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel @@ -0,0 +1,18 @@ +load("@rules_go//go:def.bzl", "go_binary", "go_library") + +go_library( + name = "gendigest_lib", + srcs = ["main.go"], + importpath = "github.com/steveyegge/beads/internal/httpapi/wireshape/cmd/gendigest", + visibility = ["//visibility:private"], + deps = [ + "//internal/httpapi", + "//internal/httpapi/wireshape", + ], +) + +go_binary( + name = "gendigest", + embed = [":gendigest_lib"], + visibility = ["//:__subpackages__"], +) diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index 942647f7f..dd7534d34 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -3255,6 +3255,12 @@ "type": "string", "required": false }, + { + "schema": "Problem", + "member": "bd_version", + "type": "string", + "required": false + }, { "schema": "Problem", "member": "blocker_id", @@ -3434,6 +3440,12 @@ "format": "uri", "required": false }, + { + "schema": "Problem", + "member": "wire_revision", + "type": "integer", + "required": false + }, { "schema": "QueryPage", "member": "has_more", From 5c6d42df9e5ce7aeb919f8048d05a3079ec74caa Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:44 +0000 Subject: [PATCH 06/16] =?UTF-8?q?fix(httpapi):=20re-review=20LOW=20follow-?= =?UTF-8?q?ups=20=E2=80=94=20retirement=20test,=20gendigest=20-init,=20doc?= =?UTF-8?q?s?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee the wire_revision property doc relies on: CurrentWireRevision >= 2 (so 0 and 1 stay permanently retired) and MinClientWireRevision <= CurrentWireRevision (so this build's own floor never refuses every request against itself). 2. gendigest now refuses to create a golden.json that does not exist yet unless run with -init. Previously a missing file (moved testdata, wrong path, accidental delete) was indistinguishable from a real first run and silently wrote a fresh golden with no revision history checked against anything. writeGolden() is extracted out of main() so this is covered directly: TestWriteGoldenRequiresInitWhenMissing, TestWriteGoldenInitCreatesWhenMissing, TestWriteGoldenIgnoresInitWhenGoldenExists (the existing-golden path is still governed only by SafeToWrite, -init or not). 3. Narrowed the two remaining stale claims that still described the digest's original, narrower coverage: the wire_revision property's prose in openapi.v0.yaml (regenerated into apigen/types.gen.go via make api-gen) and wireshape/doc.go now both list nullable, array item shape, additionalProperties value shape, request bodies, and $ref/allOf/oneOf recursion, plus the new -init guard. go test ./internal/httpapi/... and make api-check both green; bazel test //internal/httpapi/... (httpapi, wireshape, and the new gendigest test target) green; python3 tools/bazel/go_srcs.py --check clean. Co-Authored-By: Claude Sonnet 5 --- internal/httpapi/apigen/types.gen.go | 2 +- internal/httpapi/spec/openapi.v0.yaml | 14 ++- internal/httpapi/wire_revision_test.go | 17 +++ .../wireshape/cmd/gendigest/BUILD.bazel | 9 +- .../httpapi/wireshape/cmd/gendigest/main.go | 104 ++++++++++++------ .../wireshape/cmd/gendigest/main_test.go | 71 ++++++++++++ internal/httpapi/wireshape/doc.go | 32 +++--- 7 files changed, 194 insertions(+), 55 deletions(-) create mode 100644 internal/httpapi/wireshape/cmd/gendigest/main_test.go diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 6927d64f8..f8e0dac9b 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1069,7 +1069,7 @@ type ContextResponse struct { // // Because builds like these exist, a client MUST NOT trust the inferred revision to pick a decoder. Instead it MUST do ONE of: accept `revision` and `expected_version` as EITHER a JSON string OR a JSON number whenever `wire_revision` is absent, or probe the actual shape directly (for instance by inspecting the JSON type of a `revision` value already in hand). A client MUST NEVER refuse a request solely because the inferred revision is `0` — `0` here means only "this is an unreliable hint," never a confirmed fact to gate decoding on by itself. A client that still cannot cope with either shape after probing must refuse to proceed rather than guess, exactly as it would for a `wire_revision` it read directly and does not understand. // - // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member's (schema, member, type, format, enum, required) fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass. + // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, recursing through `$ref`, `allOf` and `oneOf`, fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed entry at an unbumped revision, and refuses to create a golden that does not exist yet unless run with `-init`. // // Every request PARAMETER (query, path or header) and every request BODY MEMBER, added to an EXISTING operation after this document, must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against two frozen baselines (`internal/httpapi/testdata/pretoken_parameters.json` for parameters, `pretoken_request_body_members.json` for body members) of every parameter and body member this document already had the day each rule was written. A member on its baseline is grandfathered and never needs a token; any member not on it — present now or added later, on an operation that already existed when the relevant baseline was frozen — must carry one today. A BRAND-NEW operation (one absent from the third frozen file, `pretoken_operations.json`) is exempt entirely: its own review decides its capability story, not a baseline written before it existed. The token itself must also be one `Capabilities()` actually serves — a backticked string merely shaped like a token proves nothing on its own. All three baselines are written once and never regenerated: only a pre-existing, untokened member or operation ever belongs on them, a newly tokened member satisfies the rule without needing an entry at all, and `TestPretokenBaselinesNeverGrow` pins their sizes so nothing can hand-add a new exemption instead of a token. `issues.list.sort` is the first deliberate use of the parameter rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. WireRevision int `json:"wire_revision"` diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index 6dbaa711b..d72f36166 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -7616,15 +7616,21 @@ components: This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING - response member's (schema, member, type, format, enum, required) - fails CI the moment any one of them changes without this field's - Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) + response member AND request body member's (schema, member, type, + format, enum, required, nullable), plus the item shape of an array + member and the value shape of an `additionalProperties` map, + recursing through `$ref`, `allOf` and `oneOf`, fails CI the moment + any one of them changes without this field's Go constant + (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, - never frozen in place) but needs no revision bump to pass. + never frozen in place) but needs no revision bump to pass; the + command itself refuses to write a changed or removed entry at an + unbumped revision, and refuses to create a golden that does not + exist yet unless run with `-init`. Every request PARAMETER (query, path or header) and every request diff --git a/internal/httpapi/wire_revision_test.go b/internal/httpapi/wire_revision_test.go index 537a568f2..e6a6ee7b1 100644 --- a/internal/httpapi/wire_revision_test.go +++ b/internal/httpapi/wire_revision_test.go @@ -218,6 +218,23 @@ func TestWireRevisionExemptRoutesAreExactlyHealth(t *testing.T) { } } +// TestRetiredRevisionsStayBelowTheFloor pins the PRESENCE guarantee +// documented on the `wire_revision` property: 0 and 1 are permanently +// retired, so a decoded 0 can only mean "the server omitted the field," never +// a value a current or future server actually sent. That reading breaks the +// moment either constant moves the wrong way — CurrentWireRevision dropping +// below 2 would let a real server send one of the retired values again, and +// MinClientWireRevision exceeding CurrentWireRevision would make every +// request refused as unsupported by this build's own floor. +func TestRetiredRevisionsStayBelowTheFloor(t *testing.T) { + if CurrentWireRevision < 2 { + t.Errorf("CurrentWireRevision = %d, want >= 2: 0 and 1 are retired and must stay below every revision a server can actually send", CurrentWireRevision) + } + if MinClientWireRevision > CurrentWireRevision { + t.Errorf("MinClientWireRevision = %d, want <= CurrentWireRevision (%d)", MinClientWireRevision, CurrentWireRevision) + } +} + // TestCapabilitiesAdvertiseIssuesListSort: a client learns this server // accepts GET /v0/beads/issues' sort parameter from the capability list, // never from the version string. diff --git a/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel b/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel index b377bb9b0..425d566ef 100644 --- a/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel +++ b/internal/httpapi/wireshape/cmd/gendigest/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_go//go:def.bzl", "go_binary", "go_library") +load("@rules_go//go:def.bzl", "go_binary", "go_library", "go_test") go_library( name = "gendigest_lib", @@ -16,3 +16,10 @@ go_binary( embed = [":gendigest_lib"], visibility = ["//:__subpackages__"], ) + +go_test( + name = "gendigest_test", + srcs = ["main_test.go"], + embed = [":gendigest_lib"], + deps = ["//internal/httpapi/wireshape"], +) diff --git a/internal/httpapi/wireshape/cmd/gendigest/main.go b/internal/httpapi/wireshape/cmd/gendigest/main.go index 80c2dd089..bd720e330 100644 --- a/internal/httpapi/wireshape/cmd/gendigest/main.go +++ b/internal/httpapi/wireshape/cmd/gendigest/main.go @@ -11,6 +11,8 @@ package main import ( "encoding/json" + "errors" + "flag" "fmt" "os" "path/filepath" @@ -21,50 +23,80 @@ import ( ) func main() { + initFlag := flag.Bool("init", false, "create golden.json when none exists yet") + flag.Parse() + + out, err := goldenPath() + if err != nil { + fmt.Fprintln(os.Stderr, "gendigest:", err) + os.Exit(1) + } + digest, err := wireshape.Compute(httpapi.CurrentWireRevision) if err != nil { fmt.Fprintln(os.Stderr, "gendigest:", err) os.Exit(1) } - // Resolve relative to this source file so the command works from any cwd, - // the same way `go generate` directives in this repo do. - _, thisFile, _, ok := runtime.Caller(0) - if !ok { - fmt.Fprintln(os.Stderr, "gendigest: could not resolve own source path") - os.Exit(1) - } - out := filepath.Join(filepath.Dir(thisFile), "..", "..", "testdata", "golden.json") - - // Refuse to overwrite a golden that would silently launder a failing - // TestWireShapeDigest: a changed or removed entry must come with a - // CurrentWireRevision bump ABOVE what's already committed, never from - // just re-running this command. A purely additive diff, or no existing - // golden at all (first run), always writes. - // #nosec G304 -- out is this command's own source-relative output path, - // never request- or argv-influenced; reading it back is the write guard. - if existing, err := os.ReadFile(out); err == nil { - var golden wireshape.Digest - if err := json.Unmarshal(existing, &golden); err != nil { - fmt.Fprintln(os.Stderr, "gendigest: decode existing golden:", err) - os.Exit(1) - } - if ok, reason := wireshape.SafeToWrite(golden, digest); !ok { - fmt.Fprintln(os.Stderr, "gendigest:", reason) - os.Exit(1) - } - } - - blob, err := json.MarshalIndent(digest, "", " ") - if err != nil { - fmt.Fprintln(os.Stderr, "gendigest:", err) - os.Exit(1) - } - blob = append(blob, '\n') - - if err := os.WriteFile(out, blob, 0o600); err != nil { + if err := writeGolden(out, digest, *initFlag); err != nil { fmt.Fprintln(os.Stderr, "gendigest:", err) os.Exit(1) } fmt.Println("wrote", out) } + +// goldenPath resolves testdata/golden.json relative to this source file, so +// the command works from any cwd, the same way `go generate` directives in +// this repo do. +func goldenPath() (string, error) { + _, thisFile, _, ok := runtime.Caller(0) + if !ok { + return "", errors.New("could not resolve own source path") + } + return filepath.Join(filepath.Dir(thisFile), "..", "..", "testdata", "golden.json"), nil +} + +// writeGolden writes digest to out, under two guards. +// +// First, an EXISTING golden: refuse to overwrite one that would silently +// launder a failing TestWireShapeDigest. A changed or removed entry must come +// with a CurrentWireRevision bump ABOVE what's already committed, never from +// just re-running this command (wireshape.SafeToWrite); a purely additive +// diff always writes. +// +// Second, a MISSING golden: refuse to create one unless initFlag is set. A +// golden that is merely absent — the testdata file moved, got deleted by +// accident, or this is being pointed at the wrong path — must not be treated +// as implicit permission to start a fresh history from whatever the working +// tree happens to compute right now; -init is the explicit, one-time act of +// starting that history (the real first run, or a deliberate reset), not +// gendigest's default behavior for "I don't see a file there." +func writeGolden(out string, digest wireshape.Digest, initFlag bool) error { + // #nosec G304 -- out is this command's own source-relative output path, + // never request- or argv-influenced; reading it back is the write guard. + existing, err := os.ReadFile(out) + switch { + case err == nil: + var golden wireshape.Digest + if err := json.Unmarshal(existing, &golden); err != nil { + return fmt.Errorf("decode existing golden: %w", err) + } + if ok, reason := wireshape.SafeToWrite(golden, digest); !ok { + return errors.New(reason) + } + case os.IsNotExist(err): + if !initFlag { + return fmt.Errorf("no golden at %s yet; pass -init to create one (a missing file is not this command's default to fill in)", out) + } + default: + return fmt.Errorf("read existing golden: %w", err) + } + + blob, err := json.MarshalIndent(digest, "", " ") + if err != nil { + return err + } + blob = append(blob, '\n') + + return os.WriteFile(out, blob, 0o600) +} diff --git a/internal/httpapi/wireshape/cmd/gendigest/main_test.go b/internal/httpapi/wireshape/cmd/gendigest/main_test.go new file mode 100644 index 000000000..b67d30cda --- /dev/null +++ b/internal/httpapi/wireshape/cmd/gendigest/main_test.go @@ -0,0 +1,71 @@ +package main + +import ( + "os" + "path/filepath" + "testing" + + "github.com/steveyegge/beads/internal/httpapi/wireshape" +) + +func sampleDigest(revision int) wireshape.Digest { + return wireshape.Digest{ + WireRevision: revision, + Entries: []wireshape.Entry{ + {Schema: "Widget", Member: "name", Type: "string", Required: true}, + }, + } +} + +// TestWriteGoldenRequiresInitWhenMissing is LOW-2 from the re-review: a +// missing golden.json must not be filled in silently. Without -init, the one +// failure mode that most needs a loud error — pointing this command at a path +// that lost its file by accident — would otherwise look exactly like a +// successful first run. +func TestWriteGoldenRequiresInitWhenMissing(t *testing.T) { + out := filepath.Join(t.TempDir(), "golden.json") + + if err := writeGolden(out, sampleDigest(2), false); err == nil { + t.Fatal("writeGolden with no existing file and initFlag=false: want an error naming -init, got nil") + } + if _, err := os.Stat(out); !os.IsNotExist(err) { + t.Errorf("writeGolden refused but a file exists at %s anyway", out) + } +} + +// TestWriteGoldenInitCreatesWhenMissing is the other half: -init is how a +// real first run (or a deliberate reset) still works. +func TestWriteGoldenInitCreatesWhenMissing(t *testing.T) { + out := filepath.Join(t.TempDir(), "golden.json") + + if err := writeGolden(out, sampleDigest(2), true); err != nil { + t.Fatalf("writeGolden with initFlag=true: %v", err) + } + if _, err := os.Stat(out); err != nil { + t.Errorf("writeGolden -init did not create %s: %v", out, err) + } +} + +// TestWriteGoldenIgnoresInitWhenGoldenExists: -init only matters for the +// missing-file case. An existing golden is still governed exclusively by +// SafeToWrite, whether or not -init was passed. +func TestWriteGoldenIgnoresInitWhenGoldenExists(t *testing.T) { + out := filepath.Join(t.TempDir(), "golden.json") + if err := writeGolden(out, sampleDigest(2), true); err != nil { + t.Fatalf("seed write: %v", err) + } + + // A same-revision changed entry is unsafe per SafeToWrite, -init or not. + changed := sampleDigest(2) + changed.Entries[0].Type = "integer" + if err := writeGolden(out, changed, true); err == nil { + t.Error("writeGolden over an existing golden with an unsafe diff and initFlag=true: want an error, got nil") + } + + // A higher-revision changed entry is safe, with no -init needed. + bumped := sampleDigest(3) + bumped.Entries[0].Type = "integer" + if err := writeGolden(out, bumped, false); err != nil { + t.Errorf("writeGolden over an existing golden with a revision-bumped diff and initFlag=false: %v", err) + } +} diff --git a/internal/httpapi/wireshape/doc.go b/internal/httpapi/wireshape/doc.go index a7fb078bc..68f4267e5 100644 --- a/internal/httpapi/wireshape/doc.go +++ b/internal/httpapi/wireshape/doc.go @@ -1,21 +1,27 @@ // Package wireshape is the CI drift gate for ContextResponse.wire_revision. // -// It computes a digest of every member reachable from any response schema in -// internal/httpapi/spec/openapi.v0.yaml — walking $refs through -// components.schemas and components.responses — recording, per member, the -// (schema, member, type, format, enum, required) tuple the document promises. -// TestWireShapeDigest compares that digest against the committed golden in -// testdata/golden.json and fails on any difference: a member added is fine (an -// additive change never bumps wire_revision), but a member's type, format, -// enum vocabulary, or required-ness changing — or a member disappearing — is -// exactly the class of change internal/httpapi/wire_revision.go's -// CurrentWireRevision exists to gate, and the golden's own -// "wire_revision" field is compared too, so a shape change recorded against -// the OLD revision number still fails. +// It computes a digest of every member reachable from any response schema AND +// any request body schema in internal/httpapi/spec/openapi.v0.yaml — walking +// $refs through components.schemas and components.responses, and recursing +// through allOf/oneOf — recording, per member, the (schema, member, type, +// format, enum, required, nullable) tuple the document promises, plus the +// item shape of an array member and the value shape of an +// additionalProperties map. TestWireShapeDigest compares that digest against +// the committed golden in testdata/golden.json and fails on any difference: a +// member added is fine (an additive change never bumps wire_revision), but a +// member's type, format, enum vocabulary, required-ness, or nullability +// changing — or a member disappearing — is exactly the class of change +// internal/httpapi/wire_revision.go's CurrentWireRevision exists to gate, and +// the golden's own "wire_revision" field is compared too, so a shape change +// recorded against the OLD revision number still fails. // // Regenerate the golden after a deliberate, revision-bumped change with: // // go run ./internal/httpapi/wireshape/cmd/gendigest // -// Never hand-edit testdata/golden.json. +// That command itself refuses to write a changed or removed entry unless +// wire_revision has moved past what the existing golden recorded, and refuses +// to create a golden that does not exist yet unless run with -init — a +// missing file is not treated as permission to start a fresh history +// silently. Never hand-edit testdata/golden.json. package wireshape From 6a0896529e96d702ab9249b35c7d271ac53495c6 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:39:44 +0000 Subject: [PATCH 07/16] feat(wireshape): cover operation parameters in the wire-shape digest MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The digest previously walked only response and request-body schemas, so a type, enum, required, or removal change on one of the spec's 138 operation parameters (123 query, 14 path, 1 header) passed the gate silently — the open item from PR review on #7182. Each parameter is now recorded as its own Entry, keyed by "param:"+operationId / location+":"+name (operation-scoped, not schema-scoped, since two operations' same-named parameter are independent wire contracts), capturing type, array item shape, enum, required, style, explode and default. Adding a parameter stays additive; removing, retyping, narrowing its enum, or changing required/style/explode/default now trips SafeToWrite exactly like a response or request-body change. Verified against the live spec: retyping listIssues' `limit` query parameter from integer to string now fails TestWireShapeDigest (confirmed red, then reverted — not part of this commit). golden.json regenerated via gendigest; the new parameter entries are a pure addition, no wire_revision bump needed. Added TestParameterMutationsAreCaught (remove/retype/narrow- enum against real golden entries) and updated wireshape's doc.go, the `wire_revision` spec prose, and the CHANGELOG to describe the new coverage. Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 7 +- internal/httpapi/apigen/types.gen.go | 2 +- internal/httpapi/spec/openapi.v0.yaml | 23 +- internal/httpapi/wireshape/doc.go | 30 +- .../httpapi/wireshape/testdata/golden.json | 1000 +++++++++++++++++ internal/httpapi/wireshape/wireshape.go | 122 +- internal/httpapi/wireshape/wireshape_test.go | 97 ++ 7 files changed, 1255 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ede534081..616c7cd83 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,7 +60,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 bumping to match; `go run ./internal/httpapi/wireshape/cmd/gendigest` refuses to write a changed or removed entry unless `wire_revision` has moved past what the existing golden recorded (purely additive entries - always write). + always write). The same digest also pins every operation PARAMETER + (query, path and header), keyed by operationId + location + name, and + records its type, item shape, enum, required-ness, style, explode, and + default — a parameter retyped, re-enumerated, narrowed, switched + required, or removed is non-additive exactly like a response or + request-body member. ### Changed diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index f8e0dac9b..46741ff84 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1069,7 +1069,7 @@ type ContextResponse struct { // // Because builds like these exist, a client MUST NOT trust the inferred revision to pick a decoder. Instead it MUST do ONE of: accept `revision` and `expected_version` as EITHER a JSON string OR a JSON number whenever `wire_revision` is absent, or probe the actual shape directly (for instance by inspecting the JSON type of a `revision` value already in hand). A client MUST NEVER refuse a request solely because the inferred revision is `0` — `0` here means only "this is an unreliable hint," never a confirmed fact to gate decoding on by itself. A client that still cannot cope with either shape after probing must refuse to proceed rather than guess, exactly as it would for a `wire_revision` it read directly and does not understand. // - // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, recursing through `$ref`, `allOf` and `oneOf`, fails CI the moment any one of them changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed entry at an unbumped revision, and refuses to create a golden that does not exist yet unless run with `-init`. + // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, recursing through `$ref`, `allOf` and `oneOf`. The same digest also covers every operation PARAMETER (query, path and header) across every operation, keyed by operationId + location + name rather than by schema, recording its type, item shape (for an array parameter), enum, required, style, explode and default. It fails CI the moment any one of these — member or parameter — changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member, parameter or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed entry at an unbumped revision, and refuses to create a golden that does not exist yet unless run with `-init`. // // Every request PARAMETER (query, path or header) and every request BODY MEMBER, added to an EXISTING operation after this document, must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against two frozen baselines (`internal/httpapi/testdata/pretoken_parameters.json` for parameters, `pretoken_request_body_members.json` for body members) of every parameter and body member this document already had the day each rule was written. A member on its baseline is grandfathered and never needs a token; any member not on it — present now or added later, on an operation that already existed when the relevant baseline was frozen — must carry one today. A BRAND-NEW operation (one absent from the third frozen file, `pretoken_operations.json`) is exempt entirely: its own review decides its capability story, not a baseline written before it existed. The token itself must also be one `Capabilities()` actually serves — a backticked string merely shaped like a token proves nothing on its own. All three baselines are written once and never regenerated: only a pre-existing, untokened member or operation ever belongs on them, a newly tokened member satisfies the rule without needing an entry at all, and `TestPretokenBaselinesNeverGrow` pins their sizes so nothing can hand-add a new exemption instead of a token. `issues.list.sort` is the first deliberate use of the parameter rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. WireRevision int `json:"wire_revision"` diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index d72f36166..e66651fb5 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -7619,18 +7619,23 @@ components: response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, - recursing through `$ref`, `allOf` and `oneOf`, fails CI the moment - any one of them changes without this field's Go constant - (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) - increasing to match. Regenerate the golden with + recursing through `$ref`, `allOf` and `oneOf`. The same digest also + covers every operation PARAMETER (query, path and header) across + every operation, keyed by operationId + location + name rather + than by schema, recording its type, item shape (for an array + parameter), enum, required, style, explode and default. It fails + CI the moment any one of these — member or parameter — changes + without this field's Go constant (`CurrentWireRevision`, + `internal/httpapi/wire_revision.go`) increasing to match. + Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive - member or operation also changes the golden (it is appended to, - never frozen in place) but needs no revision bump to pass; the - command itself refuses to write a changed or removed entry at an - unbumped revision, and refuses to create a golden that does not - exist yet unless run with `-init`. + member, parameter or operation also changes the golden (it is + appended to, never frozen in place) but needs no revision bump to + pass; the command itself refuses to write a changed or removed + entry at an unbumped revision, and refuses to create a golden that + does not exist yet unless run with `-init`. Every request PARAMETER (query, path or header) and every request diff --git a/internal/httpapi/wireshape/doc.go b/internal/httpapi/wireshape/doc.go index 68f4267e5..1ef36f085 100644 --- a/internal/httpapi/wireshape/doc.go +++ b/internal/httpapi/wireshape/doc.go @@ -6,14 +6,28 @@ // through allOf/oneOf — recording, per member, the (schema, member, type, // format, enum, required, nullable) tuple the document promises, plus the // item shape of an array member and the value shape of an -// additionalProperties map. TestWireShapeDigest compares that digest against -// the committed golden in testdata/golden.json and fails on any difference: a -// member added is fine (an additive change never bumps wire_revision), but a -// member's type, format, enum vocabulary, required-ness, or nullability -// changing — or a member disappearing — is exactly the class of change -// internal/httpapi/wire_revision.go's CurrentWireRevision exists to gate, and -// the golden's own "wire_revision" field is compared too, so a shape change -// recorded against the OLD revision number still fails. +// additionalProperties map. +// +// It ALSO covers every operation PARAMETER (query, path and header) across +// every operation in the document, keyed by operationId + location + name +// rather than by schema — two different operations' same-named parameter are +// two independent wire contracts, even when they happen to share a shape +// today. Each parameter entry records type, item shape (for an array +// parameter), enum, required, style, explode and default, so a parameter +// silently retyped, re-enumerated, narrowed, switched required, or removed is +// exactly as visible here as the same change to a response or request body +// member — the gap closed after the first revision of this gate shipped +// without it. +// +// TestWireShapeDigest compares that digest against the committed golden in +// testdata/golden.json and fails on any difference: a member or parameter +// added is fine (an additive change never bumps wire_revision), but a +// member's or parameter's type, format, enum vocabulary, required-ness, +// nullability, style, explode or default changing — or either disappearing — +// is exactly the class of change internal/httpapi/wire_revision.go's +// CurrentWireRevision exists to gate, and the golden's own "wire_revision" +// field is compared too, so a shape change recorded against the OLD revision +// number still fails. // // Regenerate the golden after a deliberate, revision-bumped change with: // diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index dd7534d34..cc01a387b 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -4325,6 +4325,1006 @@ "member": "revision", "type": "string", "required": true + }, + { + "schema": "param:addComment", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:claimIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:exclude_label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:exclude_type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:has_metadata_key", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:include_deferred", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:claimNextIssue", + "member": "query:include_ephemeral", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:claimNextIssue", + "member": "query:label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:label_any", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:label_pattern", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:label_regex", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:metadata_field", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:claimNextIssue", + "member": "query:parent", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:priority", + "type": "integer", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:sort", + "type": "string", + "enum": [ + "hybrid", + "oldest", + "priority" + ], + "required": false, + "default": "priority" + }, + { + "schema": "param:claimNextIssue", + "member": "query:type", + "type": "string", + "required": false + }, + { + "schema": "param:claimNextIssue", + "member": "query:unassigned", + "type": "boolean", + "required": false + }, + { + "schema": "param:closeIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:compareAndSetMetadata", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:countDependencyEdges", + "member": "query:direction", + "type": "string", + "enum": [ + "in", + "out" + ], + "required": true + }, + { + "schema": "param:countDependencyEdges", + "member": "query:issue_id", + "type": "array", + "required": true, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countDependencyEdges", + "member": "query:status", + "type": "string", + "required": false + }, + { + "schema": "param:countDependencyEdges", + "member": "query:type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countIssues", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:closed_after", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:closed_before", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:created_after", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:created_before", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:desc_contains", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:empty_description", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countIssues", + "member": "query:group_by", + "type": "string", + "enum": [ + "assignee", + "label", + "priority", + "status", + "type" + ], + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:has_metadata_key", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:id", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:include_ephemeral", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countIssues", + "member": "query:include_infra", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countIssues", + "member": "query:label", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "param:countIssues", + "member": "query:label_any", + "type": "array", + "required": false, + "item_type": "string" + }, + { + "schema": "param:countIssues", + "member": "query:metadata_field", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countIssues", + "member": "query:no_assignee", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countIssues", + "member": "query:no_labels", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countIssues", + "member": "query:notes_contains", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:priority", + "type": "integer", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:priority_max", + "type": "integer", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:priority_min", + "type": "integer", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:status", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:title", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:title_contains", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:type", + "type": "string", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:updated_after", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countIssues", + "member": "query:updated_before", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:exclude_label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countReadyWork", + "member": "query:exclude_type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countReadyWork", + "member": "query:has_metadata_key", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:include_deferred", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countReadyWork", + "member": "query:include_ephemeral", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:countReadyWork", + "member": "query:label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countReadyWork", + "member": "query:label_any", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countReadyWork", + "member": "query:label_pattern", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:label_regex", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:metadata_field", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countReadyWork", + "member": "query:parent", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:priority", + "type": "integer", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:type", + "type": "string", + "required": false + }, + { + "schema": "param:countReadyWork", + "member": "query:unassigned", + "type": "boolean", + "required": false + }, + { + "schema": "param:forgetMemory", + "member": "path:key", + "type": "string", + "required": true + }, + { + "schema": "param:getDependencyTree", + "member": "query:direction", + "type": "string", + "enum": [ + "both", + "down", + "up" + ], + "required": false, + "default": "down" + }, + { + "schema": "param:getDependencyTree", + "member": "query:max_depth", + "type": "integer", + "required": false, + "default": "50" + }, + { + "schema": "param:getDependencyTree", + "member": "query:root_id", + "type": "string", + "required": true + }, + { + "schema": "param:getDependencyTree", + "member": "query:status", + "type": "string", + "required": false + }, + { + "schema": "param:getIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:getIssue", + "member": "query:brief_deps", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:getIssue", + "member": "query:include_comments", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:getIssue", + "member": "query:include_dependents", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:getMemory", + "member": "path:key", + "type": "string", + "required": true + }, + { + "schema": "param:getSetting", + "member": "path:key", + "type": "string", + "required": true + }, + { + "schema": "param:getStats", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:getStats", + "member": "query:skip_blocked", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listBlockingAnnotations", + "member": "query:issue_id", + "type": "array", + "required": true, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listDependencies", + "member": "query:issue_id", + "type": "array", + "required": true, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listDependencies", + "member": "query:type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listEvents", + "member": "query:limit", + "type": "integer", + "required": false, + "default": "1000" + }, + { + "schema": "param:listEvents", + "member": "query:since", + "type": "integer", + "format": "int64", + "required": true + }, + { + "schema": "param:listIssues", + "member": "query:all", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:brief", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:created_after", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:created_before", + "type": "string", + "format": "date-time", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:cursor", + "type": "string", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:exclude_label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listIssues", + "member": "query:has_metadata_key", + "type": "string", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:include_ephemeral", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:include_gates", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:include_infra", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:include_templates", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listIssues", + "member": "query:label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listIssues", + "member": "query:label_any", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listIssues", + "member": "query:limit", + "type": "integer", + "required": false, + "default": "50" + }, + { + "schema": "param:listIssues", + "member": "query:metadata_field", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listIssues", + "member": "query:parent", + "type": "string", + "required": false + }, + { + "schema": "param:listIssues", + "member": "query:sort", + "type": "string", + "enum": [ + "created", + "priority" + ], + "required": false, + "default": "created" + }, + { + "schema": "param:listIssues", + "member": "query:status", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listIssues", + "member": "query:type", + "type": "string", + "required": false + }, + { + "schema": "param:listMemories", + "member": "query:search", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:assignee", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:brief", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listReadyWork", + "member": "query:exclude_label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listReadyWork", + "member": "query:exclude_type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listReadyWork", + "member": "query:has_metadata_key", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:include_deferred", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listReadyWork", + "member": "query:include_ephemeral", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:listReadyWork", + "member": "query:label", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listReadyWork", + "member": "query:label_any", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listReadyWork", + "member": "query:label_pattern", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:label_regex", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:limit", + "type": "integer", + "required": false, + "default": "100" + }, + { + "schema": "param:listReadyWork", + "member": "query:metadata_field", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:listReadyWork", + "member": "query:parent", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:priority", + "type": "integer", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:sort", + "type": "string", + "enum": [ + "hybrid", + "oldest", + "priority" + ], + "required": false, + "default": "priority" + }, + { + "schema": "param:listReadyWork", + "member": "query:type", + "type": "string", + "required": false + }, + { + "schema": "param:listReadyWork", + "member": "query:unassigned", + "type": "boolean", + "required": false + }, + { + "schema": "param:listRelatedIssues", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:listRelatedIssues", + "member": "query:direction", + "type": "string", + "enum": [ + "in", + "out" + ], + "required": true + }, + { + "schema": "param:listRelatedIssues", + "member": "query:type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:queryIssues", + "member": "query:all", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:queryIssues", + "member": "query:limit", + "type": "integer", + "required": false, + "default": "50" + }, + { + "schema": "param:queryIssues", + "member": "query:q", + "type": "string", + "required": true + }, + { + "schema": "param:queryIssues", + "member": "query:reverse", + "type": "boolean", + "required": false, + "default": "false" + }, + { + "schema": "param:queryIssues", + "member": "query:sort", + "type": "string", + "enum": [ + "assignee", + "closed", + "created", + "id", + "priority", + "status", + "title", + "type", + "updated" + ], + "required": false + }, + { + "schema": "param:releaseIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:reopenIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:setSetting", + "member": "path:key", + "type": "string", + "required": true + }, + { + "schema": "param:unsetSetting", + "member": "path:key", + "type": "string", + "required": true + }, + { + "schema": "param:updateIssue", + "member": "path:id", + "type": "string", + "required": true + }, + { + "schema": "param:watchEvents", + "member": "header:Last-Event-ID", + "type": "integer", + "format": "int64", + "required": false + }, + { + "schema": "param:watchEvents", + "member": "query:since", + "type": "integer", + "format": "int64", + "required": true } ] } diff --git a/internal/httpapi/wireshape/wireshape.go b/internal/httpapi/wireshape/wireshape.go index 1d137aee0..4eb6d3a47 100644 --- a/internal/httpapi/wireshape/wireshape.go +++ b/internal/httpapi/wireshape/wireshape.go @@ -12,10 +12,15 @@ import ( ) // Entry is one member of one schema reachable from an operation's response OR -// request body. Schema is the component's name (or, for a member whose own -// value is an inline object or array with no $ref of its own, a synthetic -// dotted path rooted at the nearest named schema) so that two members with -// the same name on different schemas are never confused with one another. +// request body, OR one operation parameter (query, path or header). Schema is +// the component's name (or, for a member whose own value is an inline object +// or array with no $ref of its own, a synthetic dotted path rooted at the +// nearest named schema) so that two members with the same name on different +// schemas are never confused with one another. For a parameter, Schema is +// instead "param:"+operationId and Member is in+":"+name (see +// paramKey) — operation-scoped rather than schema-scoped, because two +// different operations' same-named "status" query parameter are two +// independent wire contracts even when today they happen to share a shape. // // The Item* and AdditionalProps* fields exist because a property's own // Type/Format/Enum/Nullable describe the CONTAINER, not what it contains: a @@ -58,6 +63,32 @@ type Entry struct { // or `false` some operations use for "no constraint" / "sealed"). AdditionalPropsType string `json:"additional_props_type,omitempty"` AdditionalPropsFormat string `json:"additional_props_format,omitempty"` + + // Style, Explode and Default are set only for parameter entries (see + // paramKey): the serialization style OpenAPI uses to flatten an array or + // object parameter into a query/path string, whether it repeats the name + // per value or explodes/collapses, and the literal default value. A + // client that parses `limit=1,2,3` breaks the moment the server starts + // sending `limit=1&limit=2&limit=3` instead, even though neither Type nor + // Enum changed — exactly the gap the parameter digest closes. + Style string `json:"style,omitempty"` + Explode bool `json:"explode,omitempty"` + Default string `json:"default,omitempty"` +} + +// paramSchemaPrefix marks an Entry.Schema as naming an operation (via +// operationId) rather than an OpenAPI component schema. Grepping for this +// prefix is how a reader tells a parameter entry apart from a schema member +// entry in golden.json. +const paramSchemaPrefix = "param:" + +// paramKey returns the Schema/Member pair a parameter entry is filed under: +// operation-scoped by operationId, then by location and name. Location is +// part of the key (not just the name) because OpenAPI allows the same name in +// two different locations on one operation (a path `id` and a query `id` +// would otherwise collide). +func paramKey(operationID, in, name string) (schema, member string) { + return paramSchemaPrefix + operationID, in + ":" + name } // Digest is the golden document: the wire revision it was computed against, @@ -148,11 +179,16 @@ var httpVerbs = map[string]bool{ // here, and this package must stay that way to run from the standalone // gendigest command without pulling in the server). // -// Both RESPONSE and REQUEST BODY schemas are walked. A request-body-only -// shape change — DeleteIssuesRequest.expected_version moving from string to +// RESPONSE schemas, REQUEST BODY schemas, and operation PARAMETERS (query, +// path and header — see paramKey) are all walked. A request-body-only shape +// change — DeleteIssuesRequest.expected_version moving from string to // integer, say — is exactly as much an undocumented break to an existing // client as a response change, and the other half of #6053 was invisible -// here until request bodies were added alongside responses. +// here until request bodies were added alongside responses. Parameters are +// the same story again: a `limit` query parameter silently retyped, +// re-enumerated, or switched from optional to required never touched a +// response or request body schema, so it was invisible to this digest until +// parameter coverage was added. func Compute(wireRevision int) (Digest, error) { var doc map[string]any if err := yaml.Unmarshal(spec.OpenAPIV0(), &doc); err != nil { @@ -190,6 +226,16 @@ func Compute(wireRevision int) (Digest, error) { rb = c.resolveAny(rb) c.walkContent(rb) } + + operationID := asString(op["operationId"]) + params, _ := op["parameters"].([]any) + for _, rawParam := range params { + paramMap, ok := rawParam.(map[string]any) + if !ok { + continue + } + c.walkParam(operationID, c.resolveAny(paramMap)) + } } } @@ -225,6 +271,55 @@ func (c *collector) walkContent(node map[string]any) { } } +// walkParam records one resolved parameter object (already $ref-followed by +// the caller) as a single Entry keyed by paramKey(operationID, in, name). A +// parameter's own Required/Style/Explode live on the parameter object itself; +// Type/Format/Enum/Nullable/Default and, for arrays, the Item* fields, live +// one level down on its `schema` node. Unlike walk, a parameter's schema is +// never itself filed under its own named Entry — this document's parameter +// schemas are all inline scalars or inline arrays of scalars (confirmed by +// grep: no operation parameter uses an object schema), so there is nothing +// further to recurse into, and ANY future object-shaped parameter schema +// still gets its Type/Enum/Required seen here even without recursion. +func (c *collector) walkParam(operationID string, node map[string]any) { + name := asString(node["name"]) + in := asString(node["in"]) + if name == "" || in == "" { + // Not a parameter object this document recognizes (for instance, a + // dangling or unresolved $ref) — nothing to record. + return + } + + schema, _ := node["schema"].(map[string]any) + schema = c.resolveAny(schema) + + entrySchema, entryMember := paramKey(operationID, in, name) + entry := Entry{ + Schema: entrySchema, + Member: entryMember, + Type: asString(schema["type"]), + Format: asString(schema["format"]), + Enum: toStringSlice(schema["enum"]), + Required: asBool(node["required"]), + Nullable: asBool(schema["nullable"]), + Style: asString(node["style"]), + Explode: asBool(node["explode"]), + Default: defaultString(schema["default"]), + } + + if entry.Type == "array" { + if items, ok := schema["items"].(map[string]any); ok { + resolvedItems := c.resolveAny(items) + entry.ItemType = asString(resolvedItems["type"]) + entry.ItemFormat = asString(resolvedItems["format"]) + entry.ItemEnum = toStringSlice(resolvedItems["enum"]) + entry.ItemNullable = asBool(resolvedItems["nullable"]) + } + } + + c.byKey[entry.Schema+"\x00"+entry.Member] = entry +} + type collector struct { doc map[string]any visited map[string]bool @@ -305,6 +400,19 @@ func asBool(v any) bool { return b } +// defaultString renders a schema's `default` value (which, unlike enum +// members, is a single scalar of whatever type the schema declares — string, +// number or bool) as a comparable string, or "" when the schema has none. A +// literal "" default and "no default at all" are indistinguishable here, +// which matches every default this document actually declares (none are the +// empty string). +func defaultString(v any) string { + if v == nil { + return "" + } + return fmt.Sprint(v) +} + // walk records every property of the schema node reaches (following $refs and // flattening allOf and oneOf), then recurses into each property's own object // or array shape. label names an anonymous (non-$ref) node for entries diff --git a/internal/httpapi/wireshape/wireshape_test.go b/internal/httpapi/wireshape/wireshape_test.go index 6df3d7d23..77263ba39 100644 --- a/internal/httpapi/wireshape/wireshape_test.go +++ b/internal/httpapi/wireshape/wireshape_test.go @@ -139,6 +139,103 @@ func TestSafeToWrite(t *testing.T) { }) } +// TestParameterMutationsAreCaught is the falsification for the parameter side +// of the digest (review: "the digest doesn't cover operation PARAMETERS ... +// so a type, enum, required, or removal change on an existing parameter +// passes the gate silently"). It mutates real entries out of the committed +// golden — not a synthetic fixture — so a future rename of the `listIssues` +// limit/sort parameters breaks this test loudly instead of letting the +// coverage go stale unnoticed. +// +// Each case checks the same two things TestSafeToWrite checks for schema +// members: the mutation is refused at the golden's own wire_revision, and +// allowed once the candidate's wire_revision is strictly higher — proving +// SafeToWrite treats a parameter entry exactly as non-additively as a +// response or request-body entry, using nothing parameter-specific. +func TestParameterMutationsAreCaught(t *testing.T) { + golden := loadGolden(t) + + withEntry := func(mutate func(e wireshape.Entry) wireshape.Entry, schema, member string) []wireshape.Entry { + out := make([]wireshape.Entry, 0, len(golden.Entries)) + for _, e := range golden.Entries { + if e.Schema == schema && e.Member == member { + e = mutate(e) + } + out = append(out, e) + } + return out + } + withoutEntry := func(schema, member string) []wireshape.Entry { + out := make([]wireshape.Entry, 0, len(golden.Entries)) + for _, e := range golden.Entries { + if e.Schema == schema && e.Member == member { + continue + } + out = append(out, e) + } + return out + } + mustFind := func(schema, member string) wireshape.Entry { + t.Helper() + for _, e := range golden.Entries { + if e.Schema == schema && e.Member == member { + return e + } + } + t.Fatalf("golden has no %s/%s entry (has the spec changed? update this test's fixture keys)", schema, member) + return wireshape.Entry{} + } + + // listIssues' `limit` query parameter: {type: integer, default: "50"}. + limit := mustFind("param:listIssues", "query:limit") + if limit.Type != "integer" { + t.Fatalf("param:listIssues query:limit is %q in the current golden, not \"integer\" — update this test's fixture", limit.Type) + } + // listIssues' `sort` query parameter: {type: string, enum: [created, priority]}. + sortEntry := mustFind("param:listIssues", "query:sort") + if len(sortEntry.Enum) < 2 { + t.Fatalf("param:listIssues query:sort has %v enum values in the current golden, want >= 2 — update this test's fixture", sortEntry.Enum) + } + + cases := []struct { + name string + mutations []wireshape.Entry + }{ + { + name: "removing a parameter", + mutations: withoutEntry("param:listIssues", "query:limit"), + }, + { + name: "retyping a parameter", + mutations: withEntry(func(e wireshape.Entry) wireshape.Entry { + e.Type = "string" + return e + }, "param:listIssues", "query:limit"), + }, + { + name: "narrowing a parameter's enum", + mutations: withEntry(func(e wireshape.Entry) wireshape.Entry { + e.Enum = e.Enum[:1] + return e + }, "param:listIssues", "query:sort"), + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + sameRevision := wireshape.Digest{WireRevision: golden.WireRevision, Entries: tc.mutations} + if ok, _ := wireshape.SafeToWrite(golden, sameRevision); ok { + t.Fatalf("%s at the same wire_revision was allowed — the digest did not see the change", tc.name) + } + + higherRevision := wireshape.Digest{WireRevision: golden.WireRevision + 1, Entries: tc.mutations} + if ok, reason := wireshape.SafeToWrite(golden, higherRevision); !ok { + t.Fatalf("%s with a bumped wire_revision was refused: %s", tc.name, reason) + } + }) + } +} + func loadGolden(t *testing.T) wireshape.Digest { t.Helper() blob, err := os.ReadFile(filepath.Join("testdata", "golden.json")) From be1f0bad729dde9cc810d8fc40e025cf69d83968 Mon Sep 17 00:00:00 2001 From: Eddie the Engineer Date: Mon, 5 Oct 2026 03:21:06 +0000 Subject: [PATCH 08/16] =?UTF-8?q?fix(wireshape):=20review=20fixes=20?= =?UTF-8?q?=E2=80=94=20stale-golden=20and=20lowered-revision=20arms,=20eff?= =?UTF-8?q?ective=20param=20serialization,=20digest=20boundary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer fixup from the adopt-pr review loop (iterations 1 and 2), on top of e17ba924f. - goldenDrift (A, I): TestWireShapeDigest's comparison moves into a goldenDrift helper. An addition alongside a moved wire_revision no longer passes with a stale golden: it asks for a regenerate, as a changed or removed entry after a bump already did. Every message now names response/request-body members AND parameters, and TestGoldenDrift pins each arm. This takes only the test half of the stranded fixup (adopt-pr/bd-47kdge/stranded-fixup-528cd6095); its doc half, which says parameters are outside the digest, is false since e17ba924f and is not replayed (J). - Lowered wire_revision (D): a CurrentWireRevision LOWER than the golden's is a wrong constant, not a stale golden, so goldenDrift says to restore it instead of regenerating, and SafeToWrite (so gendigest) refuses any write at a lower revision, whatever the entries. - Effective style/explode (B): walkParam records style and explode with OpenAPI 3.0's defaults filled in (form for query/cookie, simple for path/header; explode defaults to style == form). Adding `explode: false` to a query array that relied on form's implicit explode is now a changed entry, while spelling out a default changes nothing. testdata/golden.json is regenerated with -init: 111 parameter entries change only in style/explode (96 to form/true, 15 to simple), 828 entries before and after, wire_revision stays 2. The golden is new in this PR and unreleased, so this needs no bump. - TestWalkParam (H): a package-internal table test (wireshape_internal_test.go) pins every field walkParam extracts, through a computeFrom(document, revision) seam that Compute now wraps: per-location defaults, $ref'd parameters and items, format, nullable and default, content parameters, and the nil returns. TestEffectiveSerializationIsCompared pins B end to end through Compare and SafeToWrite. BUILD.bazel embeds the library in the go_test, which is gazelle's own output. - Required inputs and the digest boundary (C, E, F; docs only): all five prose surfaces (openapi.v0.yaml's wire_revision description and the regenerated apigen/types.gen.go, wireshape/doc.go, CHANGELOG.md, engdocs/design/bd-serve-v0.md) now say that a new REQUIRED request input is recorded as an ordinary addition and needs its own review, that value constraints and member-level composition are outside the digest, and that object and content parameters are recorded only as their container. bd-serve-v0.md also gains the parameter coverage it was missing. - G: a brand-new operation is permanently exempt from the capability-token rule; its own review decides its capability story. - Iteration 2 (nit): walkParam's comment now says that the location default it records as a `content` parameter's style and explode is a placeholder, since OpenAPI 3.0 applies style and explode only to a parameter described by `schema`. All 138 operation parameters in the spec use `schema`, so the golden does not change. Validated: go test ./internal/httpapi/... (gms_pure_go), make api-check, go vet for linux/windows/darwin, gofmt, git diff --check, gazelle on the package (no drift), make bazel-sync-check, and bazel test //internal/httpapi/wireshape/... in the local processwrapper sandbox (2 of 2 pass). Overlay mutation probes: dropping the added-after-bump arm, the lowered-revision arm, SafeToWrite's lowered check, or the effective-default fill each turns the new tests red. Iteration 2 changes only that comment; go test ./internal/httpapi/wireshape/... (gms_pure_go), go vet, gofmt and git diff --check pass again. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 36 +- engdocs/design/bd-serve-v0.md | 52 ++- internal/httpapi/apigen/types.gen.go | 4 +- internal/httpapi/spec/openapi.v0.yaml | 37 +- internal/httpapi/wireshape/BUILD.bazel | 11 +- .../httpapi/wireshape/cmd/gendigest/main.go | 3 +- internal/httpapi/wireshape/doc.go | 47 ++- .../httpapi/wireshape/testdata/golden.json | 363 ++++++++++++++---- internal/httpapi/wireshape/wireshape.go | 70 +++- .../wireshape/wireshape_internal_test.go | 155 ++++++++ internal/httpapi/wireshape/wireshape_test.go | 185 +++++++-- 11 files changed, 778 insertions(+), 185 deletions(-) create mode 100644 internal/httpapi/wireshape/wireshape_internal_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 616c7cd83..380d4c8a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,22 +50,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 advertises (not just backticked `a.b` text shaped like one). Three frozen baselines (`internal/httpapi/testdata/pretoken_*.json`) grandfather what predates the rule and are pinned never to grow - (`TestPretokenBaselinesNeverGrow`); a brand-new operation is exempt until - its next change. + (`TestPretokenBaselinesNeverGrow`); a brand-new operation is permanently + exempt — its own review decides its capability story. - A CI golden digest (`internal/httpapi/wireshape`) pins the JSON name, - type, format, enum, required-ness, and nullability of every response AND - request-body member across the whole HTTP spec — including array item - shape and `additionalProperties` value shape — recursing through `$ref`, - `allOf`, and `oneOf`, and fails if any changes without `wire_revision` - bumping to match; `go run ./internal/httpapi/wireshape/cmd/gendigest` - refuses to write a changed or removed entry unless `wire_revision` has - moved past what the existing golden recorded (purely additive entries - always write). The same digest also pins every operation PARAMETER - (query, path and header), keyed by operationId + location + name, and - records its type, item shape, enum, required-ness, style, explode, and - default — a parameter retyped, re-enumerated, narrowed, switched - required, or removed is non-additive exactly like a response or - request-body member. + type, format, enum, required-ness, and nullability of every EXISTING + response AND request-body member across the whole HTTP spec — including + array item shape and `additionalProperties` value shape — recursing + through `$ref` and through a schema's own `allOf` or `oneOf`, and fails if + any changes without `wire_revision` bumping to match; + `go run ./internal/httpapi/wireshape/cmd/gendigest` refuses to write a + changed or removed entry unless `wire_revision` has moved past what the + existing golden recorded, and refuses any write at a lower + `wire_revision` (purely additive entries otherwise always write). The + same digest also pins every operation PARAMETER (query, path and header), + keyed by operationId + location + name, and records its type, item shape, + enum, required-ness, effective style and explode (OpenAPI's defaults + filled in where unset), and default — a parameter retyped, re-enumerated, + narrowed, re-serialized, switched required, or removed is non-additive + exactly like a response or request-body member. A new member or + parameter always counts as additive, even a REQUIRED one an old client + will not send, and value constraints (`maxLength`, `pattern` and the + like) are outside the digest: such changes need their own review against + `wire_revision`. ### Changed diff --git a/engdocs/design/bd-serve-v0.md b/engdocs/design/bd-serve-v0.md index 727a967c8..da461e51d 100644 --- a/engdocs/design/bd-serve-v0.md +++ b/engdocs/design/bd-serve-v0.md @@ -592,21 +592,38 @@ one against the other with a golden digest, `internal/httpapi/wireshape`: `wireshape.Compute` walks every response's AND every request body's every content-type schema across the whole spec — not only `application/json`, so `application/problem+json` and therefore `Problem` itself is covered too — -recursing through `$ref`, `allOf`, and `oneOf`, and recording, keyed by schema -and member name, each member's JSON name, type, format, enum values, -required-ness, and nullability, plus the scalar shape of array items and of -`additionalProperties` map values. `TestWireShapeDigest` fails one of two -ways: a changed or removed entry at an UNCHANGED `wire_revision` is drift -nobody signed off on; a changed `wire_revision` with an unchanged entry set -means the golden is simply stale. Either way the fix is the same explicit -command, `go run ./internal/httpapi/wireshape/cmd/gendigest` — never run to -make a failing test pass on an accidental shape change, only after the -revision bump and the `openapi.v0.yaml` history entry it belongs beside; the -command itself refuses to write a changed or removed entry unless -`wire_revision` has moved past what the existing golden recorded, so running -it cannot silently launder drift into the baseline. A purely additive diff — -new members, with no existing entry changed or removed — always writes, -revision bump or not. +recursing through `$ref` and through a schema's own `allOf` or `oneOf`, and +recording, keyed by schema and member name, each member's JSON name, type, +format, enum values, required-ness, and nullability, plus the scalar shape of +array items and of `additionalProperties` map values. It records every +operation parameter (query, path, and header) as well, keyed by +`operationId`, location, and name rather than by schema — two operations' +same-named parameters are independent contracts — with its type, item shape, +enum values, required-ness, style, explode, and default; style and explode +are the effective values, OpenAPI's defaults filled in where the document +leaves them unset, so spelling a default out changes nothing while flipping +one does. The digest is a boundary, not the whole wire: value constraints +such as `maxLength` or `pattern` (on members and parameters alike) and a +composition keyword on a single member's own value are outside it, an +object-typed or `content`-described parameter is recorded only as its +container, and a new member or parameter always counts as additive — even a +required one an old client will not send — so each of those needs its own +review against `wire_revision`. `TestWireShapeDigest` fails on any +disagreement between golden and spec, and the failure names the fix: a +changed or removed entry without a `wire_revision` bump is drift nobody +signed off on; a `wire_revision` LOWER than the golden's means the constant +itself is wrong, since the revision table only ever grows; any other +difference — a bumped `wire_revision`, or entries only added — means the +golden is simply stale. Drift and staleness share one explicit fix, +`go run ./internal/httpapi/wireshape/cmd/gendigest` — never run to make a +failing test pass on an accidental shape change, only after the revision +bump and the `openapi.v0.yaml` history entry it belongs beside; the command +itself refuses to write a changed or removed entry unless `wire_revision` +has moved past what the existing golden recorded, and refuses any write at a +lowered `wire_revision`, so running it cannot silently launder drift into +the baseline. A purely additive diff — new members or parameters, with no +existing entry changed or removed — writes at the golden's own +`wire_revision` or any later one. **The new-parameter and new-request-body-member token rule.** `capabilities` already required a `resource.verb` token for a new OPERATION; nothing @@ -622,8 +639,9 @@ baselines (`internal/httpapi/testdata/pretoken_parameters.json`, every parameter, body member, and operation the document already had the day the rule was written — written once and pinned never to grow (`TestPretokenBaselinesNeverGrow`), since only something pre-existing and -untokened ever belongs on them, and a brand-new operation is exempt from the -rule entirely until its next change. A backticked `a.b`(`.c`)-shaped span in a +untokened ever belongs on them, and a brand-new operation is permanently +exempt from the rule: its own review decides its capability story, not a +baseline written before it existed. A backticked `a.b`(`.c`)-shaped span in a description is not enough on its own: the rule also checks the token against `Capabilities()`'s own served list, so Go identifiers or example values that merely look like a token (`workapi.DefaultReadyLimit`, `status.custom`) do not diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 46741ff84..0e414bbc2 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1035,7 +1035,7 @@ type ContextResponse struct { // SchemaVersion The shared JSON schema version — the same constant the CLI's stdout JSON envelope reports. Diagnostic only: it can move for CLI-only reasons with no HTTP wire change, so clients MUST NOT branch on it. SchemaVersion int `json:"schema_version"` - // WireRevision A counter bumped on every release whose wire format changed in a way an unaware client could MISDECODE: an existing member's JSON type or meaning changed, or a member was removed or renamed. Adding a brand-new member to an existing schema, or a brand-new operation, is ADDITIVE and never bumps it — an old client simply never asks for the new member and is unaffected. + // WireRevision A counter bumped on every release whose wire format changed in a way an unaware client could MISDECODE: an existing member's JSON type or meaning changed, or a member was removed or renamed. Adding a brand-new member to an existing schema, or a brand-new operation, is ADDITIVE and never bumps it — an old client simply never asks for the new member and is unaffected. That promise does not extend to a new REQUIRED request input (a request-body member or a parameter): it misdecodes nothing, so it bumps nothing, but an old client that never sends it is refused. The drift gate below records a new required parameter or body member as an ordinary addition and asks for no bump, so keeping old clients working through one is that change's own review's job. // // Revision history, and it only ever grows by appending a row: // @@ -1069,7 +1069,7 @@ type ContextResponse struct { // // Because builds like these exist, a client MUST NOT trust the inferred revision to pick a decoder. Instead it MUST do ONE of: accept `revision` and `expected_version` as EITHER a JSON string OR a JSON number whenever `wire_revision` is absent, or probe the actual shape directly (for instance by inspecting the JSON type of a `revision` value already in hand). A client MUST NEVER refuse a request solely because the inferred revision is `0` — `0` here means only "this is an unreliable hint," never a confirmed fact to gate decoding on by itself. A client that still cannot cope with either shape after probing must refuse to proceed rather than guess, exactly as it would for a `wire_revision` it read directly and does not understand. // - // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, recursing through `$ref`, `allOf` and `oneOf`. The same digest also covers every operation PARAMETER (query, path and header) across every operation, keyed by operationId + location + name rather than by schema, recording its type, item shape (for an array parameter), enum, required, style, explode and default. It fails CI the moment any one of these — member or parameter — changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member, parameter or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed entry at an unbumped revision, and refuses to create a golden that does not exist yet unless run with `-init`. + // This document's own drift gate is `TestWireShapeDigest` (`internal/httpapi/wireshape`): a golden digest of every EXISTING response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, recursing through `$ref` and through a schema's own `allOf` or `oneOf`. The same digest also covers every operation PARAMETER (query, path and header) across every operation, keyed by operationId + location + name rather than by schema, recording its type, item shape (for an array parameter), enum, required, style, explode and default — style and explode as their effective values, OpenAPI's defaults filled in where the document leaves them unset. It fails CI the moment any one of these — member or parameter — changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a deliberate, revision-bumped change, and commit the result — see that package's doc.go for the exact command. A purely additive member, parameter or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed entry at an unbumped revision, refuses any write at a LOWER revision than the golden records, and refuses to create a golden that does not exist yet unless run with `-init`. The digest is a boundary, not the whole wire: value constraints such as `maxLength`, `maximum`, `pattern` or `maxItems` are outside it, on members and parameters alike, and so is a composition keyword on a single member's own value (this document uses none); an object-typed parameter, or one described by `content` rather than `schema`, is recorded only as its container. Narrowing a constraint trips no gate, so such a change must be weighed against this counter in its own review. // // Every request PARAMETER (query, path or header) and every request BODY MEMBER, added to an EXISTING operation after this document, must carry a capability token in its own description, exactly as `capabilities` members do above — `TestNewParameterOnExistingOperationHasABehaviorToken` enforces it against two frozen baselines (`internal/httpapi/testdata/pretoken_parameters.json` for parameters, `pretoken_request_body_members.json` for body members) of every parameter and body member this document already had the day each rule was written. A member on its baseline is grandfathered and never needs a token; any member not on it — present now or added later, on an operation that already existed when the relevant baseline was frozen — must carry one today. A BRAND-NEW operation (one absent from the third frozen file, `pretoken_operations.json`) is exempt entirely: its own review decides its capability story, not a baseline written before it existed. The token itself must also be one `Capabilities()` actually serves — a backticked string merely shaped like a token proves nothing on its own. All three baselines are written once and never regenerated: only a pre-existing, untokened member or operation ever belongs on them, a newly tokened member satisfies the rule without needing an entry at all, and `TestPretokenBaselinesNeverGrow` pins their sizes so nothing can hand-add a new exemption instead of a token. `issues.list.sort` is the first deliberate use of the parameter rule; it also closes the one gap the rule was written to catch, since `sort` itself shipped (#5666) with no token at all. WireRevision int `json:"wire_revision"` diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index e66651fb5..356032b1f 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -7540,7 +7540,13 @@ components: type or meaning changed, or a member was removed or renamed. Adding a brand-new member to an existing schema, or a brand-new operation, is ADDITIVE and never bumps it — an old client simply - never asks for the new member and is unaffected. + never asks for the new member and is unaffected. That promise does + not extend to a new REQUIRED request input (a request-body member + or a parameter): it misdecodes nothing, so it bumps nothing, but an + old client that never sends it is refused. The drift gate below + records a new required parameter or body member as an ordinary + addition and asks for no bump, so keeping old clients working + through one is that change's own review's job. Revision history, and it only ever grows by appending a row: @@ -7619,13 +7625,15 @@ components: response member AND request body member's (schema, member, type, format, enum, required, nullable), plus the item shape of an array member and the value shape of an `additionalProperties` map, - recursing through `$ref`, `allOf` and `oneOf`. The same digest also - covers every operation PARAMETER (query, path and header) across - every operation, keyed by operationId + location + name rather - than by schema, recording its type, item shape (for an array - parameter), enum, required, style, explode and default. It fails - CI the moment any one of these — member or parameter — changes - without this field's Go constant (`CurrentWireRevision`, + recursing through `$ref` and through a schema's own `allOf` or + `oneOf`. The same digest also covers every operation PARAMETER + (query, path and header) across every operation, keyed by + operationId + location + name rather than by schema, recording its + type, item shape (for an array parameter), enum, required, style, + explode and default — style and explode as their effective values, + OpenAPI's defaults filled in where the document leaves them unset. + It fails CI the moment any one of these — member or parameter — + changes without this field's Go constant (`CurrentWireRevision`, `internal/httpapi/wire_revision.go`) increasing to match. Regenerate the golden with `go run ./internal/httpapi/wireshape/cmd/gendigest` after a @@ -7634,8 +7642,17 @@ components: member, parameter or operation also changes the golden (it is appended to, never frozen in place) but needs no revision bump to pass; the command itself refuses to write a changed or removed - entry at an unbumped revision, and refuses to create a golden that - does not exist yet unless run with `-init`. + entry at an unbumped revision, refuses any write at a LOWER + revision than the golden records, and refuses to create a golden + that does not exist yet unless run with `-init`. The digest is a + boundary, not the whole wire: value constraints such as + `maxLength`, `maximum`, `pattern` or `maxItems` are outside it, on + members and parameters alike, and so is a composition keyword on a + single member's own value (this document uses none); an + object-typed parameter, or one described by `content` rather than + `schema`, is recorded only as its container. Narrowing a + constraint trips no gate, so such a change must be weighed against + this counter in its own review. Every request PARAMETER (query, path or header) and every request diff --git a/internal/httpapi/wireshape/BUILD.bazel b/internal/httpapi/wireshape/BUILD.bazel index 2f9f078f1..cdae05960 100644 --- a/internal/httpapi/wireshape/BUILD.bazel +++ b/internal/httpapi/wireshape/BUILD.bazel @@ -16,10 +16,11 @@ go_library( go_test( name = "wireshape_test", - srcs = ["wireshape_test.go"], - data = glob(["testdata/**"]), - deps = [ - ":wireshape", - "//internal/httpapi", + srcs = [ + "wireshape_internal_test.go", + "wireshape_test.go", ], + data = glob(["testdata/**"]), + embed = [":wireshape"], + deps = ["//internal/httpapi"], ) diff --git a/internal/httpapi/wireshape/cmd/gendigest/main.go b/internal/httpapi/wireshape/cmd/gendigest/main.go index bd720e330..546fce73b 100644 --- a/internal/httpapi/wireshape/cmd/gendigest/main.go +++ b/internal/httpapi/wireshape/cmd/gendigest/main.go @@ -62,7 +62,8 @@ func goldenPath() (string, error) { // launder a failing TestWireShapeDigest. A changed or removed entry must come // with a CurrentWireRevision bump ABOVE what's already committed, never from // just re-running this command (wireshape.SafeToWrite); a purely additive -// diff always writes. +// diff always writes, unless digest's wire_revision is LOWER than the +// golden's, which SafeToWrite refuses whatever the entries. // // Second, a MISSING golden: refuse to create one unless initFlag is set. A // golden that is merely absent — the testdata file moved, got deleted by diff --git a/internal/httpapi/wireshape/doc.go b/internal/httpapi/wireshape/doc.go index 1ef36f085..fd88c2ebe 100644 --- a/internal/httpapi/wireshape/doc.go +++ b/internal/httpapi/wireshape/doc.go @@ -2,10 +2,10 @@ // // It computes a digest of every member reachable from any response schema AND // any request body schema in internal/httpapi/spec/openapi.v0.yaml — walking -// $refs through components.schemas and components.responses, and recursing -// through allOf/oneOf — recording, per member, the (schema, member, type, -// format, enum, required, nullable) tuple the document promises, plus the -// item shape of an array member and the value shape of an +// $refs through components.schemas and components.responses, and flattening +// a schema's own allOf/oneOf — recording, per member, the (schema, member, +// type, format, enum, required, nullable) tuple the document promises, plus +// the item shape of an array member and the value shape of an // additionalProperties map. // // It ALSO covers every operation PARAMETER (query, path and header) across @@ -13,29 +13,42 @@ // rather than by schema — two different operations' same-named parameter are // two independent wire contracts, even when they happen to share a shape // today. Each parameter entry records type, item shape (for an array -// parameter), enum, required, style, explode and default, so a parameter -// silently retyped, re-enumerated, narrowed, switched required, or removed is +// parameter), enum, required, style, explode and default — style and explode +// as their effective values, OpenAPI 3.0's defaults filled in where the +// document leaves them unset — so a parameter silently retyped, +// re-enumerated, narrowed, re-serialized, switched required, or removed is // exactly as visible here as the same change to a response or request body // member — the gap closed after the first revision of this gate shipped // without it. // // TestWireShapeDigest compares that digest against the committed golden in // testdata/golden.json and fails on any difference: a member or parameter -// added is fine (an additive change never bumps wire_revision), but a -// member's or parameter's type, format, enum vocabulary, required-ness, -// nullability, style, explode or default changing — or either disappearing — -// is exactly the class of change internal/httpapi/wire_revision.go's -// CurrentWireRevision exists to gate, and the golden's own "wire_revision" -// field is compared too, so a shape change recorded against the OLD revision -// number still fails. +// added needs only a regenerate (an additive change never bumps +// wire_revision), but a member's or parameter's type, format, enum +// vocabulary, required-ness, nullability, style, explode or default changing +// — or either disappearing — is exactly the class of change +// internal/httpapi/wire_revision.go's CurrentWireRevision exists to gate, and +// the golden's own "wire_revision" field is compared too, so a shape change +// recorded against the OLD revision number still fails, and so does a +// CurrentWireRevision lower than the golden's. +// +// The digest is a boundary, not the whole wire. Value constraints such as +// maxLength, maximum, pattern or maxItems are outside it, on members and +// parameters alike, and so is a composition keyword on a single member's own +// value (the document uses none); an object-typed parameter, or one described +// by content rather than schema, is recorded only as its container. A new +// member or parameter always counts as additive, even a REQUIRED one an old +// client will not send. Each of those changes needs its own review against +// wire_revision: nothing here asks for a bump for any of them. // // Regenerate the golden after a deliberate, revision-bumped change with: // // go run ./internal/httpapi/wireshape/cmd/gendigest // // That command itself refuses to write a changed or removed entry unless -// wire_revision has moved past what the existing golden recorded, and refuses -// to create a golden that does not exist yet unless run with -init — a -// missing file is not treated as permission to start a fresh history -// silently. Never hand-edit testdata/golden.json. +// wire_revision has moved past what the existing golden recorded, refuses any +// write at a wire_revision LOWER than the golden's, and refuses to create a +// golden that does not exist yet unless run with -init — a missing file is +// not treated as permission to start a fresh history silently. Never +// hand-edit testdata/golden.json. package wireshape diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index cc01a387b..608ede82a 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -4330,19 +4330,23 @@ "schema": "param:addComment", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:claimIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:claimNextIssue", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", @@ -4366,13 +4370,17 @@ "schema": "param:claimNextIssue", "member": "query:has_metadata_key", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", "member": "query:include_deferred", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4380,6 +4388,8 @@ "member": "query:include_ephemeral", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4404,13 +4414,17 @@ "schema": "param:claimNextIssue", "member": "query:label_pattern", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", "member": "query:label_regex", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", @@ -4425,13 +4439,17 @@ "schema": "param:claimNextIssue", "member": "query:parent", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", "member": "query:priority", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", @@ -4443,31 +4461,39 @@ "priority" ], "required": false, + "style": "form", + "explode": true, "default": "priority" }, { "schema": "param:claimNextIssue", "member": "query:type", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:claimNextIssue", "member": "query:unassigned", "type": "boolean", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:closeIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:compareAndSetMetadata", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:countDependencyEdges", @@ -4477,7 +4503,9 @@ "in", "out" ], - "required": true + "required": true, + "style": "form", + "explode": true }, { "schema": "param:countDependencyEdges", @@ -4492,7 +4520,9 @@ "schema": "param:countDependencyEdges", "member": "query:status", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countDependencyEdges", @@ -4507,47 +4537,61 @@ "schema": "param:countIssues", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:closed_after", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:closed_before", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:created_after", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:created_before", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:desc_contains", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:empty_description", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4561,25 +4605,33 @@ "status", "type" ], - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:has_metadata_key", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:id", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:include_ephemeral", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4587,6 +4639,8 @@ "member": "query:include_infra", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4594,14 +4648,18 @@ "member": "query:label", "type": "array", "required": false, - "item_type": "string" + "item_type": "string", + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:label_any", "type": "array", "required": false, - "item_type": "string" + "item_type": "string", + "style": "form", + "explode": true }, { "schema": "param:countIssues", @@ -4617,6 +4675,8 @@ "member": "query:no_assignee", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4624,75 +4684,99 @@ "member": "query:no_labels", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { "schema": "param:countIssues", "member": "query:notes_contains", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:priority", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:priority_max", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:priority_min", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:status", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:title", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:title_contains", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:type", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:updated_after", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", "member": "query:updated_before", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", @@ -4716,13 +4800,17 @@ "schema": "param:countReadyWork", "member": "query:has_metadata_key", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:include_deferred", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4730,6 +4818,8 @@ "member": "query:include_ephemeral", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4754,13 +4844,17 @@ "schema": "param:countReadyWork", "member": "query:label_pattern", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:label_regex", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", @@ -4775,31 +4869,40 @@ "schema": "param:countReadyWork", "member": "query:parent", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:priority", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:type", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countReadyWork", "member": "query:unassigned", "type": "boolean", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:forgetMemory", "member": "path:key", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:getDependencyTree", @@ -4811,6 +4914,8 @@ "up" ], "required": false, + "style": "form", + "explode": true, "default": "down" }, { @@ -4818,31 +4923,40 @@ "member": "query:max_depth", "type": "integer", "required": false, + "style": "form", + "explode": true, "default": "50" }, { "schema": "param:getDependencyTree", "member": "query:root_id", "type": "string", - "required": true + "required": true, + "style": "form", + "explode": true }, { "schema": "param:getDependencyTree", "member": "query:status", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:getIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:getIssue", "member": "query:brief_deps", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4850,6 +4964,8 @@ "member": "query:include_comments", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4857,31 +4973,39 @@ "member": "query:include_dependents", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { "schema": "param:getMemory", "member": "path:key", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:getSetting", "member": "path:key", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:getStats", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:getStats", "member": "query:skip_blocked", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4916,6 +5040,8 @@ "member": "query:limit", "type": "integer", "required": false, + "style": "form", + "explode": true, "default": "1000" }, { @@ -4923,26 +5049,34 @@ "member": "query:since", "type": "integer", "format": "int64", - "required": true + "required": true, + "style": "form", + "explode": true }, { "schema": "param:listIssues", "member": "query:all", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { "schema": "param:listIssues", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", "member": "query:brief", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4950,20 +5084,26 @@ "member": "query:created_after", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", "member": "query:created_before", "type": "string", "format": "date-time", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", "member": "query:cursor", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", @@ -4978,13 +5118,17 @@ "schema": "param:listIssues", "member": "query:has_metadata_key", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", "member": "query:include_ephemeral", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4992,6 +5136,8 @@ "member": "query:include_gates", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4999,6 +5145,8 @@ "member": "query:include_infra", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5006,6 +5154,8 @@ "member": "query:include_templates", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5031,6 +5181,8 @@ "member": "query:limit", "type": "integer", "required": false, + "style": "form", + "explode": true, "default": "50" }, { @@ -5046,7 +5198,9 @@ "schema": "param:listIssues", "member": "query:parent", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listIssues", @@ -5057,6 +5211,8 @@ "priority" ], "required": false, + "style": "form", + "explode": true, "default": "created" }, { @@ -5072,25 +5228,33 @@ "schema": "param:listIssues", "member": "query:type", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listMemories", "member": "query:search", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:assignee", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:brief", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5115,13 +5279,17 @@ "schema": "param:listReadyWork", "member": "query:has_metadata_key", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:include_deferred", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5129,6 +5297,8 @@ "member": "query:include_ephemeral", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5153,19 +5323,25 @@ "schema": "param:listReadyWork", "member": "query:label_pattern", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:label_regex", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:limit", "type": "integer", "required": false, + "style": "form", + "explode": true, "default": "100" }, { @@ -5181,13 +5357,17 @@ "schema": "param:listReadyWork", "member": "query:parent", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:priority", "type": "integer", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", @@ -5199,25 +5379,32 @@ "priority" ], "required": false, + "style": "form", + "explode": true, "default": "priority" }, { "schema": "param:listReadyWork", "member": "query:type", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listReadyWork", "member": "query:unassigned", "type": "boolean", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:listRelatedIssues", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:listRelatedIssues", @@ -5227,7 +5414,9 @@ "in", "out" ], - "required": true + "required": true, + "style": "form", + "explode": true }, { "schema": "param:listRelatedIssues", @@ -5243,6 +5432,8 @@ "member": "query:all", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5250,19 +5441,25 @@ "member": "query:limit", "type": "integer", "required": false, + "style": "form", + "explode": true, "default": "50" }, { "schema": "param:queryIssues", "member": "query:q", "type": "string", - "required": true + "required": true, + "style": "form", + "explode": true }, { "schema": "param:queryIssues", "member": "query:reverse", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -5280,51 +5477,61 @@ "type", "updated" ], - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:releaseIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:reopenIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:setSetting", "member": "path:key", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:unsetSetting", "member": "path:key", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:updateIssue", "member": "path:id", "type": "string", - "required": true + "required": true, + "style": "simple" }, { "schema": "param:watchEvents", "member": "header:Last-Event-ID", "type": "integer", "format": "int64", - "required": false + "required": false, + "style": "simple" }, { "schema": "param:watchEvents", "member": "query:since", "type": "integer", "format": "int64", - "required": true + "required": true, + "style": "form", + "explode": true } ] } diff --git a/internal/httpapi/wireshape/wireshape.go b/internal/httpapi/wireshape/wireshape.go index 4eb6d3a47..f778424f0 100644 --- a/internal/httpapi/wireshape/wireshape.go +++ b/internal/httpapi/wireshape/wireshape.go @@ -68,9 +68,15 @@ type Entry struct { // paramKey): the serialization style OpenAPI uses to flatten an array or // object parameter into a query/path string, whether it repeats the name // per value or explodes/collapses, and the literal default value. A - // client that parses `limit=1,2,3` breaks the moment the server starts - // sending `limit=1&limit=2&limit=3` instead, even though neither Type nor - // Enum changed — exactly the gap the parameter digest closes. + // server that starts expecting `label=a,b` breaks every client still + // sending `label=a&label=b`, even though neither Type nor Enum changed — + // exactly the gap the parameter digest closes. + // + // Style and Explode are the EFFECTIVE values, with OpenAPI 3.0's defaults + // filled in where the document leaves them unset (see walkParam), so a + // parameter that spells out its default is the same entry as one that + // relies on it, while `explode: false` added to a form parameter that + // relied on the implicit `true` is a changed entry. Style string `json:"style,omitempty"` Explode bool `json:"explode,omitempty"` Default string `json:"default,omitempty"` @@ -150,8 +156,18 @@ func Compare(want, got Digest) CompareResult { // is only safe when candidate's WireRevision is strictly greater than // golden's — the exact bump TestWireShapeDigest itself requires to go green, // so gendigest can never be used to silently launder a failing test instead -// of fixing it. +// of fixing it. A candidate whose WireRevision is LOWER than golden's is +// refused whatever its entries: the revision table is append-only, so writing +// it would record a rollback the spec forbids. func SafeToWrite(golden, candidate Digest) (bool, string) { + if candidate.WireRevision < golden.WireRevision { + return false, fmt.Sprintf( + "refusing to write: wire_revision %d is lower than the existing golden's %d — wire_revision only "+ + "ever increases (the revision table in openapi.v0.yaml's wire_revision property is append-only), "+ + "so restore CurrentWireRevision (internal/httpapi/wire_revision.go) instead of writing the "+ + "golden backwards", + candidate.WireRevision, golden.WireRevision) + } cmp := Compare(golden, candidate) if len(cmp.Changed) == 0 && len(cmp.Removed) == 0 { return true, "" @@ -190,9 +206,16 @@ var httpVerbs = map[string]bool{ // response or request body schema, so it was invisible to this digest until // parameter coverage was added. func Compute(wireRevision int) (Digest, error) { + return computeFrom(spec.OpenAPIV0(), wireRevision) +} + +// computeFrom is Compute over any OpenAPI document, so this package's own +// tests can drive the walk with a synthetic document instead of the embedded +// one. +func computeFrom(document []byte, wireRevision int) (Digest, error) { var doc map[string]any - if err := yaml.Unmarshal(spec.OpenAPIV0(), &doc); err != nil { - return Digest{}, fmt.Errorf("parse embedded openapi document: %w", err) + if err := yaml.Unmarshal(document, &doc); err != nil { + return Digest{}, fmt.Errorf("parse openapi document: %w", err) } c := &collector{doc: doc, visited: map[string]bool{}, byKey: map[string]Entry{}} @@ -281,6 +304,16 @@ func (c *collector) walkContent(node map[string]any) { // grep: no operation parameter uses an object schema), so there is nothing // further to recurse into, and ANY future object-shaped parameter schema // still gets its Type/Enum/Required seen here even without recursion. +// +// Style and Explode are recorded as OpenAPI 3.0's effective values: an unset +// style is `form` for a query or cookie parameter and `simple` for a path or +// header one, and an unset explode is true exactly when the style is `form`. +// Recording the raw keys instead would file an unset explode and an explicit +// `explode: false` as the same entry, though one sends `label=a&label=b` and +// the other `label=a,b`. OpenAPI 3.0 applies style and explode only to a +// parameter described by `schema`: one described by `content` is serialized +// by its media type, so the location default recorded for it is a +// placeholder, not a wire promise (this document has no such parameter). func (c *collector) walkParam(operationID string, node map[string]any) { name := asString(node["name"]) in := asString(node["in"]) @@ -293,6 +326,15 @@ func (c *collector) walkParam(operationID string, node map[string]any) { schema, _ := node["schema"].(map[string]any) schema = c.resolveAny(schema) + style := asString(node["style"]) + if style == "" { + style = defaultParamStyle(in) + } + explode, ok := node["explode"].(bool) + if !ok { + explode = style == "form" + } + entrySchema, entryMember := paramKey(operationID, in, name) entry := Entry{ Schema: entrySchema, @@ -302,8 +344,8 @@ func (c *collector) walkParam(operationID string, node map[string]any) { Enum: toStringSlice(schema["enum"]), Required: asBool(node["required"]), Nullable: asBool(schema["nullable"]), - Style: asString(node["style"]), - Explode: asBool(node["explode"]), + Style: style, + Explode: explode, Default: defaultString(schema["default"]), } @@ -320,6 +362,18 @@ func (c *collector) walkParam(operationID string, node map[string]any) { c.byKey[entry.Schema+"\x00"+entry.Member] = entry } +// defaultParamStyle is OpenAPI 3.0's `style` for a parameter that sets none: +// `form` in a query or cookie, `simple` in a path or header. +func defaultParamStyle(in string) string { + switch in { + case "query", "cookie": + return "form" + case "path", "header": + return "simple" + } + return "" +} + type collector struct { doc map[string]any visited map[string]bool diff --git a/internal/httpapi/wireshape/wireshape_internal_test.go b/internal/httpapi/wireshape/wireshape_internal_test.go new file mode 100644 index 000000000..ee7f7ac47 --- /dev/null +++ b/internal/httpapi/wireshape/wireshape_internal_test.go @@ -0,0 +1,155 @@ +package wireshape + +import ( + "reflect" + "testing" +) + +// paramDoc is a minimal OpenAPI document with one operation, listWidgets, +// whose only parameter is param: a YAML flow mapping, so a case spells its +// whole parameter object on one line. components holds what the $ref cases +// point at. +func paramDoc(param string) []byte { + return []byte(`openapi: 3.0.3 +paths: + /widgets: + get: + operationId: listWidgets + parameters: + - ` + param + ` + responses: {} +components: + parameters: + WidgetColor: {name: color, in: query, required: true, schema: {$ref: '#/components/schemas/Color'}} + schemas: + Color: {type: string, enum: [red, green]} + Shade: {type: integer, format: int32, nullable: true} +`) +} + +// paramDigest runs computeFrom — the walk Compute runs — over paramDoc(param). +// The document has no response or request body, so every entry is a +// parameter entry. +func paramDigest(t *testing.T, param string) Digest { + t.Helper() + d, err := computeFrom(paramDoc(param), 2) + if err != nil { + t.Fatalf("computeFrom: %v", err) + } + return d +} + +// TestWalkParam pins every Entry field walkParam extracts, feeding each +// synthetic parameter object through computeFrom so a $ref'd parameter is +// resolved exactly as Compute resolves it. TestParameterMutationsAreCaught +// cannot see a wrongly extracted field: it mutates golden entries after the +// fact. +func TestWalkParam(t *testing.T) { + const op = "param:listWidgets" + for _, tc := range []struct { + name string + param string + want []Entry // nil: nothing recorded + }{ + {"query array relying on the defaults records form and explode", + `{name: tags, in: query, schema: {type: array, items: {type: string, enum: [b, a]}}}`, + []Entry{{Schema: op, Member: "query:tags", Type: "array", ItemType: "string", ItemEnum: []string{"a", "b"}, Style: "form", Explode: true}}}, + {"explicit explode false is recorded", + `{name: tags, in: query, explode: false, schema: {type: array, items: {type: string}}}`, + []Entry{{Schema: op, Member: "query:tags", Type: "array", ItemType: "string", Style: "form"}}}, + {"explicit non-form style defaults explode to false", + `{name: tags, in: query, style: pipeDelimited, schema: {type: array, items: {type: string}}}`, + []Entry{{Schema: op, Member: "query:tags", Type: "array", ItemType: "string", Style: "pipeDelimited"}}}, + {"path parameter defaults to simple", + `{name: id, in: path, required: true, schema: {type: string}}`, + []Entry{{Schema: op, Member: "path:id", Type: "string", Required: true, Style: "simple"}}}, + {"header parameter defaults to simple", + `{name: X-Trace, in: header, schema: {type: integer}}`, + []Entry{{Schema: op, Member: "header:X-Trace", Type: "integer", Style: "simple"}}}, + {"cookie parameter defaults to form", + `{name: session, in: cookie, schema: {type: string}}`, + []Entry{{Schema: op, Member: "cookie:session", Type: "string", Style: "form", Explode: true}}}, + {"format, nullable and default come from the schema", + `{name: limit, in: query, schema: {type: integer, format: int32, nullable: true, default: 50}}`, + []Entry{{Schema: op, Member: "query:limit", Type: "integer", Format: "int32", Nullable: true, Default: "50", Style: "form", Explode: true}}}, + {"a false default is still a default", + `{name: all, in: query, schema: {type: boolean, default: false}}`, + []Entry{{Schema: op, Member: "query:all", Type: "boolean", Default: "false", Style: "form", Explode: true}}}, + {"a $ref'd parameter and its $ref'd schema are both resolved", + `{$ref: '#/components/parameters/WidgetColor'}`, + []Entry{{Schema: op, Member: "query:color", Type: "string", Enum: []string{"green", "red"}, Required: true, Style: "form", Explode: true}}}, + {"$ref'd array items fill the Item fields", + `{name: shades, in: query, schema: {type: array, items: {$ref: '#/components/schemas/Shade'}}}`, + []Entry{{Schema: op, Member: "query:shades", Type: "array", ItemType: "integer", ItemFormat: "int32", ItemNullable: true, Style: "form", Explode: true}}}, + {"a content parameter is recorded only as its container", + `{name: filter, in: query, content: {application/json: {schema: {type: object}}}}`, + []Entry{{Schema: op, Member: "query:filter", Style: "form", Explode: true}}}, + {"missing in records nothing", + `{name: orphan, schema: {type: string}}`, + nil}, + {"missing name records nothing", + `{in: query, schema: {type: string}}`, + nil}, + {"a dangling $ref records nothing", + `{$ref: '#/components/parameters/Missing'}`, + nil}, + } { + t.Run(tc.name, func(t *testing.T) { + got := paramDigest(t, tc.param).Entries + if len(got) == 0 { + got = nil + } + if !reflect.DeepEqual(got, tc.want) { + t.Errorf("entries:\n got %+v\n want %+v", got, tc.want) + } + }) + } +} + +// TestEffectiveSerializationIsCompared is the Compute-level falsification for +// recording effective style/explode values: changing how a parameter that +// relied on its defaults is serialized must be a changed entry SafeToWrite +// refuses at the same revision, while spelling those same defaults out must +// change nothing. +func TestEffectiveSerializationIsCompared(t *testing.T) { + for _, tc := range []struct { + name string + before, after string + wantChanged bool + }{ + {"explode false on a query array that relied on form's implicit explode", + `{name: tags, in: query, schema: {type: array, items: {type: string}}}`, + `{name: tags, in: query, explode: false, schema: {type: array, items: {type: string}}}`, + true}, + {"a non-form style on a query array that relied on form", + `{name: tags, in: query, schema: {type: array, items: {type: string}}}`, + `{name: tags, in: query, style: spaceDelimited, schema: {type: array, items: {type: string}}}`, + true}, + {"label style on a path parameter that relied on simple", + `{name: id, in: path, required: true, schema: {type: string}}`, + `{name: id, in: path, required: true, style: label, schema: {type: string}}`, + true}, + {"spelling out a query parameter's default style and explode", + `{name: tags, in: query, schema: {type: array, items: {type: string}}}`, + `{name: tags, in: query, style: form, explode: true, schema: {type: array, items: {type: string}}}`, + false}, + {"spelling out a path parameter's default style and explode", + `{name: id, in: path, required: true, schema: {type: string}}`, + `{name: id, in: path, required: true, style: simple, explode: false, schema: {type: string}}`, + false}, + } { + t.Run(tc.name, func(t *testing.T) { + before, after := paramDigest(t, tc.before), paramDigest(t, tc.after) + cmp := Compare(before, after) + if len(cmp.Removed) > 0 || len(cmp.Added) > 0 { + t.Fatalf("parameter key moved: removed=%v added=%v", cmp.Removed, cmp.Added) + } + if changed := len(cmp.Changed) > 0; changed != tc.wantChanged { + t.Errorf("Compare changed=%v, want a change: %v", cmp.Changed, tc.wantChanged) + } + if ok, _ := SafeToWrite(before, after); ok == tc.wantChanged { + t.Errorf("SafeToWrite at the same revision = %v, want %v", ok, !tc.wantChanged) + } + }) + } +} diff --git a/internal/httpapi/wireshape/wireshape_test.go b/internal/httpapi/wireshape/wireshape_test.go index 77263ba39..e2d19d720 100644 --- a/internal/httpapi/wireshape/wireshape_test.go +++ b/internal/httpapi/wireshape/wireshape_test.go @@ -2,8 +2,10 @@ package wireshape_test import ( "encoding/json" + "fmt" "os" "path/filepath" + "strings" "testing" "github.com/steveyegge/beads/internal/httpapi" @@ -11,17 +13,20 @@ import ( ) // TestWireShapeDigest is the drift gate ContextResponse.wire_revision -// documents: it fails the moment the shape of any EXISTING response member -// changes without CurrentWireRevision increasing to match. +// documents: it fails the moment the shape of any EXISTING response member, +// request-body member or operation parameter changes without +// CurrentWireRevision increasing to match. // -// It is deliberately two separate comparisons rather than one struct-equal, -// because the two ways this can go red call for different instructions. A -// changed or removed entry at the SAME wire_revision is drift nobody signed -// off on — regenerate only after bumping CurrentWireRevision (see +// It is deliberately not one struct-equal, because the ways this can go red +// call for different instructions (goldenDrift picks one). A changed or +// removed entry at the SAME wire_revision is drift nobody signed off on — +// regenerate only after bumping CurrentWireRevision (see // internal/httpapi/wire_revision.go) and the `wire_revision` property's -// revision table in openapi.v0.yaml, never before. A changed wire_revision -// with an unchanged entry set (or only additions) means this golden is simply -// stale — regenerate it with: +// revision table in openapi.v0.yaml, never before. A wire_revision LOWER than +// the golden's is never fixed by regenerating: that table is append-only, so +// the constant itself is wrong. Any other difference — a bumped +// wire_revision, or entries only added — means this golden is simply stale; +// regenerate it with: // // go run ./internal/httpapi/wireshape/cmd/gendigest func TestWireShapeDigest(t *testing.T) { @@ -38,37 +43,129 @@ func TestWireShapeDigest(t *testing.T) { t.Fatalf("digest has %d entries, want the full walk", len(got.Entries)) } + if drift := goldenDrift(golden, got); drift != "" { + t.Error(drift) + } +} + +// goldenDrift is TestWireShapeDigest's verdict, factored out so +// TestGoldenDrift can drive every arm with synthetic digests. It returns "" +// only when golden records exactly got — the same entries at the same +// wire_revision. Every other state fails; the arms only choose which +// instruction the failure gives. +func goldenDrift(golden, got wireshape.Digest) string { + if got.WireRevision < golden.WireRevision { + return fmt.Sprintf("CurrentWireRevision is %d, LOWER than the %d the golden records: wire_revision only "+ + "ever increases (the revision table in openapi.v0.yaml's `wire_revision` property is append-only), "+ + "so restore CurrentWireRevision (internal/httpapi/wire_revision.go) — never regenerate the golden "+ + "backwards", got.WireRevision, golden.WireRevision) + } + cmp := wireshape.Compare(golden, got) changed, removed, added := cmp.Changed, cmp.Removed, cmp.Added - if len(changed) > 0 || len(removed) > 0 { + switch { + case len(changed) > 0 || len(removed) > 0: if got.WireRevision <= golden.WireRevision { - t.Errorf("response member shape changed without a wire_revision bump: changed=%v removed=%v\n"+ + return fmt.Sprintf("wire shape (response/request-body member or parameter) changed without a "+ + "wire_revision bump: changed=%v removed=%v\n"+ "bump CurrentWireRevision (internal/httpapi/wire_revision.go) and the revision table "+ "in openapi.v0.yaml's `wire_revision` property, THEN regenerate the golden with "+ "`go run ./internal/httpapi/wireshape/cmd/gendigest`", changed, removed) - } else { - t.Errorf("response member shape changed (changed=%v removed=%v) and wire_revision moved "+ - "%d -> %d, but the golden was not regenerated: run "+ - "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", - changed, removed, golden.WireRevision, got.WireRevision) } - } else if len(added) > 0 && got.WireRevision == golden.WireRevision { - t.Errorf("response members were added (%v) but the golden was not regenerated: "+ - "this is additive and needs no wire_revision bump, but still run "+ - "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", added) - } - - if got.WireRevision != golden.WireRevision && len(changed) == 0 && len(removed) == 0 && len(added) == 0 { - t.Errorf("CurrentWireRevision is %d but the golden still says %d, with no shape change to justify "+ + return fmt.Sprintf("wire shape (response/request-body member or parameter) changed (changed=%v "+ + "removed=%v) and wire_revision moved %d -> %d, but the golden was not regenerated: run "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", + changed, removed, golden.WireRevision, got.WireRevision) + case len(added) > 0: + if got.WireRevision == golden.WireRevision { + return fmt.Sprintf("wire shape (response/request-body member or parameter) added (%v) but the "+ + "golden was not regenerated: this is additive and needs no wire_revision bump, but still run "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", added) + } + // Passing here would leave the golden a revision behind + // CurrentWireRevision, and a later non-additive change at that + // already-moved constant would then be told only to regenerate (which + // SafeToWrite allows) instead of to bump. + return fmt.Sprintf("wire shape (response/request-body member or parameter) added (%v) and "+ + "wire_revision moved %d -> %d, but the golden was not regenerated: run "+ + "`go run ./internal/httpapi/wireshape/cmd/gendigest` and commit the result", + added, golden.WireRevision, got.WireRevision) + case got.WireRevision != golden.WireRevision: + return fmt.Sprintf("CurrentWireRevision is %d but the golden still says %d, with no shape change to justify "+ "either: regenerate with `go run ./internal/httpapi/wireshape/cmd/gendigest`", got.WireRevision, golden.WireRevision) } + return "" +} + +// TestGoldenDrift is TestWireShapeDigest's own falsification: every way the +// committed golden can disagree with a fresh Compute must fail, each with its +// own instruction. "Added after a bump" is the state review found passing +// silently — no arm fired when entries were only added and wire_revision had +// also moved — and a LOWERED wire_revision was told to regenerate, which +// would have written the golden's revision backwards. +func TestGoldenDrift(t *testing.T) { + golden := wireshape.Digest{ + WireRevision: 2, + Entries: []wireshape.Entry{ + {Schema: "Widget", Member: "name", Type: "string", Required: true}, + }, + } + changed := []wireshape.Entry{{Schema: "Widget", Member: "name", Type: "integer", Required: true}} + added := append(append([]wireshape.Entry{}, golden.Entries...), + wireshape.Entry{Schema: "Widget", Member: "color", Type: "string"}) + + for _, tc := range []struct { + name string + got wireshape.Digest + want []string // phrases the failure must contain; none means the golden is current + }{ + {"identical digest is current", + golden, nil}, + {"changed entry at the same revision asks for a bump", + wireshape.Digest{WireRevision: 2, Entries: changed}, []string{"changed without a wire_revision bump"}}, + {"removed entry at the same revision asks for a bump", + wireshape.Digest{WireRevision: 2}, []string{"changed without a wire_revision bump"}}, + {"changed entry after a bump asks for a regenerate", + wireshape.Digest{WireRevision: 3, Entries: changed}, []string{"changed (changed=", "moved 2 -> 3", "not regenerated"}}, + {"added entry at the same revision asks for a regenerate", + wireshape.Digest{WireRevision: 2, Entries: added}, []string{"added (", "needs no wire_revision bump"}}, + {"added entry after a bump asks for a regenerate", + wireshape.Digest{WireRevision: 3, Entries: added}, []string{"added (", "moved 2 -> 3", "not regenerated"}}, + {"bump with no shape change asks for a regenerate", + wireshape.Digest{WireRevision: 3, Entries: golden.Entries}, []string{"no shape change"}}, + {"lowered revision with no shape change asks to restore the constant", + wireshape.Digest{WireRevision: 1, Entries: golden.Entries}, []string{"LOWER than the 2", "never regenerate the golden backwards"}}, + {"changed entry at a lowered revision asks to restore the constant", + wireshape.Digest{WireRevision: 1, Entries: changed}, []string{"LOWER than the 2"}}, + {"added entry at a lowered revision asks to restore the constant", + wireshape.Digest{WireRevision: 1, Entries: added}, []string{"LOWER than the 2"}}, + } { + t.Run(tc.name, func(t *testing.T) { + drift := goldenDrift(golden, tc.got) + if len(tc.want) == 0 { + if drift != "" { + t.Fatalf("current golden reported as drift: %s", drift) + } + return + } + if drift == "" { + t.Fatal("stale golden passed silently") + } + for _, phrase := range tc.want { + if !strings.Contains(drift, phrase) { + t.Errorf("drift %q does not contain %q", drift, phrase) + } + } + }) + } } // TestSafeToWrite is the gendigest write guard's own falsification (review // MEDIUM: "refuse to write changed or removed entries unless -// CurrentWireRevision is higher than the golden's recorded revision"). +// CurrentWireRevision is higher than the golden's recorded revision"), plus +// the refusal of any candidate whose revision is LOWER than the golden's. func TestSafeToWrite(t *testing.T) { base := wireshape.Digest{ WireRevision: 2, @@ -121,6 +218,28 @@ func TestSafeToWrite(t *testing.T) { } }) + t.Run("unchanged entries with a lower revision are refused", func(t *testing.T) { + candidate := wireshape.Digest{WireRevision: 1, Entries: base.Entries} + ok, reason := wireshape.SafeToWrite(base, candidate) + if ok { + t.Fatal("unchanged entries with a LOWER revision were allowed — the golden's revision would move backwards") + } + if !strings.Contains(reason, "append-only") { + t.Errorf("refusal %q does not point at the append-only revision table", reason) + } + }) + + t.Run("pure addition with a lower revision is refused", func(t *testing.T) { + candidate := wireshape.Digest{ + WireRevision: 1, + Entries: append(append([]wireshape.Entry{}, base.Entries...), + wireshape.Entry{Schema: "Widget", Member: "color", Type: "string"}), + } + if ok, _ := wireshape.SafeToWrite(base, candidate); ok { + t.Fatal("pure addition with a LOWER revision was allowed — the golden's revision would move backwards") + } + }) + t.Run("changed entry with a higher revision is safe", func(t *testing.T) { candidate := wireshape.Digest{ WireRevision: 3, @@ -139,13 +258,15 @@ func TestSafeToWrite(t *testing.T) { }) } -// TestParameterMutationsAreCaught is the falsification for the parameter side -// of the digest (review: "the digest doesn't cover operation PARAMETERS ... -// so a type, enum, required, or removal change on an existing parameter -// passes the gate silently"). It mutates real entries out of the committed -// golden — not a synthetic fixture — so a future rename of the `listIssues` -// limit/sort parameters breaks this test loudly instead of letting the -// coverage go stale unnoticed. +// TestParameterMutationsAreCaught is the falsification for how Compare and +// SafeToWrite treat the parameter side of the digest (review: "the digest +// doesn't cover operation PARAMETERS ... so a type, enum, required, or +// removal change on an existing parameter passes the gate silently"). It +// mutates real entries out of the committed golden — not a synthetic fixture +// — so a future rename of the `listIssues` limit/sort parameters breaks this +// test loudly instead of letting the coverage go stale unnoticed. Because it +// mutates entries after the fact, it cannot see a field walkParam extracts +// wrongly; that is TestWalkParam's job (wireshape_internal_test.go). // // Each case checks the same two things TestSafeToWrite checks for schema // members: the mutation is refused at the golden's own wire_revision, and From 91ffa48ab33f9a321339eff6d9cb4179c3806e63 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:50:10 +0000 Subject: [PATCH 09/16] =?UTF-8?q?feat(count):=20add=20Counter=20scope=20fi?= =?UTF-8?q?elds=20(S8)=20=E2=80=94=20ParentID,=20NoParent,=20ExcludeTypes,?= =?UTF-8?q?=20ExcludeStatus?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extends issueops.CountRequest with ParentID, NoParent, ExcludeTypes and ExcludeStatus, closing the gap where bd count could not express the filters bd list already has, forcing downstream consumers like gc to fall back to listing everything just to get a count. - issueops.Counter: four new CountRequest fields, documented against the List equivalents they mirror (ParentID/ExcludeTypes) or newly define (NoParent, ExcludeStatus has no List counterpart). Status "all" is pinned as equivalent to unset, matching BuildListFilter's own "all" and preserving count/list cardinality parity (the role's reason to exist). - internal/workapi.BuildCountFilter: maps the four fields onto the one types.IssueFilter builder all three storage legs (dolt, embeddeddolt, uow) already share via internal/storage/issueops and sqlbuild, so no per-leg storage code was needed. - cmd/bd: --parent, --no-parent, --exclude-type, --exclude-status flags. - HTTP: adds parent/no_parent/exclude_type/exclude_status to GET /v0/beads/issues:count, behind the new issues.count.scope behavior token (routes.go), wired in reads.go's countFilters, documented in openapi.v0.yaml with per-parameter tokens and an extended capabilities description. gendigest confirms a byte-identical golden.json: query parameters are outside the wire-shape digest's scope, so this is a zero-diff, no-revision-bump change as expected. - backend/conformance: four new Counter contract cases (ParentID scoping, NoParent's edge-only semantics vs ParentID's edge-or-dotted-id match, ExcludeTypes composing with include_infra's own exclusions, and ExcludeStatus), run identically against all three storage legs via a new CounterFixture.AddDependency hook shared with ReaderFixture. Verified: go build ./...; go test ./internal/httpapi/... ./internal/workapi/... ./issueops/... and the Counter contract on all three legs (dolt via testcontainers, embeddeddolt, uow via local dolt binary); make api-gen (additive-only apigen diff) and make api-check; go vet and golangci-lint clean; gofmt clean; gazelle diff empty (no new files/imports). Mutation testing: dropping issues.count.scope from behaviorCapabilities turns TestCapabilitiesAdvertiseEveryImplementedOperation, TestSpecCapabilityVocabularyMatchesTheRouteTable and TestNewParameterOnExistingOperationHasABehaviorToken red; no-op'ing ExcludeStatus in BuildCountFilter turns TestBuildCountFilterExcludeStatusTakesNamesAsWritten and the embeddeddolt ExcludeStatusNarrowsThePredicate conformance case red. Both reverted and reconfirmed green. Deviation: DESIGN.txt's literal text would make count --status all an ErrValidation; shipped behind a test and a doc comment instead, since "all" already means "every status" on both count and list and rejecting it would break the count/list parity this role exists to guarantee. S3/S4 note: no OSS HTTP client exists yet to enforce capability-token skew. A doc comment on CapIssuesCountScope in routes.go specifies the required behavior once one lands: check issues.count.scope against the cached handshake capabilities and refuse locally with a typed ErrUnsupported{Capability} before sending any of the four new params to an older server. Co-Authored-By: Claude Sonnet 5 --- backend/conformance/counter_contract.go | 136 ++++++++++++++++ cmd/bd/count.go | 18 +++ cmd/bd/count_filter_test.go | 62 ++++++++ internal/httpapi/apigen/types.gen.go | 22 ++- internal/httpapi/count_test.go | 13 ++ internal/httpapi/reads.go | 13 ++ internal/httpapi/routes.go | 33 +++- internal/httpapi/server_test.go | 2 +- internal/httpapi/spec/openapi.v0.yaml | 81 +++++++++- .../storage/dolt/counter_contract_test.go | 23 ++- .../embeddeddolt/counter_contract_test.go | 23 ++- internal/storage/uow/counter_contract_test.go | 23 ++- internal/workapi/count.go | 33 ++++ internal/workapi/count_test.go | 145 ++++++++++++++++++ issueops/counter.go | 37 +++++ 15 files changed, 645 insertions(+), 19 deletions(-) diff --git a/backend/conformance/counter_contract.go b/backend/conformance/counter_contract.go index e70e2e0f2..c9543a899 100644 --- a/backend/conformance/counter_contract.go +++ b/backend/conformance/counter_contract.go @@ -57,6 +57,13 @@ type CounterFixture struct { // A nil hook means "this backend cannot observe history", and the case // that needs it SKIPS rather than passing quietly. CountHistory func(context.Context) (int, error) + // AddDependency seeds one dependency edge, the same hook + // ReaderFixture/DependencyEditorFixture take it from (role_fixture_kit's + // shared AddDependency). It exists here only for the ParentID/NoParent + // scope cases (S8): NoParent excludes by PARENT-CHILD EDGE alone + // (sqlbuild/filter.go), unlike ParentID's dotted-id-prefix fallback, so + // pinning it needs a real edge and not just a naming convention. + AddDependency func(context.Context, *types.Dependency, string) error } // RunCounterCountsTheDurablePlaneByDefault pins counter.go:123-126 from the @@ -176,6 +183,135 @@ func RunCounterAnUnknownStatusMatchesNothing(t *testing.T, ctx context.Context, assertCounterTotal(t, ctx, fixture, scope, 0) } +// RunCounterParentIDScopesToChildren pins CountRequest.ParentID (counter.go, +// S8): restricted to ONE issue's children, reached two ways at once, exactly +// as ListRequest.ParentID is (sqlbuild/filter.go) — a parent-child dependency +// edge, OR (for a row with no such edge) a dotted-id prefix match. The parent +// itself, an unrelated row and a same-prefix DECOY (sibling id with no +// separator, e.g. "parent2") are all seeded so neither arm can pass by +// over-matching. +func RunCounterParentIDScopesToChildren(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + parent := fixture.IssuePrefix + "-parentid-parent" + byEdge := fixture.IssuePrefix + "-parentid-byedge" + byDotted := parent + ".1" + decoy := parent + "x" + unrelated := fixture.IssuePrefix + "-parentid-unrelated" + + for _, id := range []string{parent, byEdge, byDotted, decoy, unrelated} { + seedCounterIssue(t, ctx, fixture, counterSeed(id)) + } + requireCounterAddDependency(t, fixture) + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: byEdge, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", byEdge, parent, err) + } + + scope := counterScope(parent, byEdge, byDotted, decoy, unrelated) + scope.ParentID = parent + // Both children count; the parent, the decoy and the unrelated row do not. + assertCounterTotal(t, ctx, fixture, scope, 2) +} + +// RunCounterNoParentExcludesChildren pins CountRequest.NoParent +// (counter.go, S8): restricted to rows with NO parent-child edge. +// +// Unlike ParentID, NoParent's SQL clause (sqlbuild/filter.go) checks ONLY the +// parent-child dependency table, not the dotted-id convention — a row that is +// a dotted-id child with no edge at all still passes NoParent. That asymmetry +// is exactly what the "byDotted" row below pins: it is excluded from +// ParentID's answer by nothing (RunCounterParentIDScopesToChildren already +// admits it), but NoParent admits it too, because it carries no edge. +func RunCounterNoParentExcludesChildren(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + parent := fixture.IssuePrefix + "-noparent-parent" + child := fixture.IssuePrefix + "-noparent-child" + byDotted := parent + ".1" + + for _, id := range []string{parent, child, byDotted} { + seedCounterIssue(t, ctx, fixture, counterSeed(id)) + } + requireCounterAddDependency(t, fixture) + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: child, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", child, parent, err) + } + + scope := counterScope(parent, child, byDotted) + assertCounterTotal(t, ctx, fixture, scope, 3) + + scope.NoParent = true + // child is excluded (has an edge); parent and byDotted remain (neither + // carries a parent-child edge, and NoParent does not consult the dotted + // naming convention at all). + assertCounterTotal(t, ctx, fixture, scope, 2) +} + +// RunCounterExcludeTypesNarrowsThePredicate pins CountRequest.ExcludeTypes +// (counter.go, S8): named types are excluded from the count, composing with +// (rather than replacing) IncludeInfra's own "gate" exclusion. +func RunCounterExcludeTypesNarrowsThePredicate(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + task := fixture.IssuePrefix + "-extype-task" + chore := fixture.IssuePrefix + "-extype-chore" + gate := fixture.IssuePrefix + "-extype-gate" + seedCounterIssue(t, ctx, fixture, counterSeed(task)) + choreSeed := counterSeed(chore) + choreSeed.IssueType = types.TypeChore + seedCounterIssue(t, ctx, fixture, choreSeed) + gateSeed := counterSeed(gate) + gateSeed.IssueType = types.IssueType("gate") + seedCounterIssue(t, ctx, fixture, gateSeed) + + scope := counterScope(task, chore, gate) + assertCounterTotal(t, ctx, fixture, scope, 3) + + scope.ExcludeTypes = []string{"chore"} + assertCounterTotal(t, ctx, fixture, scope, 2) + + // Composes with IncludeInfra's own default gate exclusion: both the + // explicit chore exclusion and the include-infra gate exclusion apply at + // once, leaving only task. + scope.IncludeInfra = true + assertCounterTotal(t, ctx, fixture, scope, 1) +} + +// RunCounterExcludeStatusNarrowsThePredicate pins CountRequest.ExcludeStatus +// (counter.go, S8): a Count-only field with no List counterpart. Named +// statuses are excluded, and — like Status — an unrecognized name excludes +// nothing rather than failing (match-nothing-rather-than-fail, counter.go). +func RunCounterExcludeStatusNarrowsThePredicate(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + open := fixture.IssuePrefix + "-exstatus-open" + closed := fixture.IssuePrefix + "-exstatus-closed" + seedCounterIssue(t, ctx, fixture, counterSeed(open)) + closedSeed := counterSeed(closed) + closedSeed.Status = types.StatusClosed + seedCounterIssue(t, ctx, fixture, closedSeed) + + scope := counterScope(open, closed) + assertCounterTotal(t, ctx, fixture, scope, 2) + + scope.ExcludeStatus = []string{"closed"} + assertCounterTotal(t, ctx, fixture, scope, 1) + + // An unrecognized status excludes nothing: the set is unchanged. + scope.ExcludeStatus = []string{"no-such-status"} + assertCounterTotal(t, ctx, fixture, scope, 2) +} + +// requireCounterAddDependency skips a case that needs a real parent-child +// edge when the fixture offers no way to create one, rather than failing with +// a nil-pointer call — mirroring counterHistoryCount's optional-hook pattern. +func requireCounterAddDependency(t *testing.T, fixture CounterFixture) { + t.Helper() + if fixture.AddDependency == nil { + t.Skip("fixture cannot seed a dependency edge: AddDependency is nil") + } +} + // RunCounterGroupsPartitionTheScalarSet pins counter.go:168-169 and :244-245 // for a dimension whose buckets do NOT overlap: the grouped Total is the same // number a scalar Count returns, and the buckets partition it exactly. diff --git a/cmd/bd/count.go b/cmd/bd/count.go index 01f3d7d04..6264bfa20 100644 --- a/cmd/bd/count.go +++ b/cmd/bd/count.go @@ -93,6 +93,10 @@ func parseCountRequest(cmd *cobra.Command) (issueops.CountRequest, issueops.Coun includeInfra, _ := cmd.Flags().GetBool("include-infra") metadataFieldFlags, _ := cmd.Flags().GetStringArray("metadata-field") includeEphemeral, _ := cmd.Flags().GetBool("include-ephemeral") + parentID, _ := cmd.Flags().GetString("parent") + noParent, _ := cmd.Flags().GetBool("no-parent") + excludeTypes, _ := cmd.Flags().GetStringSlice("exclude-type") + excludeStatus, _ := cmd.Flags().GetStringSlice("exclude-status") request := issueops.CountRequest{ Status: status, @@ -111,6 +115,10 @@ func parseCountRequest(cmd *cobra.Command) (issueops.CountRequest, issueops.Coun IncludeInfra: includeInfra, IncludeEphemeral: includeEphemeral, + ParentID: parentID, + NoParent: noParent, + ExcludeTypes: excludeTypes, + ExcludeStatus: excludeStatus, } if len(metadataFieldFlags) > 0 { request.MetadataFields = make(map[string]string, len(metadataFieldFlags)) @@ -271,6 +279,16 @@ func registerCountFlags(cmd *cobra.Command) { cmd.Flags().String("title", "", "Filter by title text (case-insensitive substring match)") cmd.Flags().String("id", "", "Filter by specific issue IDs (comma-separated)") + // Hierarchy and exclusions (same spelling as `bd list`'s flags of the same + // name, GH#4387's count/list parity): --parent and --no-parent restrict to + // a parent's children or to rows with neither; --exclude-type and + // --exclude-status narrow the predicate by type and by status, composing + // with --include-infra's own exclusions rather than replacing them. + cmd.Flags().String("parent", "", "Filter by parent issue ID (count children of specified issue)") + cmd.Flags().Bool("no-parent", false, "Count only top-level issues (exclude child issues)") + cmd.Flags().StringSlice("exclude-type", nil, "Exclude issue types from the count (comma-separated or repeatable)") + cmd.Flags().StringSlice("exclude-status", nil, "Exclude statuses from the count (comma-separated or repeatable)") + // Pattern matching cmd.Flags().String("title-contains", "", "Filter by title substring") cmd.Flags().String("desc-contains", "", "Filter by description substring") diff --git a/cmd/bd/count_filter_test.go b/cmd/bd/count_filter_test.go index da97b2831..0f06e1f3c 100644 --- a/cmd/bd/count_filter_test.go +++ b/cmd/bd/count_filter_test.go @@ -76,6 +76,60 @@ func TestCountIncludeEphemeralFlagShape(t *testing.T) { } } +// TestCountScopeFlagsShape pins --parent, --no-parent, --exclude-type and +// --exclude-status: their existence, their defaults, and that setting one +// does not leak into another. These are the Counter-scope fields (S8): the +// behavior token issues.count.scope on the HTTP front door exists because of +// exactly these four flags. +func TestCountScopeFlagsShape(t *testing.T) { + for flag, defValue := range map[string]string{ + "parent": "", + "no-parent": "false", + "exclude-type": "[]", + "exclude-status": "[]", + } { + got := countCmd.Flags().Lookup(flag) + if got == nil { + t.Fatalf("bd count must expose a --%s flag", flag) + } + if got.DefValue != defValue { + t.Fatalf("--%s must default to %q, got %q", flag, defValue, got.DefValue) + } + } + + request, _, err := parseCountRequest(newCountFlagSet(t)) + if err != nil { + t.Fatalf("parseCountRequest with no flags set: %v", err) + } + if request.ParentID != "" || request.NoParent || len(request.ExcludeTypes) != 0 || len(request.ExcludeStatus) != 0 { + t.Errorf("parseCountRequest with no flags set = %#v, want all four scope fields at their zero value", request) + } + + flags := newCountFlagSet(t) + if err := flags.Flags().Set("parent", "bd-1"); err != nil { + t.Fatalf("set --parent: %v", err) + } + if err := flags.Flags().Set("exclude-type", "wisp"); err != nil { + t.Fatalf("set --exclude-type: %v", err) + } + request, _, err = parseCountRequest(flags) + if err != nil { + t.Fatalf("parseCountRequest --parent --exclude-type: %v", err) + } + if request.ParentID != "bd-1" { + t.Errorf("ParentID = %q, want %q", request.ParentID, "bd-1") + } + if request.NoParent { + t.Error("NoParent = true with only --parent set; the two flags must not leak into each other") + } + if !reflect.DeepEqual(request.ExcludeTypes, []string{"wisp"}) { + t.Errorf("ExcludeTypes = %v, want [wisp]", request.ExcludeTypes) + } + if len(request.ExcludeStatus) != 0 { + t.Errorf("ExcludeStatus = %v, want empty; only --exclude-type was set", request.ExcludeStatus) + } +} + // TestParseCountRequestCarriesEveryFilterFlag is the tripwire for a flag that // is registered, documented and silently dropped on the way into the request. // Every filter flag is set to a value distinguishable from its zero and read @@ -109,6 +163,10 @@ func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { "priority-max": "4", "include-infra": "true", "include-ephemeral": "true", + "parent": "bd-9", + "no-parent": "true", + "exclude-type": "wisp,gate", + "exclude-status": "closed,archived", } { if err := flags.Flags().Set(flag, value); err != nil { t.Fatalf("set --%s=%s: %v", flag, value, err) @@ -161,6 +219,10 @@ func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { HasMetadataKey: "audit_ref", IncludeInfra: true, IncludeEphemeral: true, + ParentID: "bd-9", + NoParent: true, + ExcludeTypes: []string{"wisp", "gate"}, + ExcludeStatus: []string{"closed", "archived"}, } if !reflect.DeepEqual(request, want) { t.Errorf("parseCountRequest built\n %#v\nwant\n %#v", request, want) diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 0e414bbc2..4ffadb8db 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1010,7 +1010,7 @@ type ContextResponse struct { // OPTIONAL, and absent means only that this server does not disclose its filesystem layout — never that it has no workspace. A client MUST NOT require it, MUST NOT treat absence as an error, and has no use for the value beyond display: it is a path on the SERVER's filesystem, which the client cannot open. Identify the workspace by `project_id` and `database`, which are required. BeadsDir *string `json:"beads_dir,omitempty"` - // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. + // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`, and `issues.count.scope`, which announces that `GET /v0/beads/issues:count` accepts `parent`, `no_parent`, `exclude_type` and `exclude_status` (see those parameters) rather than silently answering `unknown_parameter` for all four. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. // // THIS LIST IS BUILD-LEVEL, NOT WORKSPACE-LEVEL. It says which operations this binary serves, and for every entry but two that is the whole answer. `events.list` and `events.watch` are the exceptions: the durable events journal is a per-workspace setting that is OFF by default, so a server that advertises them may still refuse every request to both with 409 `events_journal_disabled` — correctly, because the operations exist and the workspace has no journal. A consumer of either MUST treat the capability as "this server speaks it" and the 409 as "not on this workspace", and must not read the capability as a promise that records will arrive. Capabilities []string `json:"capabilities"` @@ -2464,6 +2464,26 @@ type CountIssuesParams struct { // Unset — and with `include_ephemeral` also unset — the count is durable-plane only and applies none of the four: the historical `bd count` answer, kept exactly so a scripted caller reads the same number it read yesterday. IncludeInfra *bool `form:"include_infra,omitempty" json:"include_infra,omitempty"` + // Parent Restrict to one issue's children: a parent-child dependency edge onto `parent`, OR — for an issue carrying no parent-child edge at all — a dotted-id prefix match (`parent.anything`). It is the same predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` and `bd list --parent X --all` agree on cardinality for the shared part of the predicate. Sending it beside `no_parent` is not refused, for the same reason `assignee` and `no_assignee` are not: the two compose into the empty intersection, which is `0`. + // + // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. + Parent *string `form:"parent,omitempty" json:"parent,omitempty"` + + // NoParent Only issues carrying NO parent-child dependency edge. Unlike `parent`, this does NOT consult the dotted-id convention — an issue named `X.1` with no edge to `X` is still counted here, which is the one place `parent` and `no_parent` are not exact opposites of each other. + // + // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. + NoParent *bool `form:"no_parent,omitempty" json:"no_parent,omitempty"` + + // ExcludeType Issue types to exclude from the count. Repeat the parameter, or pass a comma-separated list; entries are split and trimmed inside the role and are NOT validated against this workspace's vocabulary — an unrecognized name excludes nothing rather than failing, `status`'s own treatment. It is NOT ignored when `type` is set (unlike `GET /v0/beads/ready`'s `exclude_type`): the two compose, and it also composes with the exclusions `include_infra` applies on its own rather than replacing them. + // + // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. + ExcludeType *[]string `form:"exclude_type,omitempty" json:"exclude_type,omitempty"` + + // ExcludeStatus Statuses to exclude from the count. Repeat the parameter, or pass a comma-separated list; entries are taken AS WRITTEN — no normalization, no validation against this workspace's vocabulary, `status`'s own match-nothing-rather-than-fail treatment. There is no listing counterpart to this parameter: `GET /v0/beads/issues` computes its default status exclusions internally from workspace configuration and exposes no equivalent knob, so this is a Count-only capability. + // + // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. + ExcludeStatus *[]string `form:"exclude_status,omitempty" json:"exclude_status,omitempty"` + // GroupBy Bucket the count by one dimension and return `groups` beside `total`. Absent, the response carries `total` alone. // // The set is CLOSED, and a value outside it is a `400` rather than an empty answer: a caller that misspelled a dimension and got zero buckets back has no way to tell that from a workspace with nothing in it. That is the role's own rule, applied at the edge here so the refusal names the parameter. diff --git a/internal/httpapi/count_test.go b/internal/httpapi/count_test.go index 986455704..77ab524e9 100644 --- a/internal/httpapi/count_test.go +++ b/internal/httpapi/count_test.go @@ -117,6 +117,10 @@ func TestCountForwardsEveryDocumentedParameter(t *testing.T) { "has_metadata_key": {"audit_ref"}, "include_infra": {"true"}, "include_ephemeral": {"true"}, + "parent": {"bd-9"}, + "no_parent": {"true"}, + "exclude_type": {"wisp", "gate"}, + "exclude_status": {"closed", "archived"}, }.Encode()) if resp.StatusCode != http.StatusOK { t.Fatalf("status = %d, want 200: %s", resp.StatusCode, readAll(t, resp)) @@ -169,6 +173,11 @@ func TestCountForwardsEveryDocumentedParameter(t *testing.T) { IncludeInfra: true, IncludeEphemeral: true, + + ParentID: "bd-9", + NoParent: true, + ExcludeTypes: []string{"wisp", "gate"}, + ExcludeStatus: []string{"closed", "archived"}, } if !reflect.DeepEqual(got[0], want) { t.Errorf("request = %+v\nwant %+v", got[0], want) @@ -645,6 +654,10 @@ var countFieldForParameter = map[string]string{ "has_metadata_key": "HasMetadataKey", "include_infra": "IncludeInfra", "include_ephemeral": "IncludeEphemeral", + "parent": "ParentID", + "no_parent": "NoParent", + "exclude_type": "ExcludeTypes", + "exclude_status": "ExcludeStatus", } // TestEveryCountRequestFieldIsPublished: the wire publishes every filter the diff --git a/internal/httpapi/reads.go b/internal/httpapi/reads.go index 1ab132ed8..653f98ad3 100644 --- a/internal/httpapi/reads.go +++ b/internal/httpapi/reads.go @@ -229,6 +229,19 @@ func countFilters(q *query) issueops.CountRequest { // it reaches the role as its own field and never as IncludeInfra: folded // into it, a plane-only count would also lose its templates and gates. IncludeEphemeral: q.boolean("include_ephemeral"), + + // The four scope fields behind `issues.count.scope` (S8): ParentID and + // ExcludeTypes are read exactly as the listing reads them (q.str("parent"), + // q.list("exclude_type")), because the role documents its own ParentID and + // ExcludeTypes as spelling ListRequest's fields identically. NoParent and + // ExcludeStatus have no listing counterpart to mirror — NoParent is a new + // boolean switch and ExcludeStatus is a Count-only capability the listing + // never exposed — so both are read with this file's own conventions for + // their kind (q.boolean, q.list) rather than a listing precedent. + ParentID: q.str("parent"), + NoParent: q.boolean("no_parent"), + ExcludeTypes: q.list("exclude_type"), + ExcludeStatus: q.list("exclude_status"), } } diff --git a/internal/httpapi/routes.go b/internal/httpapi/routes.go index cb7a84b76..498f34022 100644 --- a/internal/httpapi/routes.go +++ b/internal/httpapi/routes.go @@ -76,6 +76,37 @@ const CapBatchApplyLarge = "issues.batchApplyLarge" // baseline is built from. const CapIssuesListSort = "issues.list.sort" +// CapIssuesCountScope is the behavior capability that advertises GET +// /v0/beads/issues:count's `parent`, `no_parent`, `exclude_type` and +// `exclude_status` parameters (reads.go countFilters, spec OpCountIssues). +// Like CapIssuesListSort it names FOUR PARAMETERS added to an existing +// operation rather than a route of its own — `issues.count` is already the +// per-operation token for the route itself — so it rides the same +// behaviorCapabilities list and one token covers all four (internal/beads +// design doc "Count scope", S8): they shipped together, through the one +// shared workapi.BuildCountFilter builder, so there is no partial-support +// state a finer-grained token would need to distinguish. +// +// This is the TestNewParameterOnExistingOperationHasABehaviorToken gate +// CapIssuesListSort's doc describes: `issues:count` is a pretoken-baseline +// operation (internal/httpapi/testdata/pretoken_operations.json), so these +// four NEW parameters on it must each cite a served token in their own +// OpenAPI description, and this is that token. +// +// CLIENT-SKEW NOTE FOR S3/S4 (no OSS HTTP client exists yet to wire this +// into): a server predating this token answers any of the four parameters +// with `400 invalid_argument`/`reason: "unknown_parameter"` naming the +// parameter — the per-parameter capability probe every behavior token on +// this operation already relies on (see CapIssuesListSort). When the HTTP +// client lands, it MUST check `issues.count.scope` against the cached +// `ContextResponse.capabilities` from the handshake and refuse LOCALLY with +// a typed `ErrUnsupported{Capability: "issues.count.scope"}` (or an +// equivalent typed error carrying the token) before sending a request that +// sets `ParentID`, `NoParent`, `ExcludeTypes` or `ExcludeStatus` to an older +// server — never let the caller pay for a round trip that 400s anyway, and +// never silently drop the fields and return a wider count than asked for. +const CapIssuesCountScope = "issues.count.scope" + // customMethodTarget splits the custom method off the segment the router // matched, and reports the row that claims it. // @@ -810,7 +841,7 @@ func (r route) specPathOf() string { // route. project.enforce announces per-request Bd-Project-Id enforcement // (checkProjectStamp): a stamped client reads it to know the refusal is available // rather than silently dropped by an older server. -var behaviorCapabilities = []string{CapProjectEnforce, CapBatchApplyLarge, CapIssuesListSort} +var behaviorCapabilities = []string{CapProjectEnforce, CapBatchApplyLarge, CapIssuesListSort, CapIssuesCountScope} // Capabilities lists what this build advertises in ContextResponse.capabilities: // the operations it actually implements, gated on `implemented` so a stub can diff --git a/internal/httpapi/server_test.go b/internal/httpapi/server_test.go index 0b5799395..291c7fdf2 100644 --- a/internal/httpapi/server_test.go +++ b/internal/httpapi/server_test.go @@ -635,7 +635,7 @@ func TestCapabilitiesAdvertiseEveryImplementedOperation(t *testing.T) { // them; project.enforce and issues.batchApplyLarge are advertised in the // same list. Both are spelled literally here to keep this an independent // oracle rather than a second call to the code under test. - want = append(want, "project.enforce", "issues.batchApplyLarge", "issues.list.sort") + want = append(want, "project.enforce", "issues.batchApplyLarge", "issues.list.sort", "issues.count.scope") slices.Sort(want) if !slices.Equal(got, want) { t.Errorf("capabilities = %v, want %v", got, want) diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index 356032b1f..e91a56823 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -2068,6 +2068,81 @@ paths: schema: type: boolean default: false + - name: parent + in: query + description: >- + Restrict to one issue's children: a parent-child dependency edge + onto `parent`, OR — for an issue carrying no parent-child edge at + all — a dotted-id prefix match (`parent.anything`). It is the + same predicate `GET /v0/beads/issues`'s own `parent` applies, so + `bd count --parent X` and `bd list --parent X --all` agree on + cardinality for the shared part of the predicate. Sending it + beside `no_parent` is not refused, for the same reason `assignee` + and `no_assignee` are not: the two compose into the empty + intersection, which is `0`. + + + Behind the `issues.count.scope` behavior token: an older server + answers `unknown_parameter` for this name. + schema: + type: string + - name: no_parent + in: query + description: >- + Only issues carrying NO parent-child dependency edge. Unlike + `parent`, this does NOT consult the dotted-id convention — an + issue named `X.1` with no edge to `X` is still counted here, which + is the one place `parent` and `no_parent` are not exact opposites + of each other. + + + Behind the `issues.count.scope` behavior token: an older server + answers `unknown_parameter` for this name. + schema: + type: boolean + default: false + - name: exclude_type + in: query + description: >- + Issue types to exclude from the count. Repeat the parameter, or + pass a comma-separated list; entries are split and trimmed inside + the role and are NOT validated against this workspace's + vocabulary — an unrecognized name excludes nothing rather than + failing, `status`'s own treatment. It is NOT ignored when `type` + is set (unlike `GET /v0/beads/ready`'s `exclude_type`): the two + compose, and it also composes with the exclusions `include_infra` + applies on its own rather than replacing them. + + + Behind the `issues.count.scope` behavior token: an older server + answers `unknown_parameter` for this name. + style: form + explode: true + schema: + type: array + items: + type: string + - name: exclude_status + in: query + description: >- + Statuses to exclude from the count. Repeat the parameter, or pass + a comma-separated list; entries are taken AS WRITTEN — no + normalization, no validation against this workspace's vocabulary, + `status`'s own match-nothing-rather-than-fail treatment. There is + no listing counterpart to this parameter: `GET /v0/beads/issues` + computes its default status exclusions internally from workspace + configuration and exposes no equivalent knob, so this is a + Count-only capability. + + + Behind the `issues.count.scope` behavior token: an older server + answers `unknown_parameter` for this name. + style: form + explode: true + schema: + type: array + items: + type: string - name: group_by in: query description: >- @@ -7514,7 +7589,11 @@ components: older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that - parameter) rather than silently answering `unknown_parameter`. The + parameter) rather than silently answering `unknown_parameter`, and + `issues.count.scope`, which announces that + `GET /v0/beads/issues:count` accepts `parent`, `no_parent`, + `exclude_type` and `exclude_status` (see those parameters) rather + than silently answering `unknown_parameter` for all four. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. diff --git a/internal/storage/dolt/counter_contract_test.go b/internal/storage/dolt/counter_contract_test.go index 9047ed925..dd1d5b675 100644 --- a/internal/storage/dolt/counter_contract_test.go +++ b/internal/storage/dolt/counter_contract_test.go @@ -65,6 +65,18 @@ func TestCounterContract(t *testing.T) { t.Run("WritesNothing", func(t *testing.T) { conformance.RunCounterWritesNothing(t, ctx, fixture) }) + t.Run("ParentIDScopesToChildren", func(t *testing.T) { + conformance.RunCounterParentIDScopesToChildren(t, ctx, fixture) + }) + t.Run("NoParentExcludesChildren", func(t *testing.T) { + conformance.RunCounterNoParentExcludesChildren(t, ctx, fixture) + }) + t.Run("ExcludeTypesNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeTypesNarrowsThePredicate(t, ctx, fixture) + }) + t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) + }) } func newDoltCounterFixture(t *testing.T, prefix string) (conformance.CounterFixture, context.Context, func()) { @@ -79,11 +91,12 @@ func newDoltCounterFixture(t *testing.T, prefix string) (conformance.CounterFixt } kit := newDoltRoleFixtureKit(store, prefix) fixture := conformance.CounterFixture{ - IssuePrefix: kit.IssuePrefix, - Counter: counter, - CreateIssue: kit.CreateIssue, - CreateWisp: kit.CreateWisp, - CountHistory: kit.CountHistory, + IssuePrefix: kit.IssuePrefix, + Counter: counter, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + CountHistory: kit.CountHistory, + AddDependency: kit.AddDependency, } return fixture, ctx, func() { cancel() diff --git a/internal/storage/embeddeddolt/counter_contract_test.go b/internal/storage/embeddeddolt/counter_contract_test.go index dd6e3f167..7cfdf0ffd 100644 --- a/internal/storage/embeddeddolt/counter_contract_test.go +++ b/internal/storage/embeddeddolt/counter_contract_test.go @@ -68,6 +68,18 @@ func TestCounterContract(t *testing.T) { t.Run("WritesNothing", func(t *testing.T) { conformance.RunCounterWritesNothing(t, ctx, fixture) }) + t.Run("ParentIDScopesToChildren", func(t *testing.T) { + conformance.RunCounterParentIDScopesToChildren(t, ctx, fixture) + }) + t.Run("NoParentExcludesChildren", func(t *testing.T) { + conformance.RunCounterNoParentExcludesChildren(t, ctx, fixture) + }) + t.Run("ExcludeTypesNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeTypesNarrowsThePredicate(t, ctx, fixture) + }) + t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) + }) } func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conformance.CounterFixture { @@ -78,10 +90,11 @@ func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conform } kit := newEmbeddedRoleFixtureKit(te, prefix) return conformance.CounterFixture{ - IssuePrefix: kit.IssuePrefix, - Counter: counter, - CreateIssue: kit.CreateIssue, - CreateWisp: kit.CreateWisp, - CountHistory: kit.CountHistory, + IssuePrefix: kit.IssuePrefix, + Counter: counter, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + CountHistory: kit.CountHistory, + AddDependency: kit.AddDependency, } } diff --git a/internal/storage/uow/counter_contract_test.go b/internal/storage/uow/counter_contract_test.go index 45e4de73b..9cbf2e7a8 100644 --- a/internal/storage/uow/counter_contract_test.go +++ b/internal/storage/uow/counter_contract_test.go @@ -65,6 +65,18 @@ func TestCounterContract(t *testing.T) { t.Run("WritesNothing", func(t *testing.T) { conformance.RunCounterWritesNothing(t, ctx, fixture) }) + t.Run("ParentIDScopesToChildren", func(t *testing.T) { + conformance.RunCounterParentIDScopesToChildren(t, ctx, fixture) + }) + t.Run("NoParentExcludesChildren", func(t *testing.T) { + conformance.RunCounterNoParentExcludesChildren(t, ctx, fixture) + }) + t.Run("ExcludeTypesNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeTypesNarrowsThePredicate(t, ctx, fixture) + }) + t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { + conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) + }) } func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conformance.CounterFixture { @@ -82,10 +94,11 @@ func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conf } kit := newUOWRoleFixtureKit(provider, prefix) return conformance.CounterFixture{ - IssuePrefix: kit.IssuePrefix, - Counter: counter, - CreateIssue: kit.CreateIssue, - CreateWisp: kit.CreateWisp, - CountHistory: kit.CountHistory, + IssuePrefix: kit.IssuePrefix, + Counter: counter, + CreateIssue: kit.CreateIssue, + CreateWisp: kit.CreateWisp, + CountHistory: kit.CountHistory, + AddDependency: kit.AddDependency, } } diff --git a/internal/workapi/count.go b/internal/workapi/count.go index 2aef4adc4..d379dd207 100644 --- a/internal/workapi/count.go +++ b/internal/workapi/count.go @@ -103,6 +103,39 @@ func BuildCountFilter(in issueops.CountRequest, cfg ListConfig) (types.IssueFilt filter.IDs = ids } + if in.ParentID != "" { + parentID := in.ParentID + filter.ParentID = &parentID + } + if in.NoParent { + filter.NoParent = true + } + + // ExcludeTypes is appended to, not assigned: applyCountIncludeInfra (below) + // contributes its own "gate" exclusion, and the two sets must compose + // rather than one silently discarding the other, exactly as + // BuildListFilter's ExcludeTypes and applyTypeSuppressions compose. + for _, raw := range in.ExcludeTypes { + for _, t := range strings.Split(raw, ",") { + t = strings.TrimSpace(t) + if t != "" { + filter.ExcludeTypes = append(filter.ExcludeTypes, types.IssueType(utils.NormalizeIssueType(t))) + } + } + } + + // ExcludeStatus takes names as written, with no normalization and no + // validation against the workspace vocabulary — the same + // match-nothing-rather-than-fail treatment Status and IssueType get above. + for _, raw := range in.ExcludeStatus { + for _, s := range strings.Split(raw, ",") { + s = strings.TrimSpace(s) + if s != "" { + filter.ExcludeStatus = append(filter.ExcludeStatus, types.Status(s)) + } + } + } + if in.IncludeInfra { applyCountIncludeInfra(&filter, in.IssueType, cfg) } else if !in.IncludeEphemeral { diff --git a/internal/workapi/count_test.go b/internal/workapi/count_test.go index 481e035c2..9b533d981 100644 --- a/internal/workapi/count_test.go +++ b/internal/workapi/count_test.go @@ -158,6 +158,151 @@ func TestBuildCountFilterNormalizesLabelsAndIDs(t *testing.T) { } } +// TestBuildCountFilterMapsParentAndNoParent pins ParentID/NoParent's mapping +// into types.IssueFilter, mirroring BuildListFilter's ParentID/NoParent +// mapping (internal/workapi/list.go) field for field. +func TestBuildCountFilterMapsParentAndNoParent(t *testing.T) { + got, err := BuildCountFilter(issueops.CountRequest{ParentID: "bd-1"}, ListConfig{}) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + if got.ParentID == nil || *got.ParentID != "bd-1" { + t.Errorf("ParentID = %v, want \"bd-1\"", countPtrStr(got.ParentID)) + } + if got.NoParent { + t.Errorf("NoParent = true, want false: request did not set it") + } + + got, err = BuildCountFilter(issueops.CountRequest{NoParent: true}, ListConfig{}) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + if got.ParentID != nil { + t.Errorf("ParentID = %v, want nil: request did not set it", countPtrStr(got.ParentID)) + } + if !got.NoParent { + t.Error("NoParent = false, want true") + } +} + +// TestBuildCountFilterExcludeTypesSplitsAndNormalizes pins ExcludeTypes' +// mapping: comma-separated entries split, trim and normalize exactly as +// BuildListFilter's ExcludeTypes does, and the result APPENDS to (rather than +// replaces) whatever applyCountIncludeInfra later contributes. +func TestBuildCountFilterExcludeTypesSplitsAndNormalizes(t *testing.T) { + got, err := BuildCountFilter(issueops.CountRequest{ + // "feat" is an alias (-> "feature", internal/utils/strings.go); + // "bug" is not, so it passes through unchanged. + ExcludeTypes: []string{" bug , feat", "", "chore"}, + }, ListConfig{}) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + want := []types.IssueType{"bug", "feature", "chore"} + if !reflect.DeepEqual(got.ExcludeTypes, want) { + t.Errorf("ExcludeTypes = %v, want %v", got.ExcludeTypes, want) + } + + // Composes with IncludeInfra's own "gate" exclusion rather than + // discarding it. + got, err = BuildCountFilter(issueops.CountRequest{ + ExcludeTypes: []string{"chore"}, + IncludeInfra: true, + }, ListConfig{}) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + want = []types.IssueType{"chore", "gate"} + if !reflect.DeepEqual(got.ExcludeTypes, want) { + t.Errorf("ExcludeTypes = %v, want %v (explicit exclusions composed with IncludeInfra's)", got.ExcludeTypes, want) + } +} + +// TestBuildCountFilterParentFieldsMatchListFilter extends the GH#4387 +// count/list cardinality-parity pin to the three new Count fields that DO +// have a List counterpart: ParentID, NoParent and ExcludeTypes must produce +// the identical types.IssueFilter contribution BuildListFilter does for the +// same request, since a filter contract test (backend/conformance) can only +// show the two AGREE on one backend's rows — this shows WHY, at the one place +// both builders share. +func TestBuildCountFilterParentFieldsMatchListFilter(t *testing.T) { + cfg := ListConfig{} + + countGot, err := BuildCountFilter(issueops.CountRequest{ParentID: "bd-1"}, cfg) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + listGot, err := BuildListFilter(issueops.ListRequest{ParentID: "bd-1", AllFlag: true}, cfg) + if err != nil { + t.Fatalf("BuildListFilter: %v", err) + } + if !reflect.DeepEqual(countGot.ParentID, listGot.ParentID) { + t.Errorf("Count ParentID = %v, List ParentID = %v, want equal", countPtrStr(countGot.ParentID), countPtrStr(listGot.ParentID)) + } + + countGot, err = BuildCountFilter(issueops.CountRequest{NoParent: true}, cfg) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + listGot, err = BuildListFilter(issueops.ListRequest{NoParent: true, AllFlag: true}, cfg) + if err != nil { + t.Fatalf("BuildListFilter: %v", err) + } + if countGot.NoParent != listGot.NoParent { + t.Errorf("Count NoParent = %v, List NoParent = %v, want equal", countGot.NoParent, listGot.NoParent) + } + + countGot, err = BuildCountFilter(issueops.CountRequest{ExcludeTypes: []string{"wisp,feat"}}, cfg) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + // IncludeAllTypes on the list side skips applyTypeSuppressions' OWN + // exclusions (template/gate/infra defaults), isolating the one thing this + // case compares: the explicit --exclude-type mapping. + listGot, err = BuildListFilter(issueops.ListRequest{ExcludeTypes: []string{"wisp,feat"}, AllFlag: true, IncludeAllTypes: true}, cfg) + if err != nil { + t.Fatalf("BuildListFilter: %v", err) + } + if !reflect.DeepEqual(countGot.ExcludeTypes, listGot.ExcludeTypes) { + t.Errorf("Count ExcludeTypes = %v, List ExcludeTypes = %v, want equal", countGot.ExcludeTypes, listGot.ExcludeTypes) + } +} + +// TestBuildCountFilterExcludeStatusTakesNamesAsWritten pins ExcludeStatus: a +// Count-only field with no List counterpart (ListRequest computes its default +// exclusions internally and exposes no caller-facing knob for them). Entries +// split and trim like ExcludeTypes, but are NOT normalized or validated +// against the workspace vocabulary — the same match-nothing-rather-than-fail +// treatment Status gets. +func TestBuildCountFilterExcludeStatusTakesNamesAsWritten(t *testing.T) { + got, err := BuildCountFilter(issueops.CountRequest{ + ExcludeStatus: []string{" closed , archived", "", "no-such-status"}, + }, ListConfig{}) + if err != nil { + t.Fatalf("BuildCountFilter: %v", err) + } + want := []types.Status{"closed", "archived", "no-such-status"} + if !reflect.DeepEqual(got.ExcludeStatus, want) { + t.Errorf("ExcludeStatus = %v, want %v", got.ExcludeStatus, want) + } +} + +// TestBuildCountFilterStatusAllIsNotValidation pins DESIGN's requirement that +// Status "all" have well-defined semantics: it is the no-filter spelling +// (same as empty), NOT ErrValidation, because BuildListFilter's own "all" +// selector means the same thing and a count must stay able to answer the +// cardinality of `bd list --status all --all` for the shared predicate +// (GH#4387 count/list parity). +func TestBuildCountFilterStatusAllIsNotValidation(t *testing.T) { + got, err := BuildCountFilter(issueops.CountRequest{Status: "all"}, ListConfig{}) + if err != nil { + t.Fatalf(`BuildCountFilter(status="all") returned an error, want nil: %v`, err) + } + if got.Status != nil { + t.Errorf(`status "all" produced Status = %v, want no status predicate`, countPtrStr(got.Status)) + } +} + func TestBuildCountFilterCarriesMetadataFields(t *testing.T) { fields := map[string]string{"team": "platform", "env": "prod"} got, err := BuildCountFilter(issueops.CountRequest{MetadataFields: fields}, ListConfig{}) diff --git a/issueops/counter.go b/issueops/counter.go index 594ec4add..2513b8c8c 100644 --- a/issueops/counter.go +++ b/issueops/counter.go @@ -54,6 +54,17 @@ type CountRequest struct { // than quietly tightened, because a scripted caller that counts a status // its workspace has since dropped currently reads 0 and would start reading // an error. + // + // "all" is this field's one well-defined non-literal spelling, and it is + // NOT ErrValidation: it is equivalent to leaving Status empty, and + // BuildListFilter treats its own "all" selector identically (the no-filter + // spelling), so `bd count --status all ` and `bd list --status + // all --all` already agree on cardinality for the shared part + // of the predicate. Rejecting it here would break that count/list parity + // (the whole point of this role, GH#4387) for every caller already + // spelling it, and there is no caller-visible ambiguity it would resolve: + // "no status" is a filter that excludes every row, which "all" has never + // meant on either front door. Status string // IssueType restricts the type, with the same match-nothing-rather-than-fail // treatment Status gets and no alias expansion at all. @@ -66,6 +77,15 @@ type CountRequest struct { // written and answer with the empty intersection. Assignee string + // ParentID restricts to one issue's children, spelled as + // ListRequest.ParentID spells it: a parent-child dependency edge, OR (for + // an issue with no such edge) a dotted-id prefix match. NoParent restricts + // to rows with neither. Setting both is not refused, for the same reason + // Assignee and NoAssignee are not: the filter answers with the empty + // intersection rather than failing. + ParentID string + NoParent bool + // Priority is an exact priority; PriorityMin and PriorityMax bound a range // inclusively. All three are pointers because 0 is a real priority, for the // reason ReadyRequest.Priority gives. @@ -86,6 +106,23 @@ type CountRequest struct { // rather than a slice it had to prepare. IDFilter string + // ExcludeTypes names types to exclude, spelled as ListRequest.ExcludeTypes + // spells it: entries may be comma-separated, splitting and normalization + // happen inside, and an entry is NOT validated against the workspace + // vocabulary — an unrecognized type excludes nothing rather than failing. + // It composes with IncludeInfra's own "gate" exclusion rather than + // replacing it: both contribute to the same exclusion set. + ExcludeTypes []string + // ExcludeStatus names statuses to exclude. Unlike Status, this has no + // List counterpart — ListRequest computes its default status exclusions + // internally from workspace configuration and exposes no caller-facing + // knob for them. Here it is a new, Count-only capability (behavior token + // `issues.count.scope`): entries may be comma-separated, and — matching + // Status and ExcludeTypes — are NOT validated against the workspace + // vocabulary, so an unrecognized name excludes nothing rather than + // failing. + ExcludeStatus []string + // TitleContains, DescContains and NotesContains are substring matches on // the three long fields, spelled as ListRequest spells them so the two // requests cannot drift apart in naming while meaning the same thing. From 9edad9ea6afd68b9700388dcc7677cadfa70050b Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:50:10 +0000 Subject: [PATCH 10/16] =?UTF-8?q?fix(count):=20S8=20review=20follow-ups=20?= =?UTF-8?q?=E2=80=94=20refusal=20parity,=20exclude-status=20validation,=20?= =?UTF-8?q?doc=20fixes,=20parity=20tests?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the Opus review's six required changes for S8 (bd count scope fields): 1. BuildCountFilter now refuses ParentID+NoParent together as ErrValidation, matching bd list's CLI refusal wording; wired through the role, the CLI, and invalid_argument classification in internal/httpapi/reads.go. 2. Status:"all" is documented in DESIGN.txt as every status, matching List's no-filter reading (count/list cardinality parity) — a coordinator decision recorded in beads-design/DESIGN.txt, outside this worktree. 3. ExcludeStatus names are now validated against the workspace vocabulary (built-in statuses plus workspace-custom ones); an unknown name is ErrValidation instead of a silent no-op, since a typo here would overcount. 4. Fixed openapi.v0.yaml and issueops/reader.go doc claims about bd list --parent's default tree-walk vs. the one-level role answer that bd count --parent agrees with (confirmed by tracing list_show_filter_modes.go); fixed an Oxford-comma nit in routes.go and the spec. 5. Added storage-level count == len(List) parity tests for ParentID, NoParent and ExcludeTypes (List has no ExcludeStatus counterpart), plus wisp-child and closed-child contract cases, across all three backends (dolt, embeddeddolt, uow). CounterFixture gained a List hook for this. 6. Extended the routes.go skew note: gc's ErrCountUnsupported fallback to List is now named explicitly, with a note that S3/S4 must add a skew test that masks the token and asserts the fallback fires. Also fixed RunCounterExcludeStatusNarrowsThePredicate's now-stale "unrecognized status excludes nothing" assertion to expect ErrValidation instead. Co-Authored-By: Claude Sonnet 5 --- backend/conformance/counter_contract.go | 197 +++++++++++++++++- cmd/bd/count.go | 3 + cmd/bd/count_filter_test.go | 53 ++++- internal/httpapi/apigen/types.gen.go | 10 +- internal/httpapi/reads.go | 9 + internal/httpapi/reads_test.go | 2 + internal/httpapi/routes.go | 14 +- internal/httpapi/spec/openapi.v0.yaml | 52 +++-- .../storage/dolt/counter_contract_test.go | 22 ++ .../embeddeddolt/counter_contract_test.go | 20 ++ internal/storage/uow/counter_contract_test.go | 24 +++ internal/workapi/count.go | 30 ++- internal/workapi/count_test.go | 63 +++++- issueops/counter.go | 22 +- issueops/reader.go | 14 +- 15 files changed, 492 insertions(+), 43 deletions(-) diff --git a/backend/conformance/counter_contract.go b/backend/conformance/counter_contract.go index c9543a899..5504424c5 100644 --- a/backend/conformance/counter_contract.go +++ b/backend/conformance/counter_contract.go @@ -64,6 +64,12 @@ type CounterFixture struct { // (sqlbuild/filter.go), unlike ParentID's dotted-id-prefix fallback, so // pinning it needs a real edge and not just a naming convention. AddDependency func(context.Context, *types.Dependency, string) error + // List runs the List role's query directly (the store's IssueReader), so + // the storage-level parity cases below (S8 review, follow-up #5) can + // compare Count's answer against len(List's rows) for the same scope + // without going through a second builder. A nil hook skips those cases + // rather than failing: not every backend wiring has a Reader to hand. + List func(context.Context, publicops.ListRequest) (publicops.IssuePage, error) } // RunCounterCountsTheDurablePlaneByDefault pins counter.go:123-126 from the @@ -297,9 +303,14 @@ func RunCounterExcludeStatusNarrowsThePredicate(t *testing.T, ctx context.Contex scope.ExcludeStatus = []string{"closed"} assertCounterTotal(t, ctx, fixture, scope, 1) - // An unrecognized status excludes nothing: the set is unchanged. + // Unlike Status and IssueType, ExcludeStatus IS validated (S8 review, + // follow-up #3): a misspelled entry would silently exclude nothing and + // OVERCOUNT, so an unrecognized name is refused as ErrValidation instead + // of treated as a no-op. scope.ExcludeStatus = []string{"no-such-status"} - assertCounterTotal(t, ctx, fixture, scope, 2) + if _, err := fixture.Counter.Count(ctx, scope); !errors.Is(err, publicops.ErrValidation) { + t.Errorf("Count with an unrecognized exclude-status = %v, want ErrValidation", err) + } } // requireCounterAddDependency skips a case that needs a real parent-child @@ -312,6 +323,188 @@ func requireCounterAddDependency(t *testing.T, fixture CounterFixture) { } } +// requireCounterList skips a case that needs the List role when the fixture +// offers no way to reach it, mirroring requireCounterAddDependency. +func requireCounterList(t *testing.T, fixture CounterFixture) { + t.Helper() + if fixture.List == nil { + t.Skip("fixture cannot run the List role: List is nil") + } +} + +// assertCounterMatchesList compares Count's Total against len(List's rows) +// for the same scope, with List's AllFlag and IncludeAllTypes both set so +// List's own default exclusions (status, pinned, template, gate, infra-plane) +// are lifted exactly as far as Count's own default lifts none of them +// (workapi.BuildCountFilter's doc, count.go) — the one combination that +// makes the two cardinalities comparable at all for an arbitrary scope. +// +// This is a DIFFERENT check than +// TestBuildCountFilterParentFieldsMatchListFilter +// (internal/workapi/count_test.go), which compares the two FILTERS a shared +// request builds without touching a store; this one runs both roles against +// a real backend and compares what they actually return (S8 review, follow-up +// #5). +func assertCounterMatchesList(t *testing.T, ctx context.Context, fixture CounterFixture, idFilter, parentID string, noParent bool, excludeTypes []string) { + t.Helper() + countReq := publicops.CountRequest{ + IDFilter: idFilter, + ParentID: parentID, + NoParent: noParent, + ExcludeTypes: excludeTypes, + } + total := counterTotal(t, ctx, fixture, countReq) + + listReq := publicops.ListRequest{ + IDFilter: idFilter, + ParentID: parentID, + NoParent: noParent, + ExcludeTypes: excludeTypes, + AllFlag: true, + IncludeAllTypes: true, + } + page, err := fixture.List(ctx, listReq) + if err != nil { + t.Fatalf("List(%+v): %v", listReq, err) + } + if got := int64(len(page.Items)); got != total { + t.Errorf("Count(%+v) = %d, len(List(%+v).Items) = %d, want equal", countReq, total, listReq, got) + } +} + +// RunCounterParentIDMatchesListCardinality pins the storage-level half of +// CountRequest.ParentID's promise (S8 review, follow-up #5a): for one scope, +// Count's Total equals len(List's rows) for the equivalent ParentID scope. +func RunCounterParentIDMatchesListCardinality(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + requireCounterList(t, fixture) + requireCounterAddDependency(t, fixture) + + parent := fixture.IssuePrefix + "-parity-parentid-parent" + byEdge := fixture.IssuePrefix + "-parity-parentid-byedge" + byDotted := parent + ".1" + unrelated := fixture.IssuePrefix + "-parity-parentid-unrelated" + + for _, id := range []string{parent, byEdge, byDotted, unrelated} { + seedCounterIssue(t, ctx, fixture, counterSeed(id)) + } + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: byEdge, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", byEdge, parent, err) + } + + idFilter := counterScope(parent, byEdge, byDotted, unrelated).IDFilter + assertCounterMatchesList(t, ctx, fixture, idFilter, parent, false, nil) +} + +// RunCounterNoParentMatchesListCardinality pins the storage-level half of +// CountRequest.NoParent's promise (S8 review, follow-up #5a). +func RunCounterNoParentMatchesListCardinality(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + requireCounterList(t, fixture) + requireCounterAddDependency(t, fixture) + + parent := fixture.IssuePrefix + "-parity-noparent-parent" + child := fixture.IssuePrefix + "-parity-noparent-child" + byDotted := parent + ".1" + + for _, id := range []string{parent, child, byDotted} { + seedCounterIssue(t, ctx, fixture, counterSeed(id)) + } + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: child, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", child, parent, err) + } + + idFilter := counterScope(parent, child, byDotted).IDFilter + assertCounterMatchesList(t, ctx, fixture, idFilter, "", true, nil) +} + +// RunCounterExcludeTypesMatchesListCardinality pins the storage-level half of +// CountRequest.ExcludeTypes' promise (S8 review, follow-up #5a). ExcludeStatus +// has no List counterpart (counter.go) so it is not part of this parity +// family; RunCounterExcludeStatusNarrowsThePredicate above is its only +// coverage. +func RunCounterExcludeTypesMatchesListCardinality(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + requireCounterList(t, fixture) + + task := fixture.IssuePrefix + "-parity-extype-task" + chore := fixture.IssuePrefix + "-parity-extype-chore" + seedCounterIssue(t, ctx, fixture, counterSeed(task)) + choreSeed := counterSeed(chore) + choreSeed.IssueType = types.TypeChore + seedCounterIssue(t, ctx, fixture, choreSeed) + + idFilter := counterScope(task, chore).IDFilter + assertCounterMatchesList(t, ctx, fixture, idFilter, "", false, []string{"chore"}) +} + +// RunCounterParentIDIncludesAWispChild pins that ParentID's edge match reaches +// into the wisp tier exactly as IncludeInfra's merge does (S8 review, +// follow-up #5b): a child seeded in the ephemeral/wisp tier, linked to its +// parent by the same parent-child edge ParentID matches on, is invisible to +// Count's durable-only default and counted once IncludeInfra merges the wisps +// table in. +func RunCounterParentIDIncludesAWispChild(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + requireCounterAddDependency(t, fixture) + + parent := fixture.IssuePrefix + "-wispchild-parent" + wispChild := fixture.IssuePrefix + "-wispchild-child" + seedCounterIssue(t, ctx, fixture, counterSeed(parent)) + seedCounterWisp(t, ctx, fixture, counterSeed(wispChild)) + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: wispChild, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", wispChild, parent, err) + } + + scope := counterScope(parent, wispChild) + scope.ParentID = parent + // Durable-only default (counter.go, RunCounterCountsTheDurablePlaneByDefault): + // the wisp child is invisible without IncludeInfra. + assertCounterTotal(t, ctx, fixture, scope, 0) + + scope.IncludeInfra = true + assertCounterTotal(t, ctx, fixture, scope, 1) +} + +// RunCounterParentIDAndExcludeStatusComposeOnAClosedChild pins that ParentID +// and ExcludeStatus apply as a conjunction, not alternatives (S8 review, +// follow-up #5b): a closed child under its parent is counted by ParentID +// alone (Count hides nothing by default), and excluded once ExcludeStatus +// names its status, leaving only the open child. +func RunCounterParentIDAndExcludeStatusComposeOnAClosedChild(t *testing.T, ctx context.Context, fixture CounterFixture) { + t.Helper() + requireCounterAddDependency(t, fixture) + + parent := fixture.IssuePrefix + "-closedchild-parent" + openChild := fixture.IssuePrefix + "-closedchild-open" + closedChild := fixture.IssuePrefix + "-closedchild-closed" + seedCounterIssue(t, ctx, fixture, counterSeed(parent)) + seedCounterIssue(t, ctx, fixture, counterSeed(openChild)) + closedSeed := counterSeed(closedChild) + closedSeed.Status = types.StatusClosed + seedCounterIssue(t, ctx, fixture, closedSeed) + for _, child := range []string{openChild, closedChild} { + if err := fixture.AddDependency(ctx, &types.Dependency{ + IssueID: child, DependsOnID: parent, Type: types.DepParentChild, + }, "seed"); err != nil { + t.Fatalf("seed edge %s -> %s: %v", child, parent, err) + } + } + + scope := counterScope(parent, openChild, closedChild) + scope.ParentID = parent + assertCounterTotal(t, ctx, fixture, scope, 2) + + scope.ExcludeStatus = []string{"closed"} + assertCounterTotal(t, ctx, fixture, scope, 1) +} + // RunCounterGroupsPartitionTheScalarSet pins counter.go:168-169 and :244-245 // for a dimension whose buckets do NOT overlap: the grouped Total is the same // number a scalar Count returns, and the buckets partition it exactly. diff --git a/cmd/bd/count.go b/cmd/bd/count.go index 6264bfa20..b367cfda8 100644 --- a/cmd/bd/count.go +++ b/cmd/bd/count.go @@ -95,6 +95,9 @@ func parseCountRequest(cmd *cobra.Command) (issueops.CountRequest, issueops.Coun includeEphemeral, _ := cmd.Flags().GetBool("include-ephemeral") parentID, _ := cmd.Flags().GetString("parent") noParent, _ := cmd.Flags().GetBool("no-parent") + if parentID != "" && noParent { + return issueops.CountRequest{}, "", HandleErrorRespectJSON("--parent and --no-parent are mutually exclusive") + } excludeTypes, _ := cmd.Flags().GetStringSlice("exclude-type") excludeStatus, _ := cmd.Flags().GetStringSlice("exclude-status") diff --git a/cmd/bd/count_filter_test.go b/cmd/bd/count_filter_test.go index 0f06e1f3c..cfac118d1 100644 --- a/cmd/bd/count_filter_test.go +++ b/cmd/bd/count_filter_test.go @@ -134,6 +134,13 @@ func TestCountScopeFlagsShape(t *testing.T) { // is registered, documented and silently dropped on the way into the request. // Every filter flag is set to a value distinguishable from its zero and read // back off the request. +// +// --no-parent is NOT in this matrix: review S8 follow-up #1 made --parent and +// --no-parent mutually exclusive, and this test already sets --parent to +// cover that field, so setting --no-parent alongside it would refuse the +// whole request instead of pinning a mapping. --no-parent's carriage into the +// request is pinned on its own in TestParseCountRequestRefusesParentAndNoParentTogether's +// sibling cases below and in TestCountScopeFlagsShape above. func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { flags := newCountFlagSet(t) for flag, value := range map[string]string{ @@ -164,7 +171,6 @@ func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { "include-infra": "true", "include-ephemeral": "true", "parent": "bd-9", - "no-parent": "true", "exclude-type": "wisp,gate", "exclude-status": "closed,archived", } { @@ -220,7 +226,6 @@ func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { IncludeInfra: true, IncludeEphemeral: true, ParentID: "bd-9", - NoParent: true, ExcludeTypes: []string{"wisp", "gate"}, ExcludeStatus: []string{"closed", "archived"}, } @@ -229,6 +234,50 @@ func TestParseCountRequestCarriesEveryFilterFlag(t *testing.T) { } } +// TestParseCountRequestRefusesParentAndNoParentTogether pins review S8 +// follow-up #1: `bd count --parent X --no-parent` is refused at the CLI flag +// layer, before a request is even built — the same early-refusal shape every +// other mutually-exclusive pair on this CLI gets (e.g. --pinned/--no-pinned). +// +// HandleErrorRespectJSON prints "--parent and --no-parent are mutually +// exclusive" (`bd list`'s own wording, cmd/bd/list_input.go) to stderr/JSON +// and returns an opaque exit error, so — as every other parseCountRequest +// refusal test in this file does — only the refusal itself is asserted here, +// not the string inside the returned error. +func TestParseCountRequestRefusesParentAndNoParentTogether(t *testing.T) { + flags := newCountFlagSet(t) + if err := flags.Flags().Set("parent", "bd-1"); err != nil { + t.Fatalf("set --parent: %v", err) + } + if err := flags.Flags().Set("no-parent", "true"); err != nil { + t.Fatalf("set --no-parent: %v", err) + } + if _, _, err := parseCountRequest(flags); err == nil { + t.Fatal("parseCountRequest accepted --parent with --no-parent, want a refusal") + } +} + +// TestParseCountRequestCarriesNoParentAlone pins --no-parent's own mapping, +// split out of TestParseCountRequestCarriesEveryFilterFlag because that test's +// --parent case cannot also set --no-parent now that the two refuse each +// other. +func TestParseCountRequestCarriesNoParentAlone(t *testing.T) { + flags := newCountFlagSet(t) + if err := flags.Flags().Set("no-parent", "true"); err != nil { + t.Fatalf("set --no-parent: %v", err) + } + request, _, err := parseCountRequest(flags) + if err != nil { + t.Fatalf("parseCountRequest --no-parent: %v", err) + } + if !request.NoParent { + t.Error("NoParent = false, want true") + } + if request.ParentID != "" { + t.Errorf("ParentID = %q, want empty: --parent was not set", request.ParentID) + } +} + func TestParseCountRequestRejectsInvalidMetadataField(t *testing.T) { for _, value := range []string{"team", "bad$key=value"} { t.Run(value, func(t *testing.T) { diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 4ffadb8db..18bc84ac5 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -1010,7 +1010,7 @@ type ContextResponse struct { // OPTIONAL, and absent means only that this server does not disclose its filesystem layout — never that it has no workspace. A client MUST NOT require it, MUST NOT treat absence as an error, and has no use for the value beyond display: it is a path on the SERVER's filesystem, which the client cannot open. Identify the workspace by `project_id` and `database`, which are required. BeadsDir *string `json:"beads_dir,omitempty"` - // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`, and `issues.count.scope`, which announces that `GET /v0/beads/issues:count` accepts `parent`, `no_parent`, `exclude_type` and `exclude_status` (see those parameters) rather than silently answering `unknown_parameter` for all four. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. + // Capabilities The tokens this server advertises: the OPERATIONS it implements, derived from its route table, and the server-wide BEHAVIORS it enforces. v0's operation vocabulary is `ready.list`, `ready.count`, `issues.list`, `issues.query`, `issues.count`, `issues.get`, `issues.related`, `issues.create`, `issues.addComment`, `issues.batchClose`, `issues.claim`, `issues.claimNext`, `issues.release`, `issues.close`, `issues.reopen`, `issues.update`, `issues.sweep`, `issues.delete`, `issues.batchCreate`, `issues.batchApply`, `stats.get`, `config.list`, `config.get`, `config.set`, `config.unset`, `dependencies.cycles`, `dependencies.list`, `dependencies.count`, `dependencies.blocking`, `dependencies.tree`, `dependencies.add`, `dependencies.remove`, `memories.list`, `memories.get`, `memories.remember`, `memories.forget`, `events.list`, `events.watch`, `issues.casMetadata`; the behavior tokens are `project.enforce`, which announces that a `Bd-Project-Id` stamp for the wrong workspace is refused here rather than silently ignored, `issues.batchApplyLarge`, which announces that `POST /v0/beads/issues:batchApply` accepts the RAISED envelope — up to 1000 items (`ApplyBatchRequest.items.maxItems`), a 16 MiB body, and (for a request over 100 items) a run budget that EXTENDS to an operator-configured ceiling (`bd serve --large-apply-ceiling`, 5 minutes by default) — rather than the original 100-item, 4 MiB bound. The ceiling is a flat whole-run budget, not a value scaled by item count: a 101-item request and a 1000-item request both get the same extended budget. A request at or under 100 items is unaffected either way and keeps the server's ordinary deadline exactly as before. A client that checks for this token before sending a plan over 100 items learns whether THIS server accepts it without paying for a round trip an older server would refuse anyway, and `issues.list.sort`, which announces that `GET /v0/beads/issues` accepts the `sort` query parameter (two closed, pageable total orders — see that parameter) rather than silently answering `unknown_parameter`, and `issues.count.scope`, which announces that `GET /v0/beads/issues:count` accepts `parent`, `no_parent`, `exclude_type`, and `exclude_status` (see those parameters) rather than silently answering `unknown_parameter` for all four. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an operation or a behavior — never the version string. // // THIS LIST IS BUILD-LEVEL, NOT WORKSPACE-LEVEL. It says which operations this binary serves, and for every entry but two that is the whole answer. `events.list` and `events.watch` are the exceptions: the durable events journal is a per-workspace setting that is OFF by default, so a server that advertises them may still refuse every request to both with 409 `events_journal_disabled` — correctly, because the operations exist and the workspace has no journal. A consumer of either MUST treat the capability as "this server speaks it" and the 409 as "not on this workspace", and must not read the capability as a promise that records will arrive. Capabilities []string `json:"capabilities"` @@ -2464,13 +2464,17 @@ type CountIssuesParams struct { // Unset — and with `include_ephemeral` also unset — the count is durable-plane only and applies none of the four: the historical `bd count` answer, kept exactly so a scripted caller reads the same number it read yesterday. IncludeInfra *bool `form:"include_infra,omitempty" json:"include_infra,omitempty"` - // Parent Restrict to one issue's children: a parent-child dependency edge onto `parent`, OR — for an issue carrying no parent-child edge at all — a dotted-id prefix match (`parent.anything`). It is the same predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` and `bd list --parent X --all` agree on cardinality for the shared part of the predicate. Sending it beside `no_parent` is not refused, for the same reason `assignee` and `no_assignee` are not: the two compose into the empty intersection, which is `0`. + // Parent Restrict to one issue's DIRECT children only: a parent-child dependency edge onto `parent`, OR — for an issue carrying no parent-child edge at all — a dotted-id prefix match (`parent.anything`). It is the same ONE-LEVEL predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` agrees in cardinality with `bd list --parent X --flat --all` (or `--json`) for the shared part of the predicate — NOT with a bare `bd list --parent X --all`, whose default tree mode recursively walks every descendant (grandchildren and below) by re-querying this same one-level predicate once per level client-side (cmd/bd/list_show_filter_modes.go). `--flat` and `--json` both skip that recursion and return this field's one-level answer directly. + // + // Sending it beside `no_parent` IS refused: `invalid_argument` on `no_parent` with `reason: invalid_value`, matching `bd list`'s CLI refusal of the same combination. // // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. Parent *string `form:"parent,omitempty" json:"parent,omitempty"` // NoParent Only issues carrying NO parent-child dependency edge. Unlike `parent`, this does NOT consult the dotted-id convention — an issue named `X.1` with no edge to `X` is still counted here, which is the one place `parent` and `no_parent` are not exact opposites of each other. // + // Sending it beside `parent` IS refused: `invalid_argument` on this parameter with `reason: invalid_value`, matching `bd list`'s CLI refusal of the same combination (cmd/bd/list_input.go) rather than the empty-intersection treatment `assignee`/`no_assignee` get. + // // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. NoParent *bool `form:"no_parent,omitempty" json:"no_parent,omitempty"` @@ -2479,7 +2483,7 @@ type CountIssuesParams struct { // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. ExcludeType *[]string `form:"exclude_type,omitempty" json:"exclude_type,omitempty"` - // ExcludeStatus Statuses to exclude from the count. Repeat the parameter, or pass a comma-separated list; entries are taken AS WRITTEN — no normalization, no validation against this workspace's vocabulary, `status`'s own match-nothing-rather-than-fail treatment. There is no listing counterpart to this parameter: `GET /v0/beads/issues` computes its default status exclusions internally from workspace configuration and exposes no equivalent knob, so this is a Count-only capability. + // ExcludeStatus Statuses to exclude from the count. Repeat the parameter, or pass a comma-separated list; entries are taken as written (no normalization) but — UNLIKE `status` and `exclude_type` above — each name IS validated against this workspace's vocabulary (built-in statuses plus this workspace's own custom ones): an unrecognized name is `invalid_argument` on this parameter with `reason: invalid_value`, rather than silently excluding nothing and overcounting. There is no listing counterpart to this parameter: `GET /v0/beads/issues` computes its default status exclusions internally from workspace configuration and exposes no equivalent knob, so this is a Count-only capability. // // Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. ExcludeStatus *[]string `form:"exclude_status,omitempty" json:"exclude_status,omitempty"` diff --git a/internal/httpapi/reads.go b/internal/httpapi/reads.go index 653f98ad3..79caba90f 100644 --- a/internal/httpapi/reads.go +++ b/internal/httpapi/reads.go @@ -611,6 +611,15 @@ func invalidFilterParam(err error) (string, bool) { switch { case strings.HasPrefix(msg, "invalid status "): return "status", true + // The count role's own refusals (S8 review fix): a misspelled + // --exclude-status entry, and --parent set together with --no-parent. + // Both are the ROLE's ErrValidation, named here exactly as `status` and + // the metadata-key rows above are, so the client learns which parameter + // to fix rather than reading an unclassified 500. + case strings.HasPrefix(msg, "invalid exclude-status "): + return "exclude_status", true + case strings.HasPrefix(msg, "--parent and --no-parent are mutually exclusive"): + return "no_parent", true case strings.HasPrefix(msg, "invalid issue type "): return "type", true case strings.HasPrefix(msg, "invalid sort policy "): diff --git a/internal/httpapi/reads_test.go b/internal/httpapi/reads_test.go index 672f5147e..51cbb88ab 100644 --- a/internal/httpapi/reads_test.go +++ b/internal/httpapi/reads_test.go @@ -161,6 +161,8 @@ func TestABuilderRefusalIsTheDocumentedBadRequest(t *testing.T) { {"a has-metadata key on the ready surface", "/v0/beads/ready?has_metadata_key=1bad", "has_metadata_key"}, {"a metadata field key on the count surface", "/v0/beads/issues:count?metadata_field=1bad=x", "metadata_field"}, {"a has-metadata key on the count surface", "/v0/beads/issues:count?has_metadata_key=1bad", "has_metadata_key"}, + {"an unknown exclude-status name on the count surface", "/v0/beads/issues:count?exclude_status=bogus", "exclude_status"}, + {"parent and no_parent together on the count surface", "/v0/beads/issues:count?parent=bd-1&no_parent=true", "no_parent"}, } { t.Run(tc.name, func(t *testing.T) { ts, _ := newReadServer(t, Config{}) diff --git a/internal/httpapi/routes.go b/internal/httpapi/routes.go index 498f34022..1fe26aab0 100644 --- a/internal/httpapi/routes.go +++ b/internal/httpapi/routes.go @@ -77,7 +77,7 @@ const CapBatchApplyLarge = "issues.batchApplyLarge" const CapIssuesListSort = "issues.list.sort" // CapIssuesCountScope is the behavior capability that advertises GET -// /v0/beads/issues:count's `parent`, `no_parent`, `exclude_type` and +// /v0/beads/issues:count's `parent`, `no_parent`, `exclude_type`, and // `exclude_status` parameters (reads.go countFilters, spec OpCountIssues). // Like CapIssuesListSort it names FOUR PARAMETERS added to an existing // operation rather than a route of its own — `issues.count` is already the @@ -105,6 +105,18 @@ const CapIssuesListSort = "issues.list.sort" // sets `ParentID`, `NoParent`, `ExcludeTypes` or `ExcludeStatus` to an older // server — never let the caller pay for a round trip that 400s anyway, and // never silently drop the fields and return a wider count than asked for. +// +// THE DOWNSTREAM FALLBACK THIS PROTECTS (S8 review, follow-up #6): gc's own +// client maps that local `ErrUnsupported{Capability: "issues.count.scope"}` +// refusal to its own `ErrCountUnsupported` and falls back to the List role for +// the same predicate (beads-design DESIGN.txt §3.2, "gc maps this to +// ErrCountUnsupported and falls back to List") rather than surfacing the +// refusal to its own caller or guessing at a count. S3/S4, when they build +// that client, MUST add a skew test that masks this token out of a handshake +// response and asserts the fallback actually fires — not merely that the +// local refusal is raised — because a fallback that compiles but never runs +// in CI is indistinguishable from one that silently regressed to a wrong +// count. const CapIssuesCountScope = "issues.count.scope" // customMethodTarget splits the custom method off the segment the router diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index e91a56823..fae2baa58 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -2071,15 +2071,24 @@ paths: - name: parent in: query description: >- - Restrict to one issue's children: a parent-child dependency edge - onto `parent`, OR — for an issue carrying no parent-child edge at - all — a dotted-id prefix match (`parent.anything`). It is the - same predicate `GET /v0/beads/issues`'s own `parent` applies, so - `bd count --parent X` and `bd list --parent X --all` agree on - cardinality for the shared part of the predicate. Sending it - beside `no_parent` is not refused, for the same reason `assignee` - and `no_assignee` are not: the two compose into the empty - intersection, which is `0`. + Restrict to one issue's DIRECT children only: a parent-child + dependency edge onto `parent`, OR — for an issue carrying no + parent-child edge at all — a dotted-id prefix match + (`parent.anything`). It is the same ONE-LEVEL predicate + `GET /v0/beads/issues`'s own `parent` applies, so `bd count + --parent X` agrees in cardinality with `bd list --parent X --flat + --all` (or `--json`) for the shared part of the predicate — NOT + with a bare `bd list --parent X --all`, whose default tree mode + recursively walks every descendant (grandchildren and below) by + re-querying this same one-level predicate once per level + client-side (cmd/bd/list_show_filter_modes.go). `--flat` and + `--json` both skip that recursion and return this field's + one-level answer directly. + + + Sending it beside `no_parent` IS refused: `invalid_argument` on + `no_parent` with `reason: invalid_value`, matching `bd list`'s CLI + refusal of the same combination. Behind the `issues.count.scope` behavior token: an older server @@ -2096,6 +2105,12 @@ paths: of each other. + Sending it beside `parent` IS refused: `invalid_argument` on this + parameter with `reason: invalid_value`, matching `bd list`'s CLI + refusal of the same combination (cmd/bd/list_input.go) rather than + the empty-intersection treatment `assignee`/`no_assignee` get. + + Behind the `issues.count.scope` behavior token: an older server answers `unknown_parameter` for this name. schema: @@ -2126,13 +2141,16 @@ paths: in: query description: >- Statuses to exclude from the count. Repeat the parameter, or pass - a comma-separated list; entries are taken AS WRITTEN — no - normalization, no validation against this workspace's vocabulary, - `status`'s own match-nothing-rather-than-fail treatment. There is - no listing counterpart to this parameter: `GET /v0/beads/issues` - computes its default status exclusions internally from workspace - configuration and exposes no equivalent knob, so this is a - Count-only capability. + a comma-separated list; entries are taken as written (no + normalization) but — UNLIKE `status` and `exclude_type` above — + each name IS validated against this workspace's vocabulary + (built-in statuses plus this workspace's own custom ones): an + unrecognized name is `invalid_argument` on this parameter with + `reason: invalid_value`, rather than silently excluding nothing + and overcounting. There is no listing counterpart to this + parameter: `GET /v0/beads/issues` computes its default status + exclusions internally from workspace configuration and exposes no + equivalent knob, so this is a Count-only capability. Behind the `issues.count.scope` behavior token: an older server @@ -7592,7 +7610,7 @@ components: parameter) rather than silently answering `unknown_parameter`, and `issues.count.scope`, which announces that `GET /v0/beads/issues:count` accepts `parent`, `no_parent`, - `exclude_type` and `exclude_status` (see those parameters) rather + `exclude_type`, and `exclude_status` (see those parameters) rather than silently answering `unknown_parameter` for all four. The list grows additively, and an operation never appears here unless it is fully implemented. This is how a client checks for an diff --git a/internal/storage/dolt/counter_contract_test.go b/internal/storage/dolt/counter_contract_test.go index dd1d5b675..1166eef66 100644 --- a/internal/storage/dolt/counter_contract_test.go +++ b/internal/storage/dolt/counter_contract_test.go @@ -77,6 +77,21 @@ func TestCounterContract(t *testing.T) { t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) }) + t.Run("ParentIDMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterParentIDMatchesListCardinality(t, ctx, fixture) + }) + t.Run("NoParentMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterNoParentMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ExcludeTypesMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterExcludeTypesMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ParentIDIncludesAWispChild", func(t *testing.T) { + conformance.RunCounterParentIDIncludesAWispChild(t, ctx, fixture) + }) + t.Run("ParentIDAndExcludeStatusComposeOnAClosedChild", func(t *testing.T) { + conformance.RunCounterParentIDAndExcludeStatusComposeOnAClosedChild(t, ctx, fixture) + }) } func newDoltCounterFixture(t *testing.T, prefix string) (conformance.CounterFixture, context.Context, func()) { @@ -90,6 +105,12 @@ func newDoltCounterFixture(t *testing.T, prefix string) (conformance.CounterFixt t.Fatalf("Counter(): %v", err) } kit := newDoltRoleFixtureKit(store, prefix) + reader, err := store.IssueReader() + if err != nil { + cancel() + storeCleanup() + t.Fatalf("IssueReader(): %v", err) + } fixture := conformance.CounterFixture{ IssuePrefix: kit.IssuePrefix, Counter: counter, @@ -97,6 +118,7 @@ func newDoltCounterFixture(t *testing.T, prefix string) (conformance.CounterFixt CreateWisp: kit.CreateWisp, CountHistory: kit.CountHistory, AddDependency: kit.AddDependency, + List: reader.List, } return fixture, ctx, func() { cancel() diff --git a/internal/storage/embeddeddolt/counter_contract_test.go b/internal/storage/embeddeddolt/counter_contract_test.go index 7cfdf0ffd..b10c18c3c 100644 --- a/internal/storage/embeddeddolt/counter_contract_test.go +++ b/internal/storage/embeddeddolt/counter_contract_test.go @@ -80,6 +80,21 @@ func TestCounterContract(t *testing.T) { t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) }) + t.Run("ParentIDMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterParentIDMatchesListCardinality(t, ctx, fixture) + }) + t.Run("NoParentMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterNoParentMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ExcludeTypesMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterExcludeTypesMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ParentIDIncludesAWispChild", func(t *testing.T) { + conformance.RunCounterParentIDIncludesAWispChild(t, ctx, fixture) + }) + t.Run("ParentIDAndExcludeStatusComposeOnAClosedChild", func(t *testing.T) { + conformance.RunCounterParentIDAndExcludeStatusComposeOnAClosedChild(t, ctx, fixture) + }) } func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conformance.CounterFixture { @@ -88,6 +103,10 @@ func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conform if err != nil { t.Fatalf("Counter(): %v", err) } + reader, err := te.store.IssueReader() + if err != nil { + t.Fatalf("IssueReader(): %v", err) + } kit := newEmbeddedRoleFixtureKit(te, prefix) return conformance.CounterFixture{ IssuePrefix: kit.IssuePrefix, @@ -96,5 +115,6 @@ func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conform CreateWisp: kit.CreateWisp, CountHistory: kit.CountHistory, AddDependency: kit.AddDependency, + List: reader.List, } } diff --git a/internal/storage/uow/counter_contract_test.go b/internal/storage/uow/counter_contract_test.go index 9cbf2e7a8..6dbe8cf63 100644 --- a/internal/storage/uow/counter_contract_test.go +++ b/internal/storage/uow/counter_contract_test.go @@ -77,6 +77,21 @@ func TestCounterContract(t *testing.T) { t.Run("ExcludeStatusNarrowsThePredicate", func(t *testing.T) { conformance.RunCounterExcludeStatusNarrowsThePredicate(t, ctx, fixture) }) + t.Run("ParentIDMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterParentIDMatchesListCardinality(t, ctx, fixture) + }) + t.Run("NoParentMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterNoParentMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ExcludeTypesMatchesListCardinality", func(t *testing.T) { + conformance.RunCounterExcludeTypesMatchesListCardinality(t, ctx, fixture) + }) + t.Run("ParentIDIncludesAWispChild", func(t *testing.T) { + conformance.RunCounterParentIDIncludesAWispChild(t, ctx, fixture) + }) + t.Run("ParentIDAndExcludeStatusComposeOnAClosedChild", func(t *testing.T) { + conformance.RunCounterParentIDAndExcludeStatusComposeOnAClosedChild(t, ctx, fixture) + }) } func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conformance.CounterFixture { @@ -92,6 +107,14 @@ func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conf if err != nil { t.Fatalf("Counter(): %v", err) } + readerSource, ok := provider.(IssueReaderSource) + if !ok { + t.Fatalf("provider %T does not offer the IssueReader accessor", provider) + } + reader, err := readerSource.IssueReader() + if err != nil { + t.Fatalf("IssueReader(): %v", err) + } kit := newUOWRoleFixtureKit(provider, prefix) return conformance.CounterFixture{ IssuePrefix: kit.IssuePrefix, @@ -100,5 +123,6 @@ func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conf CreateWisp: kit.CreateWisp, CountHistory: kit.CountHistory, AddDependency: kit.AddDependency, + List: reader.List, } } diff --git a/internal/workapi/count.go b/internal/workapi/count.go index d379dd207..9d2dd26fc 100644 --- a/internal/workapi/count.go +++ b/internal/workapi/count.go @@ -103,6 +103,15 @@ func BuildCountFilter(in issueops.CountRequest, cfg ListConfig) (types.IssueFilt filter.IDs = ids } + // ParentID and NoParent are refused together, matching `bd list`'s CLI + // refusal of the same combination (cmd/bd/list_input.go) with the same + // wording — the role raises it as ErrValidation so a caller reaching this + // builder from any front door (HTTP, proxied CLI, a future client) gets + // the same refusal the primary CLI's flag parser gives, rather than the + // role silently answering the empty intersection. + if in.ParentID != "" && in.NoParent { + return types.IssueFilter{}, fmt.Errorf("--parent and --no-parent are mutually exclusive%.0w", issueops.ErrValidation) + } if in.ParentID != "" { parentID := in.ParentID filter.ParentID = &parentID @@ -124,15 +133,26 @@ func BuildCountFilter(in issueops.CountRequest, cfg ListConfig) (types.IssueFilt } } - // ExcludeStatus takes names as written, with no normalization and no - // validation against the workspace vocabulary — the same - // match-nothing-rather-than-fail treatment Status and IssueType get above. + // ExcludeStatus takes names as written but — unlike Status and IssueType + // above — IS validated against the workspace vocabulary (built-in statuses + // plus cfg's custom ones, the same set ApplyStatusFilter checks in + // list.go). An exclusion list is built by hand from a status name, so a + // misspelled entry here would silently exclude nothing and OVERCOUNT + // rather than undercount, which is a worse failure mode than the + // match-nothing treatment Status and IssueType accept above — so a typo + // is ErrValidation instead. for _, raw := range in.ExcludeStatus { for _, s := range strings.Split(raw, ",") { s = strings.TrimSpace(s) - if s != "" { - filter.ExcludeStatus = append(filter.ExcludeStatus, types.Status(s)) + if s == "" { + continue } + status := types.Status(s) + if !status.IsValidWithCustom(cfg.CustomStatusNames()) { + return types.IssueFilter{}, fmt.Errorf("invalid exclude-status %q (valid: %s)%.0w", + s, ValidStatusList(cfg.CustomStatusNames()), issueops.ErrValidation) + } + filter.ExcludeStatus = append(filter.ExcludeStatus, status) } } diff --git a/internal/workapi/count_test.go b/internal/workapi/count_test.go index 9b533d981..2d8b102fe 100644 --- a/internal/workapi/count_test.go +++ b/internal/workapi/count_test.go @@ -271,22 +271,75 @@ func TestBuildCountFilterParentFieldsMatchListFilter(t *testing.T) { // TestBuildCountFilterExcludeStatusTakesNamesAsWritten pins ExcludeStatus: a // Count-only field with no List counterpart (ListRequest computes its default // exclusions internally and exposes no caller-facing knob for them). Entries -// split and trim like ExcludeTypes, but are NOT normalized or validated -// against the workspace vocabulary — the same match-nothing-rather-than-fail -// treatment Status gets. +// split and trim like ExcludeTypes. UNLIKE Status and ExcludeTypes, each name +// IS validated against the workspace vocabulary (review S8 follow-up #3): a +// built-in name passes as written, and an unrecognized one is ErrValidation +// rather than silently excluding nothing. func TestBuildCountFilterExcludeStatusTakesNamesAsWritten(t *testing.T) { got, err := BuildCountFilter(issueops.CountRequest{ - ExcludeStatus: []string{" closed , archived", "", "no-such-status"}, + ExcludeStatus: []string{" closed , pinned", ""}, }, ListConfig{}) if err != nil { t.Fatalf("BuildCountFilter: %v", err) } - want := []types.Status{"closed", "archived", "no-such-status"} + want := []types.Status{"closed", "pinned"} if !reflect.DeepEqual(got.ExcludeStatus, want) { t.Errorf("ExcludeStatus = %v, want %v", got.ExcludeStatus, want) } } +// TestBuildCountFilterExcludeStatusRejectsAnUnknownName pins the ErrValidation +// refusal (review S8 follow-up #3): a typo'd status name in --exclude-status +// must fail loudly rather than silently excluding nothing and overcounting. +func TestBuildCountFilterExcludeStatusRejectsAnUnknownName(t *testing.T) { + _, err := BuildCountFilter(issueops.CountRequest{ + ExcludeStatus: []string{"no-such-status"}, + }, ListConfig{}) + if err == nil { + t.Fatal("BuildCountFilter accepted an unknown exclude-status name, want ErrValidation") + } + if !errors.Is(err, issueops.ErrValidation) { + t.Errorf("err = %v, want errors.Is(err, issueops.ErrValidation)", err) + } +} + +// TestBuildCountFilterExcludeStatusAllowsAWorkspaceCustomStatus pins the other +// half of #3: a workspace-defined custom status is NOT a typo and must be +// accepted, following ApplyStatusFilter's own IsValidWithCustom precedent in +// list.go. +func TestBuildCountFilterExcludeStatusAllowsAWorkspaceCustomStatus(t *testing.T) { + cfg := ListConfig{CustomStatuses: []types.CustomStatus{{Name: "triaged"}}} + got, err := BuildCountFilter(issueops.CountRequest{ + ExcludeStatus: []string{"triaged"}, + }, cfg) + if err != nil { + t.Fatalf("BuildCountFilter rejected the workspace's own custom status: %v", err) + } + want := []types.Status{"triaged"} + if !reflect.DeepEqual(got.ExcludeStatus, want) { + t.Errorf("ExcludeStatus = %v, want %v", got.ExcludeStatus, want) + } +} + +// TestBuildCountFilterRefusesParentAndNoParentTogether pins review S8 +// follow-up #1: the role refuses the combination as ErrValidation, with the +// same wording `bd list`'s CLI already uses for the same combination +// (cmd/bd/list_input.go), so a caller reaching this role from any front door +// gets the identical refusal. +func TestBuildCountFilterRefusesParentAndNoParentTogether(t *testing.T) { + _, err := BuildCountFilter(issueops.CountRequest{ParentID: "bd-1", NoParent: true}, ListConfig{}) + if err == nil { + t.Fatal("BuildCountFilter accepted --parent with --no-parent, want ErrValidation") + } + if !errors.Is(err, issueops.ErrValidation) { + t.Errorf("err = %v, want errors.Is(err, issueops.ErrValidation)", err) + } + const want = "--parent and --no-parent are mutually exclusive" + if err.Error() != want { + t.Errorf("err = %q, want %q", err.Error(), want) + } +} + // TestBuildCountFilterStatusAllIsNotValidation pins DESIGN's requirement that // Status "all" have well-defined semantics: it is the no-filter spelling // (same as empty), NOT ErrValidation, because BuildListFilter's own "all" diff --git a/issueops/counter.go b/issueops/counter.go index 2513b8c8c..03deb5760 100644 --- a/issueops/counter.go +++ b/issueops/counter.go @@ -80,9 +80,15 @@ type CountRequest struct { // ParentID restricts to one issue's children, spelled as // ListRequest.ParentID spells it: a parent-child dependency edge, OR (for // an issue with no such edge) a dotted-id prefix match. NoParent restricts - // to rows with neither. Setting both is not refused, for the same reason - // Assignee and NoAssignee are not: the filter answers with the empty - // intersection rather than failing. + // to rows with neither. + // + // Setting both IS refused, as ErrValidation, UNLIKE Assignee/NoAssignee + // above: `bd list`'s CLI already refuses `--parent` with `--no-parent` + // (cmd/bd/list_input.go) with this same wording, and a caller reaching + // this role directly — over HTTP, or through a future client that skips + // the primary CLI's flag parser — gets the identical refusal rather than + // a silent empty-intersection answer the primary CLI would never have let + // it ask for. ParentID string NoParent bool @@ -117,10 +123,12 @@ type CountRequest struct { // List counterpart — ListRequest computes its default status exclusions // internally from workspace configuration and exposes no caller-facing // knob for them. Here it is a new, Count-only capability (behavior token - // `issues.count.scope`): entries may be comma-separated, and — matching - // Status and ExcludeTypes — are NOT validated against the workspace - // vocabulary, so an unrecognized name excludes nothing rather than - // failing. + // `issues.count.scope`): entries may be comma-separated, and — UNLIKE + // Status and ExcludeTypes above — each name IS validated against the + // workspace vocabulary (built-in statuses plus the workspace's own custom + // ones). An unrecognized name is ErrValidation rather than excluding + // nothing: a caller building an exclusion list by hand gets a loud failure + // on a typo instead of a silently wider, overcounted answer. ExcludeStatus []string // TitleContains, DescContains and NotesContains are substring matches on diff --git a/issueops/reader.go b/issueops/reader.go index 0a0219f52..387bd4007 100644 --- a/issueops/reader.go +++ b/issueops/reader.go @@ -54,7 +54,19 @@ type ReadyRequest struct { // the other, and P0 has already been lost that way once. Priority *int - // ParentID restricts to recursive descendants of one issue. + // ParentID restricts to one issue's DIRECT children only: a parent-child + // dependency edge, OR (for an issue with no such edge) a dotted-id prefix + // match — the same one-level sqlbuild.ParentID clause CountRequest.ParentID + // compiles to (internal/storage/sqlbuild/filter.go). It is NOT a recursive + // descendants walk, despite this field's name and this comment's own prior + // wording (fixed in the S8 review, GH#4387): the recursive tree that + // `bd list --parent X` renders by DEFAULT comes from + // cmd/bd/list_show_filter_modes.go's findAllDescendants calling this ROLE + // once per level and accumulating the results client-side, not from the + // role answering a deeper question. `bd list --parent X --flat` (or + // --json) skips that client-side recursion and returns exactly this + // field's one-level answer, which is the invocation that agrees with + // `bd count --parent X`'s cardinality. ParentID string // MolType restricts to one molecule type. MolType *MolType From f23bf5ddbfa215807c1b7ce3c61db79b57299a4a Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:50:11 +0000 Subject: [PATCH 11/16] =?UTF-8?q?fix(count):=20PR=20#7199=20review=20follo?= =?UTF-8?q?w-ups=20=E2=80=94=20exclude-status=20config=20load,=20ParentID?= =?UTF-8?q?=20doc=20swap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses bee's REQUEST-CHANGES on gastownhall/beads#7199 (S8): 1. BLOCKER: storecounter.filter and uow.countFilter only loaded the workspace list config when IncludeInfra was set, so BuildCountFilter's ExcludeStatus validation refused a workspace's own custom status unless the caller also passed the unrelated IncludeInfra flag. Both legs now load the config when IncludeInfra OR len(ExcludeStatus) > 0, with comments corrected to say so. Added leg-level regression tests (the conformance fixture cannot drive a workspace-configured custom status) on embeddeddolt and uow, reproducing the reviewer's CLI repro directly, plus an HTTP-level probe in cmd/bd's real-Dolt proxied count test. 2. issueops.ReadyRequest.ParentID's comment had been overwritten with the one-level direct-children wording that belongs to ListRequest.ParentID; Ready actually walks all descendants via GetDescendantIDsInTx (GH#3396). Restored the descendants comment on ReadyRequest.ParentID and moved the one-level explanation to ListRequest.ParentID, which had none. Verified: make api-check; dolt-server, embeddeddolt and uow Counter contract legs (BEADS_TEST_DOLT_SERVER=local with a pinned dolt binary for uow; Docker for the dolt-server leg); the new proxied HTTP exclude_status probe (BEADS_TEST_PROXIED_SERVER=1). Co-Authored-By: Claude Sonnet 5 --- .../serve_count_proxied_integration_test.go | 45 +++++++++++ .../embeddeddolt/counter_contract_test.go | 66 +++++++++++++++ internal/storage/uow/counter.go | 9 ++- internal/storage/uow/counter_contract_test.go | 80 +++++++++++++++++++ internal/workapi/storecounter/counter.go | 17 ++-- issueops/reader.go | 35 +++++--- 6 files changed, 229 insertions(+), 23 deletions(-) diff --git a/cmd/bd/serve_count_proxied_integration_test.go b/cmd/bd/serve_count_proxied_integration_test.go index 577a09d7e..9747f40c4 100644 --- a/cmd/bd/serve_count_proxied_integration_test.go +++ b/cmd/bd/serve_count_proxied_integration_test.go @@ -255,6 +255,51 @@ func TestProxiedServerServeCount(t *testing.T) { } }) + // PR #7199 REVIEW: exclude_status against a WORKSPACE CUSTOM STATUS, over + // the wire and with include_infra left unset. The fake-role tests in + // internal/httpapi cannot write this case — a fake cannot be asked whether a + // custom status is in its own vocabulary — and the bug this pins lived below + // the HTTP layer anyway (storecounter and uow's countFilter only loaded the + // workspace's list configuration when include_infra was set, so + // BuildCountFilter's validation refused a status the workspace itself + // defined unless the caller also happened to pass the unrelated flag). + t.Run("exclude_status accepts a workspace custom status without include_infra", func(t *testing.T) { + label := countSliceLabel + "-customstatus" + scopedCustom := "?label=" + label + + if out, err := bdProxiedRun(t, bd, p.dir, "config", "set", "status.custom", "review,qa"); err != nil { + t.Fatalf("bd config set status.custom: %v\n%s", err, out) + } + + bdProxiedCreate(t, bd, p.dir, "custom status open row", "-p", "2", "-l", label) + reviewRow := bdProxiedCreate(t, bd, p.dir, "custom status review row", "-p", "2", "-l", label) + if out, err := bdProxiedRun(t, bd, p.dir, "update", reviewRow.ID, "--status", "review"); err != nil { + t.Fatalf("bd update --status review: %v\n%s", err, out) + } + + _, total, _ := sp.countIssues(t, scopedCustom) + if total != 2 { + t.Fatalf("total = %d, want 2", total) + } + + // include_infra is NOT in this query string. Before the fix this 400'd + // with an "invalid exclude-status" refusal because the config load that + // feeds BuildCountFilter's vocabulary check was gated on include_infra + // alone. + status, narrowed, _ := sp.countIssues(t, scopedCustom+"&exclude_status=review") + if status != http.StatusOK { + t.Fatalf("exclude_status=review (a workspace custom status, include_infra unset): status = %d, want 200", status) + } + if narrowed != 1 { + t.Fatalf("exclude_status=review counted %d, want 1", narrowed) + } + + // The parity oracle, same as the sibling cases above. + if cli := bdProxiedCountTotal(t, bd, p.dir, "--label", label, "--exclude-status", "review"); cli != narrowed { + t.Errorf("bd count --exclude-status review = %d, the server said %d", cli, narrowed) + } + }) + // The refusals, over the wire: a closed vocabulary and a typed parameter // set, both refused before any query runs. t.Run("the document's refusals are refused", func(t *testing.T) { diff --git a/internal/storage/embeddeddolt/counter_contract_test.go b/internal/storage/embeddeddolt/counter_contract_test.go index b10c18c3c..b39570cb3 100644 --- a/internal/storage/embeddeddolt/counter_contract_test.go +++ b/internal/storage/embeddeddolt/counter_contract_test.go @@ -6,6 +6,8 @@ import ( "testing" "github.com/steveyegge/beads/backend/conformance" + "github.com/steveyegge/beads/internal/types" + publicops "github.com/steveyegge/beads/issueops" ) // TestCounterContract runs the Counter contract against the embedded store, @@ -118,3 +120,67 @@ func newEmbeddedCounterFixture(t *testing.T, te *testEnv, prefix string) conform List: reader.List, } } + +// TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra +// pins the PR #7199 review's blocker fix: the store-backed Counter must load +// the workspace's list configuration when ExcludeStatus is non-empty, not +// only when IncludeInfra is set, so BuildCountFilter's validation sees the +// workspace's own custom statuses. A leg-level test, not a conformance case, +// because CounterFixture cannot drive a workspace-configured custom status +// (counter_contract.go's CounterFixture doc) — this needs a real store with +// status.custom set. +// +// Reproduces the reviewer's CLI repro directly against the store: with +// status.custom="review,qa", an issue moved to "review", and IncludeInfra +// left unset, `--exclude-status review` must narrow the count instead of +// being refused as an unrecognized status. +func TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra(t *testing.T) { + skipUnlessEmbeddedDolt(t) + te := newTestEnv(t, "cntcs") + ctx := t.Context() + + if err := te.store.SetConfig(ctx, "status.custom", "review,qa"); err != nil { + t.Fatalf("SetConfig(status.custom): %v", err) + } + + const openID = "cntcs-open" + const reviewID = "cntcs-review" + for _, id := range []string{openID, reviewID} { + if err := te.store.CreateIssue(ctx, &types.Issue{ + ID: id, Title: id, Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask, + }, "tester"); err != nil { + t.Fatalf("CreateIssue(%s): %v", id, err) + } + } + if err := te.store.UpdateIssue(ctx, reviewID, map[string]interface{}{ + "status": types.Status("review"), + }, "tester"); err != nil { + t.Fatalf("UpdateIssue(%s) to review: %v", reviewID, err) + } + + counter, err := te.store.Counter() + if err != nil { + t.Fatalf("Counter(): %v", err) + } + + scope := publicops.CountRequest{IDFilter: openID + "," + reviewID} + total, err := counter.Count(ctx, scope) + if err != nil { + t.Fatalf("Count(unfiltered): %v", err) + } + if total.Total != 2 { + t.Fatalf("Count(unfiltered).Total = %d, want 2", total.Total) + } + + // IncludeInfra left UNSET: before the fix this refused with "invalid + // exclude-status \"review\"" because the config load was gated on + // IncludeInfra alone. + scope.ExcludeStatus = []string{"review"} + result, err := counter.Count(ctx, scope) + if err != nil { + t.Fatalf("Count(exclude-status=review, IncludeInfra unset): %v, want the custom status accepted", err) + } + if result.Total != 1 { + t.Fatalf("Count(exclude-status=review).Total = %d, want 1", result.Total) + } +} diff --git a/internal/storage/uow/counter.go b/internal/storage/uow/counter.go index 03a155788..467712fa2 100644 --- a/internal/storage/uow/counter.go +++ b/internal/storage/uow/counter.go @@ -85,11 +85,14 @@ func (c *counter) CountByGroup(ctx context.Context, req publicops.CountByGroupRe } // countFilter builds the storage filter from the unit of work the call already -// holds, loading configuration only when IncludeInfra can read it — the same two -// decisions the store-backed body makes, through the same builder. +// holds, loading configuration only when IncludeInfra or ExcludeStatus can +// read it — the same two decisions the store-backed body makes, through the +// same builder. ExcludeStatus needs the workspace's custom status names so +// BuildCountFilter's validation does not refuse a status the workspace itself +// defines (PR #7199 review). func countFilter(ctx context.Context, uw UnitOfWork, req publicops.CountRequest) (types.IssueFilter, error) { var cfg workapi.ListConfig - if req.IncludeInfra { + if req.IncludeInfra || len(req.ExcludeStatus) > 0 { loaded, err := workapi.LoadUOWListConfig(ctx, uw) if err != nil { return types.IssueFilter{}, err diff --git a/internal/storage/uow/counter_contract_test.go b/internal/storage/uow/counter_contract_test.go index 6dbe8cf63..9936e77e1 100644 --- a/internal/storage/uow/counter_contract_test.go +++ b/internal/storage/uow/counter_contract_test.go @@ -5,6 +5,9 @@ import ( "testing" "github.com/steveyegge/beads/backend/conformance" + "github.com/steveyegge/beads/internal/storage/domain" + "github.com/steveyegge/beads/internal/types" + publicops "github.com/steveyegge/beads/issueops" ) // TestCounterContract runs the Counter contract against the unit-of-work @@ -126,3 +129,80 @@ func newUOWCounterFixture(t *testing.T, ctx context.Context, prefix string) conf List: reader.List, } } + +// TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra +// pins the PR #7199 review's blocker fix on the unit-of-work leg: countFilter +// must load the workspace's list configuration when ExcludeStatus is +// non-empty, not only when IncludeInfra is set, so BuildCountFilter's +// validation sees the workspace's own custom statuses. A leg-level test, not +// a conformance case, because CounterFixture cannot drive a +// workspace-configured custom status (counter_contract.go's CounterFixture +// doc) — this needs a real provider with status.custom set. +// +// Its own provider (not newUOWCounterFixture's), so status.custom does not +// leak into the other subtests sharing that fixture's one database. +func TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra(t *testing.T) { + ctx := context.Background() + provider := newUOWRoleFixtureProvider(t, ctx, "cntcs") + + if err := RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) { + return "set status.custom", uw.ConfigUseCase().SetConfig(ctx, "status.custom", "review,qa") + }); err != nil { + t.Fatalf("SetConfig(status.custom): %v", err) + } + + const openID = "cntcs-open" + const reviewID = "cntcs-review" + for _, id := range []string{openID, reviewID} { + id := id + if err := RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) { + _, err := uw.IssueUseCase().CreateIssue(ctx, domain.CreateIssueParams{ + Issue: &types.Issue{ + ID: id, Title: id, Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask, + }, + ExplicitID: id, + CreateOnly: true, + }, "tester") + return "seed " + id, err + }); err != nil { + t.Fatalf("CreateIssue(%s): %v", id, err) + } + } + if err := RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) { + return "move to review", uw.IssueUseCase().UpdateIssue(ctx, reviewID, map[string]any{ + "status": types.Status("review"), + }, "tester") + }); err != nil { + t.Fatalf("UpdateIssue(%s) to review: %v", reviewID, err) + } + + source, ok := provider.(CounterSource) + if !ok { + t.Fatalf("provider %T does not offer the Counter accessor", provider) + } + counter, err := source.Counter() + if err != nil { + t.Fatalf("Counter(): %v", err) + } + + scope := publicops.CountRequest{IDFilter: openID + "," + reviewID} + total, err := counter.Count(ctx, scope) + if err != nil { + t.Fatalf("Count(unfiltered): %v", err) + } + if total.Total != 2 { + t.Fatalf("Count(unfiltered).Total = %d, want 2", total.Total) + } + + // IncludeInfra left UNSET: before the fix this refused with "invalid + // exclude-status \"review\"" because the config load was gated on + // IncludeInfra alone. + scope.ExcludeStatus = []string{"review"} + result, err := counter.Count(ctx, scope) + if err != nil { + t.Fatalf("Count(exclude-status=review, IncludeInfra unset): %v, want the custom status accepted", err) + } + if result.Total != 1 { + t.Fatalf("Count(exclude-status=review).Total = %d, want 1", result.Total) + } +} diff --git a/internal/workapi/storecounter/counter.go b/internal/workapi/storecounter/counter.go index f873d5900..0c9fa1b29 100644 --- a/internal/workapi/storecounter/counter.go +++ b/internal/workapi/storecounter/counter.go @@ -75,15 +75,18 @@ func (c *storeCounter) CountByGroup(ctx context.Context, req issueops.CountByGro // filter builds the storage filter, loading the workspace configuration only // when the request can read it. // -// TWO THINGS ARE DELIBERATE. The load is SKIPPED unless IncludeInfra is set, -// because that flag is the only thing in a count request the configuration -// reaches. And when it does run it runs PER CALL rather than once at -// construction: the infra vocabulary is workspace state a caller can change -// between two counts, and a counter that cached it would answer for the older -// workspace. +// TWO THINGS ARE DELIBERATE. The load is SKIPPED unless IncludeInfra is set OR +// ExcludeStatus is non-empty — those are the two things in a count request the +// configuration reaches: IncludeInfra needs the infra vocabulary, and +// ExcludeStatus needs the workspace's custom status names so +// BuildCountFilter's validation does not refuse a status the workspace itself +// defines (PR #7199 review). And when it does run it runs PER CALL rather +// than once at construction: the infra vocabulary and custom statuses are +// workspace state a caller can change between two counts, and a counter that +// cached it would answer for the older workspace. func (c *storeCounter) filter(ctx context.Context, req issueops.CountRequest) (types.IssueFilter, error) { var cfg workapi.ListConfig - if req.IncludeInfra { + if req.IncludeInfra || len(req.ExcludeStatus) > 0 { loaded, err := workapi.LoadStoreListConfig(ctx, c.store) if err != nil { return types.IssueFilter{}, err diff --git a/issueops/reader.go b/issueops/reader.go index 387bd4007..a49892e03 100644 --- a/issueops/reader.go +++ b/issueops/reader.go @@ -54,19 +54,15 @@ type ReadyRequest struct { // the other, and P0 has already been lost that way once. Priority *int - // ParentID restricts to one issue's DIRECT children only: a parent-child - // dependency edge, OR (for an issue with no such edge) a dotted-id prefix - // match — the same one-level sqlbuild.ParentID clause CountRequest.ParentID - // compiles to (internal/storage/sqlbuild/filter.go). It is NOT a recursive - // descendants walk, despite this field's name and this comment's own prior - // wording (fixed in the S8 review, GH#4387): the recursive tree that - // `bd list --parent X` renders by DEFAULT comes from - // cmd/bd/list_show_filter_modes.go's findAllDescendants calling this ROLE - // once per level and accumulating the results client-side, not from the - // role answering a deeper question. `bd list --parent X --flat` (or - // --json) skips that client-side recursion and returns exactly this - // field's one-level answer, which is the invocation that agrees with - // `bd count --parent X`'s cardinality. + // ParentID restricts to ALL TRANSITIVE DESCENDANTS of one issue, not just + // its direct children: internal/storage/issueops/ready_work.go calls + // GetDescendantIDsInTx(ctx, tx, *filter.ParentID, 0), the recursive walk. + // GH#3396: this used to be a one-hop subquery against the dependency + // table, so grandchildren were silently dropped despite this comment and + // the help text both promising "descendants (recursive)" — do not + // mistake this for ListRequest.ParentID's one-level clause, which is a + // DIFFERENT query (PR #7199 review: an earlier pass here wrongly copied + // that one-level wording onto this field). ParentID string // MolType restricts to one molecule type. MolType *MolType @@ -351,6 +347,19 @@ type ListRequest struct { // ExcludeTypes entries may be comma-separated; splitting happens inside. ExcludeTypes []string + // ParentID restricts to one issue's DIRECT children only: a parent-child + // dependency edge, OR (for an issue with no such edge) a dotted-id prefix + // match — the same one-level sqlbuild.ParentID clause CountRequest.ParentID + // compiles to (internal/storage/sqlbuild/filter.go). It is NOT a recursive + // descendants walk — that is ReadyRequest.ParentID above, a DIFFERENT + // field on a different request, backed by GetDescendantIDsInTx (GH#3396). + // The recursive tree that `bd list --parent X` renders by DEFAULT comes + // from cmd/bd/list_show_filter_modes.go's findAllDescendants calling this + // ROLE once per level and accumulating the results client-side, not from + // the role answering a deeper question. `bd list --parent X --flat` (or + // --json) skips that client-side recursion and returns exactly this + // field's one-level answer, which is the invocation that agrees with + // `bd count --parent X`'s cardinality. ParentID string NoParent bool MolType *MolType From b67e9f0f379dfa7bb68046a1e954b12c70894d90 Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Mon, 5 Oct 2026 00:52:27 +0000 Subject: [PATCH 12/16] chore(wireshape): add the countIssues scope parameters to the wire-shape golden #7182 now digests operation parameters. The four new issues:count parameters (parent, no_parent, exclude_type, exclude_status) are additive entries, so no wire_revision bump is needed; regenerated with gendigest. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../httpapi/wireshape/testdata/golden.json | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index 608ede82a..7ce6dd492 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -4594,6 +4594,24 @@ "explode": true, "default": "false" }, + { + "schema": "param:countIssues", + "member": "query:exclude_status", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, + { + "schema": "param:countIssues", + "member": "query:exclude_type", + "type": "array", + "required": false, + "item_type": "string", + "style": "form", + "explode": true + }, { "schema": "param:countIssues", "member": "query:group_by", @@ -4688,6 +4706,13 @@ "explode": true, "default": "false" }, + { + "schema": "param:countIssues", + "member": "query:no_parent", + "type": "boolean", + "required": false, + "default": "false" + }, { "schema": "param:countIssues", "member": "query:notes_contains", @@ -4696,6 +4721,12 @@ "style": "form", "explode": true }, + { + "schema": "param:countIssues", + "member": "query:parent", + "type": "string", + "required": false + }, { "schema": "param:countIssues", "member": "query:priority", From 4ba6c8a9319e6d2c469f46031b88f857d9ac4b00 Mon Sep 17 00:00:00 2001 From: Eddie the Engineer Date: Mon, 5 Oct 2026 01:38:57 +0000 Subject: [PATCH 13/16] test(conformance): register the S8 count-scope cases with RunRoleContracts (adopt-pr #7199) The PR adds nine RunCounter* contract cases to counter_contract.go and wires them into the three in-tree TestCounterContract legs, but not into the Counter row of roleContractCases. TestRoleContractCasesCoverEveryContractCase derives the expected set from source, so it fails on the PR head, and an external backend running RunRoleContracts would never see the new cases. Register them in source-declaration order (after RunCounterAnUnknownStatusMatchesNothing), which the same guard enforces. Co-Authored-By: Claude Opus 5.5 --- backend/conformance/role_bundle_cases.go | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/backend/conformance/role_bundle_cases.go b/backend/conformance/role_bundle_cases.go index 95c565cb2..6f0218290 100644 --- a/backend/conformance/role_bundle_cases.go +++ b/backend/conformance/role_bundle_cases.go @@ -186,6 +186,15 @@ var roleContractCases = []roleContract{ RunCounterIncludeInfraExcludesGates, RunCounterCountsClosedRows, RunCounterAnUnknownStatusMatchesNothing, + RunCounterParentIDScopesToChildren, + RunCounterNoParentExcludesChildren, + RunCounterExcludeTypesNarrowsThePredicate, + RunCounterExcludeStatusNarrowsThePredicate, + RunCounterParentIDMatchesListCardinality, + RunCounterNoParentMatchesListCardinality, + RunCounterExcludeTypesMatchesListCardinality, + RunCounterParentIDIncludesAWispChild, + RunCounterParentIDAndExcludeStatusComposeOnAClosedChild, RunCounterGroupsPartitionTheScalarSet, RunCounterLabelBucketsOverlapSoTotalIsNotTheirSum, RunCounterNamesTheEmptyBuckets, From 044476f8ae511da6eb399cbf42f09eab6bab2c1c Mon Sep 17 00:00:00 2001 From: Eddie the Engineer Date: Mon, 5 Oct 2026 03:22:35 +0000 Subject: [PATCH 14/16] docs(count): correct the S8 scope docs the review found wrong (adopt-pr #7199) Comment and spec-description fixes only; no behavior change. Review-loop iterations 1 and 2 of the #7199 relaunch. - ListRequest.ParentID (issueops/reader.go) said "DIRECT children only ... the same one-level clause". The sqlbuild clause has two arms and neither walks: the parent-child edge arm stops at one level, but the dotted-prefix arm (`id LIKE CONCAT(?, '.%')` for rows with no parent-child edge) reaches every edge-less dotted id at any depth, which RunReaderListParentReachesEveryDescendantAndOnlyItsOwn pins with an edge-less X.1.1. State both arms, and say the CLI tree view re-runs the clause against the store once per level (findAllDescendants calls SearchIssues, not the role). Document ListRequest.NoParent, which had no doc. The contributor's ReadyRequest.ParentID text is kept; only its cross-reference to this field changes from "one-level" to "non-walking". - CountRequest.NoParent (issueops/counter.go) and the count flag comment (cmd/bd/count.go) said NoParent counts rows with "neither" an edge nor a dotted prefix. Its SQL checks only the edge table, so an edge-less X.1 is counted by both --parent X and --no-parent, as bd list lists it. - RunCounterExcludeStatusNarrowsThePredicate's header said an unrecognized name "excludes nothing rather than failing" while its body asserts ErrValidation. The header now matches the body. - The OpCountIssues entry in problem.go's operationCodes said the role has "exactly one reachable refusal". It has three: an invalid metadata key, an unrecognized exclude_status name, and parent with no_parent (named no_parent), all classified by invalidFilterParam. The code set is unchanged. - BuildCountFilter's header said cfg is "only read under IncludeInfra". It is also read to validate ExcludeStatus against the workspace's custom statuses, which is why both callers load it when ExcludeStatus is non-empty. - The countIssues `parent` description in openapi.v0.yaml carried the same one-level claim as ListRequest.ParentID; it now states both arms. types.gen.go is regenerated (make api-gen); make api-check passes. Review fixes (iteration 2), still comment and spec text only: - The iteration-1 problem.go rewrite called all three count refusals ErrValidation. Only the exclude_status and parent-with-no_parent refusals wrap issueops.ErrValidation (internal/workapi/count.go). The metadata-key refusal is ValidateMetadataFilters' plain fmt.Errorf chain, so errors.Is(err, ErrValidation) is false for it; it reaches the wire as a 400 only because invalidFilterParam matches message prefixes. The comment now says that, so a later move to typed errors cannot trust it and demote the metadata-key refusal to a 500. The sentinel itself is not added here: ValidateMetadataFilters is shared with the list and ready builders. - invalidFilterParam's comment on the count rows pointed at the metadata-key rows as "above"; they come after it in the switch. - The countIssues `parent` description said the CLI tree mode walks "every edge-linked descendant". findAllDescendants re-runs the full two-arm predicate per level, so it also descends through edge-less dotted ids; it now says "every descendant reachable through either arm". types.gen.go is regenerated; make api-check passes. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 --- backend/conformance/counter_contract.go | 5 ++-- cmd/bd/count.go | 8 +++--- internal/httpapi/apigen/types.gen.go | 2 +- internal/httpapi/problem.go | 16 +++++++---- internal/httpapi/reads.go | 6 ++--- internal/httpapi/spec/openapi.v0.yaml | 21 ++++++++------- internal/workapi/count.go | 7 +++-- issueops/counter.go | 5 +++- issueops/reader.go | 36 ++++++++++++++++--------- 9 files changed, 67 insertions(+), 39 deletions(-) diff --git a/backend/conformance/counter_contract.go b/backend/conformance/counter_contract.go index 5504424c5..b18f5fd3f 100644 --- a/backend/conformance/counter_contract.go +++ b/backend/conformance/counter_contract.go @@ -286,8 +286,9 @@ func RunCounterExcludeTypesNarrowsThePredicate(t *testing.T, ctx context.Context // RunCounterExcludeStatusNarrowsThePredicate pins CountRequest.ExcludeStatus // (counter.go, S8): a Count-only field with no List counterpart. Named -// statuses are excluded, and — like Status — an unrecognized name excludes -// nothing rather than failing (match-nothing-rather-than-fail, counter.go). +// statuses are excluded, and — UNLIKE Status — an unrecognized name is +// refused as ErrValidation rather than excluding nothing, because a typo +// would silently overcount (counter.go). func RunCounterExcludeStatusNarrowsThePredicate(t *testing.T, ctx context.Context, fixture CounterFixture) { t.Helper() open := fixture.IssuePrefix + "-exstatus-open" diff --git a/cmd/bd/count.go b/cmd/bd/count.go index b367cfda8..46ea46845 100644 --- a/cmd/bd/count.go +++ b/cmd/bd/count.go @@ -284,9 +284,11 @@ func registerCountFlags(cmd *cobra.Command) { // Hierarchy and exclusions (same spelling as `bd list`'s flags of the same // name, GH#4387's count/list parity): --parent and --no-parent restrict to - // a parent's children or to rows with neither; --exclude-type and - // --exclude-status narrow the predicate by type and by status, composing - // with --include-infra's own exclusions rather than replacing them. + // a parent's children or to rows with no parent-child edge (an edge-less + // dotted-id child still counts as top-level, as in `bd list --no-parent`); + // --exclude-type and --exclude-status narrow the predicate by type and by + // status, composing with --include-infra's own exclusions rather than + // replacing them. cmd.Flags().String("parent", "", "Filter by parent issue ID (count children of specified issue)") cmd.Flags().Bool("no-parent", false, "Count only top-level issues (exclude child issues)") cmd.Flags().StringSlice("exclude-type", nil, "Exclude issue types from the count (comma-separated or repeatable)") diff --git a/internal/httpapi/apigen/types.gen.go b/internal/httpapi/apigen/types.gen.go index 18bc84ac5..1edee6f7b 100644 --- a/internal/httpapi/apigen/types.gen.go +++ b/internal/httpapi/apigen/types.gen.go @@ -2464,7 +2464,7 @@ type CountIssuesParams struct { // Unset — and with `include_ephemeral` also unset — the count is durable-plane only and applies none of the four: the historical `bd count` answer, kept exactly so a scripted caller reads the same number it read yesterday. IncludeInfra *bool `form:"include_infra,omitempty" json:"include_infra,omitempty"` - // Parent Restrict to one issue's DIRECT children only: a parent-child dependency edge onto `parent`, OR — for an issue carrying no parent-child edge at all — a dotted-id prefix match (`parent.anything`). It is the same ONE-LEVEL predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` agrees in cardinality with `bd list --parent X --flat --all` (or `--json`) for the shared part of the predicate — NOT with a bare `bd list --parent X --all`, whose default tree mode recursively walks every descendant (grandchildren and below) by re-querying this same one-level predicate once per level client-side (cmd/bd/list_show_filter_modes.go). `--flat` and `--json` both skip that recursion and return this field's one-level answer directly. + // Parent Restrict to one issue's children through one predicate with two arms: a parent-child dependency edge onto `parent`, OR — for an issue carrying no parent-child edge at all — a dotted-id prefix match (`parent.anything`). NEITHER ARM WALKS. The edge arm stops at one level, so a grandchild with an edge of its own is not counted, while the prefix arm reaches every edge-less dotted id at any depth (`X.1.1` as well as `X.1`). It is the same predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` agrees in cardinality with `bd list --parent X --flat --all` (or `--json`) for the shared part of the predicate — NOT with a bare `bd list --parent X --all`, whose default tree mode walks every descendant reachable through either arm (grandchildren and below) by re-querying this same predicate once per level client-side (cmd/bd/list_show_filter_modes.go). `--flat` and `--json` both skip that walk and return this field's answer directly. // // Sending it beside `no_parent` IS refused: `invalid_argument` on `no_parent` with `reason: invalid_value`, matching `bd list`'s CLI refusal of the same combination. // diff --git a/internal/httpapi/problem.go b/internal/httpapi/problem.go index eeee88f25..31e336eac 100644 --- a/internal/httpapi/problem.go +++ b/internal/httpapi/problem.go @@ -741,11 +741,17 @@ var operationCodes = map[string][]Code{ // Its 400s come from both the transport and the ROLE. The transport refuses // malformed values, repeated single-valued parameters and a `group_by` // outside the closed set. countGroupOf stops that last case at the edge. - // The role has exactly one reachable refusal: BuildCountFilter rejects an - // invalid metadata key, from `metadata_field` or `has_metadata_key`. - // failReadErr classifies it through invalidFilterParam as a 400 naming the - // parameter it came from. An unrecognized status or type is not a refusal; - // the role promises it matches nothing and answers 0. + // The role has three reachable refusals, all from BuildCountFilter: an + // invalid metadata key (`metadata_field` or `has_metadata_key`), an + // unrecognized `exclude_status` name, and `parent` set together with + // `no_parent` (named `no_parent`). The last two carry ErrValidation; the + // metadata-key refusal does not, being a plain error from + // ValidateMetadataFilters, which the list and ready builders share. + // failReadErr classifies all three through invalidFilterParam by message + // prefix, not by sentinel, as a 400 naming the parameter each came from. + // An unrecognized `status` or `type` is not a refusal; the role promises + // it matches nothing and answers 0. Nor is an unrecognized `exclude_type`, + // which excludes nothing. OpCountIssues: {CodeInvalidArgument, CodeUnauthenticated, CodeBusy, CodeDBUnavailable, CodeInternal}, // The listing's vocabulary minus the cursor: this operation has none, so // invalid_cursor cannot arise. An unparseable EXPRESSION is an diff --git a/internal/httpapi/reads.go b/internal/httpapi/reads.go index 79caba90f..988f6298e 100644 --- a/internal/httpapi/reads.go +++ b/internal/httpapi/reads.go @@ -613,9 +613,9 @@ func invalidFilterParam(err error) (string, bool) { return "status", true // The count role's own refusals (S8 review fix): a misspelled // --exclude-status entry, and --parent set together with --no-parent. - // Both are the ROLE's ErrValidation, named here exactly as `status` and - // the metadata-key rows above are, so the client learns which parameter - // to fix rather than reading an unclassified 500. + // Both are the ROLE's ErrValidation. They are named here the same way as + // the `status` row above and the metadata-key rows below, so the client + // learns which parameter to fix rather than reading an unclassified 500. case strings.HasPrefix(msg, "invalid exclude-status "): return "exclude_status", true case strings.HasPrefix(msg, "--parent and --no-parent are mutually exclusive"): diff --git a/internal/httpapi/spec/openapi.v0.yaml b/internal/httpapi/spec/openapi.v0.yaml index fae2baa58..4b36dee2d 100644 --- a/internal/httpapi/spec/openapi.v0.yaml +++ b/internal/httpapi/spec/openapi.v0.yaml @@ -2071,19 +2071,22 @@ paths: - name: parent in: query description: >- - Restrict to one issue's DIRECT children only: a parent-child - dependency edge onto `parent`, OR — for an issue carrying no - parent-child edge at all — a dotted-id prefix match - (`parent.anything`). It is the same ONE-LEVEL predicate + Restrict to one issue's children through one predicate with two + arms: a parent-child dependency edge onto `parent`, OR — for an + issue carrying no parent-child edge at all — a dotted-id prefix + match (`parent.anything`). NEITHER ARM WALKS. The edge arm stops + at one level, so a grandchild with an edge of its own is not + counted, while the prefix arm reaches every edge-less dotted id at + any depth (`X.1.1` as well as `X.1`). It is the same predicate `GET /v0/beads/issues`'s own `parent` applies, so `bd count --parent X` agrees in cardinality with `bd list --parent X --flat --all` (or `--json`) for the shared part of the predicate — NOT with a bare `bd list --parent X --all`, whose default tree mode - recursively walks every descendant (grandchildren and below) by - re-querying this same one-level predicate once per level - client-side (cmd/bd/list_show_filter_modes.go). `--flat` and - `--json` both skip that recursion and return this field's - one-level answer directly. + walks every descendant reachable through either arm (grandchildren + and below) by re-querying this same predicate once per level + client-side + (cmd/bd/list_show_filter_modes.go). `--flat` and `--json` both + skip that walk and return this field's answer directly. Sending it beside `no_parent` IS refused: `invalid_argument` on diff --git a/internal/workapi/count.go b/internal/workapi/count.go index 9d2dd26fc..66c1f509b 100644 --- a/internal/workapi/count.go +++ b/internal/workapi/count.go @@ -43,8 +43,11 @@ func ValidateCountGroup(group issueops.CountGroup) (string, error) { // pinned by a golden-style test comparing this builder's output against // BuildListFilter's for the same request (count_test.go, GH#4387). // -// cfg supplies the workspace's infra vocabulary and is only read under -// IncludeInfra; a zero ListConfig falls back to the default infra set. +// cfg supplies the workspace's infra vocabulary and its custom status names. +// It is read under IncludeInfra AND whenever ExcludeStatus is non-empty (the +// exclusion is validated against the workspace's statuses), so a caller must +// load it in both cases; a zero ListConfig falls back to the default infra set +// and the built-in statuses only. func BuildCountFilter(in issueops.CountRequest, cfg ListConfig) (types.IssueFilter, error) { filter := types.IssueFilter{ TitleSearch: in.TitleSearch, diff --git a/issueops/counter.go b/issueops/counter.go index 03deb5760..1f97939f4 100644 --- a/issueops/counter.go +++ b/issueops/counter.go @@ -80,7 +80,10 @@ type CountRequest struct { // ParentID restricts to one issue's children, spelled as // ListRequest.ParentID spells it: a parent-child dependency edge, OR (for // an issue with no such edge) a dotted-id prefix match. NoParent restricts - // to rows with neither. + // to rows with no parent-child dependency edge. It does not consult the + // dotted-id convention, so an edge-less dotted-id child (X.1) still counts + // as top-level — it is counted by BOTH ParentID X and NoParent, as it is + // listed by both `bd list --parent X --flat` and `bd list --no-parent`. // // Setting both IS refused, as ErrValidation, UNLIKE Assignee/NoAssignee // above: `bd list`'s CLI already refuses `--parent` with `--no-parent` diff --git a/issueops/reader.go b/issueops/reader.go index a49892e03..3a7d3b0ce 100644 --- a/issueops/reader.go +++ b/issueops/reader.go @@ -60,7 +60,7 @@ type ReadyRequest struct { // GH#3396: this used to be a one-hop subquery against the dependency // table, so grandchildren were silently dropped despite this comment and // the help text both promising "descendants (recursive)" — do not - // mistake this for ListRequest.ParentID's one-level clause, which is a + // mistake this for ListRequest.ParentID's non-walking clause, which is a // DIFFERENT query (PR #7199 review: an earlier pass here wrongly copied // that one-level wording onto this field). ParentID string @@ -347,19 +347,29 @@ type ListRequest struct { // ExcludeTypes entries may be comma-separated; splitting happens inside. ExcludeTypes []string - // ParentID restricts to one issue's DIRECT children only: a parent-child - // dependency edge, OR (for an issue with no such edge) a dotted-id prefix - // match — the same one-level sqlbuild.ParentID clause CountRequest.ParentID - // compiles to (internal/storage/sqlbuild/filter.go). It is NOT a recursive - // descendants walk — that is ReadyRequest.ParentID above, a DIFFERENT - // field on a different request, backed by GetDescendantIDsInTx (GH#3396). + // ParentID restricts to one issue's children through one SQL clause, the + // same one CountRequest.ParentID compiles to + // (internal/storage/sqlbuild/filter.go): a parent-child dependency edge + // onto ParentID, OR, for a row with no parent-child edge at all, an id + // under ParentID's dotted prefix ("X.1", "X.1.1"). NEITHER ARM WALKS. The + // edge arm stops at one level, so a grandchild with an edge of its own is + // not in the answer, while the prefix arm reaches every edge-less dotted + // id at any depth (backend/conformance/reader_contract.go pins that arm). + // It is NOT the recursive descendants walk — that is ReadyRequest.ParentID + // above, a DIFFERENT field on a different request, backed by + // GetDescendantIDsInTx (GH#3396); on the ReadyFlag arm this field crosses + // to that query and resolves recursively there. // The recursive tree that `bd list --parent X` renders by DEFAULT comes - // from cmd/bd/list_show_filter_modes.go's findAllDescendants calling this - // ROLE once per level and accumulating the results client-side, not from - // the role answering a deeper question. `bd list --parent X --flat` (or - // --json) skips that client-side recursion and returns exactly this - // field's one-level answer, which is the invocation that agrees with - // `bd count --parent X`'s cardinality. + // from cmd/bd/list_show_filter_modes.go's findAllDescendants re-running + // this clause against the store once per level and accumulating the + // results client-side, not from this field answering a deeper question. + // --flat, --json and --format skip that walk and return exactly this + // field's answer, which is why the flat listing is the invocation that + // agrees with `bd count --parent X`'s cardinality. + // + // NoParent restricts to rows with no parent-child dependency edge. It has + // no dotted-id arm, so an edge-less "X.1" matches BOTH ParentID X and + // NoParent. The ReadyFlag arm refuses it; see that field. ParentID string NoParent bool MolType *MolType From 89a387de6c2920ef66f4293e6526c3e0323566eb Mon Sep 17 00:00:00 2001 From: Eddie the Engineer Date: Mon, 5 Oct 2026 05:27:16 +0000 Subject: [PATCH 15/16] chore(wireshape): re-record the count scope params under the base's param digest (adopt-pr #7199) The base branch (feat/wire-revision-signal, #7182 be1f0bad72) now digests the effective style/explode of scalar query parameters. This PR's golden entries for countIssues query:parent and query:no_parent were written by the old algorithm, so after the base refresh TestWireShapeDigest reported them as "changed without a wire_revision bump". No bump is owed: all four count-scope entries (parent, no_parent, exclude_type, exclude_status) are additions this PR introduces and are absent from the base golden. golden.json was reset to the base copy and regenerated with `go run ./internal/httpapi/wireshape/cmd/gendigest`. The result is four pure additions against the base (+35/-0); parent and no_parent gain "style": "form" and "explode": true, and wire_revision stays 2. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 --- internal/httpapi/wireshape/testdata/golden.json | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/internal/httpapi/wireshape/testdata/golden.json b/internal/httpapi/wireshape/testdata/golden.json index 7ce6dd492..30ff61ebe 100644 --- a/internal/httpapi/wireshape/testdata/golden.json +++ b/internal/httpapi/wireshape/testdata/golden.json @@ -4711,6 +4711,8 @@ "member": "query:no_parent", "type": "boolean", "required": false, + "style": "form", + "explode": true, "default": "false" }, { @@ -4725,7 +4727,9 @@ "schema": "param:countIssues", "member": "query:parent", "type": "string", - "required": false + "required": false, + "style": "form", + "explode": true }, { "schema": "param:countIssues", From aca89d2bfdf1e1a34ea14864560b3c1234004d65 Mon Sep 17 00:00:00 2001 From: Eddie the Engineer Date: Mon, 5 Oct 2026 09:35:36 +0000 Subject: [PATCH 16/16] ci: place the PR's new embedded counter test in the shard manifest TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra, added to internal/storage/embeddeddolt in f23bf5ddbf, is a new top-level embedded storage test. The 15-shard block of .github/scripts/embedded-storage-test-shards.txt must list every discovered test exactly once. Without the entry, TestStorageEmbeddedShardManifestGeneratorNotStale (scripts/) fails, both on the reviewed head 89a387de6c and on the merge. CI never ran the check on this PR: pr.yml runs only for PRs based on main, release/** or hotfix/**, and this PR was based on feat/wire-revision-signal until the retarget. Generated with `python3 scripts/ci/gen_embedded_storage_shard_manifest.py 15 --weights=duration --write`. The incremental write adds this one line, in shard 4, and moves no other test. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 --- .github/scripts/embedded-storage-test-shards.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/scripts/embedded-storage-test-shards.txt b/.github/scripts/embedded-storage-test-shards.txt index 8723a463d..1502ba993 100644 --- a/.github/scripts/embedded-storage-test-shards.txt +++ b/.github/scripts/embedded-storage-test-shards.txt @@ -211,6 +211,7 @@ 15 3 TestUpdateMergeOps_DoltEngine_TwoIssues 15 3 TestWithPeerAuth_RestoresAbsentEnv +15 4 TestCounterExcludeStatusAcceptsAWorkspaceCustomStatusWithoutIncludeInfra 15 4 TestEmbeddedBatchCloserSettlesTheDependersOfWhatItClosed 15 4 TestEmbeddedBlockedRecheckSettlesWhatTheWriteRecorded 15 4 TestEmbeddedClaimIssueWrapperClaimsTheWispPlane