Review iteration 1 of the relaunched adoption raised six majors. This commit fixes all six. Five are the majors the first adoption's iteration 1 raised. Its fixup, 696803f250 (tag adopt-pr/bd-f5illd/stranded-fixup-696803f250), fixed them but never reached the branch. That fix is ported here onto the contributor's current head, with its reasoning unchanged; the sixth is new. Creates refused what the CLI sends. Every `bd create` carries CreatedBy, a workspace whose config.yaml names a prefix also carries IDPrefix, and the client refused both. - The server stamps created_by from the actor on all three create shapes: batch_create.go and batch_apply.go now match create.go. - The client sends a CreatedBy that names the actor and refuses any other value, which the server's stamp would silently replace. createIssue and the apply create item share refuseUnwirableIssueMembers. - IDPrefix refuses only where the role would act on it: an explicit, unforced id. - `bd create --file` stays refused over http. Dropping its Status was safe but not enough: the batch create still refuses its Provenance and Owner. The markdown.go and batchcreator.go comments that called it servable now say so. keysetFilter ignored AfterPriority. It now compares priority first, mirroring sqlbuild's KeysetPriorityCreatedAtIDPredicate, and a new served case resumes a priority keyset position. AfterID or AfterPriority without AfterCreatedAt no longer reads as a zero-instant position that discarded every row (the AfterID minor). Claim with a patch was refused on a false ledger claim. `claim`, `force_assignee_transfer` and `force_close_policy` now ride updateBody like `force_notes_overwrite`, so a claim with a patch or a version guard is one updateIssue call. A claim ALONE sent to a server that predates the member (#6890) falls back to claimIssue. The three ledger rows are retired, and the served cases parked on them now run. OpenReadOnly returns a writable store. It stays writable, as a documented exception: backends.Backend.OpenReadOnly also serves the root pre-run's classified reads, and `bd ready --claim` writes through that store. The exception is named in the backends.Backend contract, the store_factory invariant and the store doc, and TestReadOnlyOpenServesTheClaimBdReadyMakes pins it. An OpenNonMutating backend hook is the follow-up. Security surface. - http_target.json is in the doctor gitignore template and its required patterns. - BEADS_HTTP_TOKEN and BEADS_HTTP_TOKEN_COMMAND are host-scoped with BEADS_HTTP_CA_FILE's "host[:port]=value" syntax. A value scoped to another server falls through to the next rung, and a command scoped elsewhere never runs. - LoadTarget and SaveTarget refuse a url that carries userinfo, and Target.String drops userinfo. The served Tier 3 had no CI home. The new target //internal/httpclient:httpclient_served_test is an `embedded`-tagged sh_test. It reruns httpclient_test's binary through tools/bazel/go_test_variant.sh with the script's BEADS_TEST_EMBEDDED_DOLT=1 and BEADS_HTTP_TEST_REQUIRED=1, in 24 shards on the embedded lane's race build. bazel.yml's bazel-embedded job already runs `bazel test //... --config=embedded`, so no workflow changes. - -test.timeout=19m, not the script's 40m: Go's own timeout then fires, and dumps every goroutine, before the action's 1200s limit. - TestBazelRetiredLanesCannotBeNarrowed pins its args and env. The new TestConformanceServedTierRunsUnderBazel ties the script's Tier 3 switches to the rule. - The new untagged TestServedTierIsLinkedWhenRequired fails a required run built without cgo, which drops every served file (the sentinel minor). - A first 12-shard run found two test defects; both are fixed. TestServedBatchApplyBoundsTheItemCount's 1000-item apply outlasts the client's 60s per-request timeout under -race, so a race_detector_{on,off}_test.go pair scales its "at bound" half to 150 there, as embeddeddolt's TestBatchApplyContract does. And TestTheCompositionCatchesAWrongParameter's dropped-filter case leaned on rows earlier tests had seeded: run alone, or first in its shard, it failed. It now seeds its own row outside the scope. - conformance.sh, its script test and engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md name the target. Deviations from the review's suggested fixes: - A bare BEADS_HTTP_TOKEN(_COMMAND) fails closed rather than falling back, and the error does not echo the value: a bare token split at an "=" would put token text into the pattern half. Dropping the bare syntax is safe today: the backend has no Register call site and no docs/ page names the variables. - Target.String drops userinfo instead of using url.URL.Redacted, which masks only the password, so a token used as the username would print. - The pre-#6890 claimIssue fallback leaves RowVersion unset instead of borrowing the follow-up read's token. That read is a second snapshot, so its token could postdate a write this claim never saw and arm a guard that should miss; a guard built from the unset token fails closed. Against a current server the claim rides updateIssue, whose `revision` is stitched as for any update. Also fixed: the nit about the completeness failure message, which now says to regenerate the shell, and the offrole.go tombstone comment that named a skip list which no longer exists. Left for follow-up (all latent, since httpclient.Register has no production caller): vocabOnce latching a failed fetch, snapshot holding s.mu across the handshake dial, unchunked graph anchors, private tracker IDs in park owners, and the bounded journal limit above 10000. Validation: - git diff --check; gofmt -l; go vet over internal/httpclient/... and scripts - go test -tags gms_pure_go over internal/httpapi/..., internal/httpclient/..., internal/storage/backends/... and backend/conformance/...; the doctor gitignore tests; the cmd/bd events-journal construction guard; ./scripts - bazel test --config=embedded //internal/httpclient:httpclient_served_test on the lane's remote executors: 24 shards, 702 top-level tests, 632 pass, 70 skip (69 ledgered KNOWN DIVERGENCE, plus TestCAFileHygieneRefusesGroupWritableFile, which finds no supplementary group on the executor), 0 fail, 0 data races; slowest shard 283s. tools/bazel/check_testcases.py --bep passes. - each of the 702 tests run alone (go test -run '^Name$', without -race): 0 fail - the whole tier under go test without -race, with the script's switches and -timeout=40m, before the two test fixes: 633 pass, 69 skip, 0 fail - removing BEADS_HTTP_TEST_REQUIRED from the rule fails both policy tests - bazel test --config=ci over //internal/httpclient/..., //internal/httpapi/..., //scripts/..., //test/docsync/..., //internal/storage/backends/..., //backend/conformance/... and //cmd/bd/doctor/...: 32 tests pass - make ci-pr-lint: nogo native plus windows/darwin, passes Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
373 lines
15 KiB
Go
373 lines
15 KiB
Go
package httpapi
|
|
|
|
import (
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"net/http"
|
|
"slices"
|
|
"strings"
|
|
|
|
"github.com/steveyegge/beads/internal/httpapi/apigen"
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/types"
|
|
"github.com/steveyegge/beads/issueops"
|
|
)
|
|
|
|
const (
|
|
// maxBatchCreateItems is the document's cap on `items`. It bounds how long
|
|
// one request may hold a write transaction — not batch semantics, which
|
|
// have no size in them.
|
|
maxBatchCreateItems = 100
|
|
// maxBatchCreateBodyBytes bounds the request body. A hundred items each
|
|
// carrying a description, a design and acceptance criteria is the shape this
|
|
// has to admit, so it refuses the absurd before any of it is parsed.
|
|
maxBatchCreateBodyBytes = 4 << 20
|
|
)
|
|
|
|
// batchCreateMembers is the document's member list at each level. The schemas
|
|
// are additionalProperties: false, so anything else is refused BY NAME, which is
|
|
// why these are decoded as raw members first: encoding/json's
|
|
// DisallowUnknownFields reports the offender only inside an error string.
|
|
var (
|
|
batchCreateRequestMembers = []string{"actor", "items"}
|
|
batchCreateItemMembers = []string{"title", "description", "design", "acceptance_criteria", "priority", "issue_type", "assignee", "labels", "dependencies"}
|
|
batchCreateDependencyMembers = []string{"target_id", "type"}
|
|
)
|
|
|
|
// handleBatchCreate creates every issue in the request body, or none of them.
|
|
//
|
|
// The create-only guard, the vocabulary, the id assignment, the wisp routing,
|
|
// the retry and the history entry all belong to issueops.BatchCreator. This
|
|
// function decodes a body, refuses the shapes the document refuses, and
|
|
// marshals what came back.
|
|
//
|
|
// It shares the claim's posture exactly: the actor is caller-ASSERTED
|
|
// provenance and not authenticated identity, hooks do not fire, and the
|
|
// per-command auto-commit machinery never runs. The only durable effect is the
|
|
// single storage commit the role makes inside its own transaction.
|
|
func (s *Server) handleBatchCreate(w http.ResponseWriter, r *http.Request) {
|
|
if !s.requireNoQuery(w, r) {
|
|
return
|
|
}
|
|
if !s.requireJSONContent(w, r) {
|
|
return
|
|
}
|
|
request, ok := s.batchCreateRequest(w, r)
|
|
if !ok {
|
|
return
|
|
}
|
|
|
|
creator, err := s.batchCreator(r)
|
|
if err != nil {
|
|
s.failBatchCreate(w, r, err)
|
|
return
|
|
}
|
|
result, err := creator.CreateBatch(r.Context(), request)
|
|
if err != nil {
|
|
s.failBatchCreate(w, r, err)
|
|
return
|
|
}
|
|
items := make([]types.Issue, len(result.Issues))
|
|
for i, issue := range result.Issues {
|
|
items[i] = *issue
|
|
}
|
|
writeJSON(w, apigen.BatchCreateResponse{Items: items})
|
|
}
|
|
|
|
// batchCreateRequest decodes and validates the body, and reports whether the
|
|
// request may proceed.
|
|
//
|
|
// Every refusal here happens BEFORE any database work, which is what lets the
|
|
// 400s reflect the caller's own input back. The role's own refusals cannot be
|
|
// that specific without quoting a storage error into a response, so
|
|
// failBatchCreate answers those in the server's own words.
|
|
func (s *Server) batchCreateRequest(w http.ResponseWriter, r *http.Request) (issueops.CreateBatchRequest, bool) {
|
|
members, res := decodeJSONObject(w, r, maxBatchCreateBodyBytes)
|
|
if res != nil {
|
|
s.fail(w, r, *res)
|
|
return issueops.CreateBatchRequest{}, false
|
|
}
|
|
if offender, unknown := unknownMember(members, batchCreateRequestMembers); unknown {
|
|
s.failUnknownMember(w, r, offender, batchCreateRequestMembers)
|
|
return issueops.CreateBatchRequest{}, false
|
|
}
|
|
|
|
actor, ok := s.bodyActor(w, r, members)
|
|
if !ok {
|
|
return issueops.CreateBatchRequest{}, false
|
|
}
|
|
items, ok := s.batchCreateItems(w, r, members)
|
|
if !ok {
|
|
return issueops.CreateBatchRequest{}, false
|
|
}
|
|
// created_by is stamped from the actor on every item, createIssue's rule
|
|
// (create.go): the item publishes no created_by and the role copies the
|
|
// issue's rather than stamping one, so without this a batch-created issue
|
|
// would store none where `bd create --file` stores its actor.
|
|
for _, item := range items {
|
|
item.Issue.CreatedBy = actor
|
|
}
|
|
return issueops.CreateBatchRequest{Actor: actor, Items: items}, true
|
|
}
|
|
|
|
// batchCreateItems validates `items` and projects it onto the role's items.
|
|
func (s *Server) batchCreateItems(w http.ResponseWriter, r *http.Request, members map[string]json.RawMessage) ([]issueops.BatchCreateItem, bool) {
|
|
raw, ok := members["items"]
|
|
if !ok {
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue, "`items` is required"))
|
|
return nil, false
|
|
}
|
|
var rawItems []map[string]json.RawMessage
|
|
if err := json.Unmarshal(raw, &rawItems); err != nil || rawItems == nil {
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue, "`items` must be an array of objects"))
|
|
return nil, false
|
|
}
|
|
switch {
|
|
case len(rawItems) == 0:
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue,
|
|
"`items` must carry at least one issue; a create that creates nothing is refused rather than answered"))
|
|
return nil, false
|
|
case len(rawItems) > maxBatchCreateItems:
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue,
|
|
fmt.Sprintf("`items` carries %d issues; the limit is %d per request", len(rawItems), maxBatchCreateItems)))
|
|
return nil, false
|
|
}
|
|
|
|
items := make([]issueops.BatchCreateItem, 0, len(rawItems))
|
|
for i, rawItem := range rawItems {
|
|
if rawItem == nil {
|
|
s.fail(w, r, InvalidArgument(batchCreateItemParam(i, ""), ReasonInvalidValue, "an item must be a JSON object"))
|
|
return nil, false
|
|
}
|
|
if offender, unknown := unknownMember(rawItem, batchCreateItemMembers); unknown {
|
|
s.failUnknownMember(w, r, batchCreateItemParam(i, offender), batchCreateItemMembers)
|
|
return nil, false
|
|
}
|
|
item, res := batchCreateItem(i, rawItem)
|
|
if res != nil {
|
|
s.fail(w, r, *res)
|
|
return nil, false
|
|
}
|
|
items = append(items, item)
|
|
}
|
|
return items, true
|
|
}
|
|
|
|
// batchCreateItem projects one decoded item onto the role's item, or reports the
|
|
// refusal it earned. It decodes into the GENERATED struct, which is what makes a
|
|
// member's type the document's type: `priority: "high"` is refused here rather
|
|
// than reaching a role that would have to guess what the caller meant.
|
|
func batchCreateItem(index int, raw map[string]json.RawMessage) (issueops.BatchCreateItem, *Result) {
|
|
refuse := func(member, detail string) *Result {
|
|
res := InvalidArgument(batchCreateItemParam(index, member), ReasonInvalidValue, detail)
|
|
return &res
|
|
}
|
|
encoded, err := json.Marshal(raw)
|
|
if err != nil {
|
|
return issueops.BatchCreateItem{}, refuse("", "an item must be a JSON object")
|
|
}
|
|
var wire apigen.BatchCreateItem
|
|
if err := json.Unmarshal(encoded, &wire); err != nil {
|
|
return issueops.BatchCreateItem{}, refuse("", "an item member carries the wrong JSON type")
|
|
}
|
|
if strings.TrimSpace(wire.Title) == "" {
|
|
return issueops.BatchCreateItem{}, refuse("title", "`title` is required and must not be blank")
|
|
}
|
|
if types.CheckFieldLen("title", wire.Title) != nil {
|
|
return issueops.BatchCreateItem{}, refuse("title",
|
|
fmt.Sprintf("`title` is longer than the %d characters storage holds", types.MaxFieldLen))
|
|
}
|
|
// The role validates the type against the workspace's configured vocabulary,
|
|
// which this server cannot read without a transaction; what is checked here
|
|
// is only what the schema declares. A SLICE and not a map, so that an item
|
|
// breaking both rules always names the same offender: `param` is what a
|
|
// client dispatches on, and it must not depend on map order.
|
|
for _, bounded := range []struct {
|
|
member string
|
|
value *string
|
|
}{{"assignee", wire.Assignee}, {"issue_type", wire.IssueType}} {
|
|
if bounded.value != nil && types.CheckFieldLen(bounded.member, *bounded.value) != nil {
|
|
return issueops.BatchCreateItem{}, refuse(bounded.member,
|
|
fmt.Sprintf("`%s` is longer than the %d characters storage holds", bounded.member, types.MaxFieldLen))
|
|
}
|
|
}
|
|
issue := &types.Issue{
|
|
Title: wire.Title,
|
|
Description: derefString(wire.Description),
|
|
Design: derefString(wire.Design),
|
|
AcceptanceCriteria: derefString(wire.AcceptanceCriteria),
|
|
Status: types.StatusOpen,
|
|
Assignee: derefString(wire.Assignee),
|
|
IssueType: types.IssueType(derefString(wire.IssueType)),
|
|
}
|
|
if wire.Priority != nil {
|
|
if *wire.Priority < 0 || *wire.Priority > 4 {
|
|
return issueops.BatchCreateItem{}, refuse("priority",
|
|
fmt.Sprintf("`priority` is %d; the range is 0 to 4", *wire.Priority))
|
|
}
|
|
issue.Priority = *wire.Priority
|
|
}
|
|
if wire.Labels != nil {
|
|
for _, label := range *wire.Labels {
|
|
if types.CheckFieldLen("label", label) != nil {
|
|
return issueops.BatchCreateItem{}, refuse("labels",
|
|
fmt.Sprintf("a label is longer than the %d characters storage holds", types.MaxFieldLen))
|
|
}
|
|
}
|
|
issue.Labels = *wire.Labels
|
|
}
|
|
item := issueops.BatchCreateItem{Issue: issue}
|
|
if wire.Dependencies == nil {
|
|
return item, nil
|
|
}
|
|
rawEdges := rawDependencyMembers(raw)
|
|
for j, dependency := range *wire.Dependencies {
|
|
if offender, unknown := unknownMember(rawEdgeAt(rawEdges, j), batchCreateDependencyMembers); unknown {
|
|
res := InvalidArgument(batchCreateItemParam(index, "dependencies."+offender), ReasonUnknownParameter,
|
|
"an edge carries "+strings.Join(batchCreateDependencyMembers, ", ")+" and nothing else")
|
|
return issueops.BatchCreateItem{}, &res
|
|
}
|
|
if dependency.TargetId == "" || types.CheckFieldLen("target_id", dependency.TargetId) != nil {
|
|
return issueops.BatchCreateItem{}, refuse("dependencies.target_id",
|
|
"`target_id` is required and must be at most "+fmt.Sprint(types.MaxFieldLen)+" characters")
|
|
}
|
|
// A value at all, and one the column holds — never membership of a
|
|
// known-types list. The edge vocabulary is OPEN, as EdgeReadRequest.Types
|
|
// says, so a workspace's own type passes and only an unstorable one is
|
|
// refused.
|
|
if !types.DependencyType(dependency.Type).IsValid() {
|
|
return issueops.BatchCreateItem{}, refuse("dependencies.type",
|
|
fmt.Sprintf("`type` must be 1 to %d characters", types.MaxDependencyTypeLen))
|
|
}
|
|
item.Dependencies = append(item.Dependencies, issueops.CreateDependency{
|
|
TargetID: dependency.TargetId,
|
|
Type: types.DependencyType(dependency.Type),
|
|
})
|
|
}
|
|
return item, nil
|
|
}
|
|
|
|
// rawDependencyMembers decodes an item's edges as raw members, so an unknown
|
|
// member INSIDE an edge is refused by name like every other one. It answers nil
|
|
// — carrying no unknown member — when the shape is not the one the generated
|
|
// decode accepted, because a disagreement between the two decodes is not a
|
|
// client-attributable refusal.
|
|
func rawDependencyMembers(item map[string]json.RawMessage) []map[string]json.RawMessage {
|
|
raw, ok := item["dependencies"]
|
|
if !ok {
|
|
return nil
|
|
}
|
|
var edges []map[string]json.RawMessage
|
|
if err := json.Unmarshal(raw, &edges); err != nil {
|
|
return nil
|
|
}
|
|
return edges
|
|
}
|
|
|
|
// rawEdgeAt is the bounds-checked read of one decoded edge.
|
|
func rawEdgeAt(edges []map[string]json.RawMessage, j int) map[string]json.RawMessage {
|
|
if j >= len(edges) {
|
|
return nil
|
|
}
|
|
return edges[j]
|
|
}
|
|
|
|
// batchCreateItemParam spells the `param` member for a refusal inside `items`,
|
|
// so a client dispatching on it learns WHICH item and WHICH member.
|
|
func batchCreateItemParam(index int, member string) string {
|
|
param := fmt.Sprintf("items[%d]", index)
|
|
if member == "" {
|
|
return param
|
|
}
|
|
return param + "." + member
|
|
}
|
|
|
|
// failUnknownMember answers an unknown body member the way an unknown query
|
|
// parameter is answered: one offender, chosen deterministically, and the
|
|
// allowed set in the detail.
|
|
func (s *Server) failUnknownMember(w http.ResponseWriter, r *http.Request, offender string, allowed []string) {
|
|
requestInfo(r.Context()).refuse(offender)
|
|
s.fail(w, r, InvalidArgument(offender, ReasonUnknownParameter,
|
|
"this operation's request body carries "+strings.Join(allowed, ", ")+" and nothing else"))
|
|
}
|
|
|
|
// failBatchCreate answers a failed batch.
|
|
//
|
|
// The role's ErrValidation is answered HERE rather than in ClassifyError: the
|
|
// role wraps its dangling-edge refusal in BOTH ErrValidation and ErrNotFound,
|
|
// so the shared classifier — which reaches ErrNotFound first — would answer 404
|
|
// for a request that addressed no resource at all.
|
|
//
|
|
// NEITHER BRANCH QUOTES THE ROLE'S MESSAGE. A refused edge arrives as a driver
|
|
// error naming tables and constraints, and 4xx details on this surface reflect
|
|
// the caller's own input back rather than server internals. The real error goes
|
|
// to the log with the request id.
|
|
func (s *Server) failBatchCreate(w http.ResponseWriter, r *http.Request, err error) {
|
|
if !errors.Is(err, storage.ErrValidation) {
|
|
s.failErr(w, r, err)
|
|
return
|
|
}
|
|
// The 4xx path does not log by default, so this is the one place the real
|
|
// refusal is recorded for the operator.
|
|
s.event("request_refused", "request_id", requestInfo(r.Context()).id, "error", err.Error())
|
|
if errors.Is(err, storage.ErrNotFound) {
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue,
|
|
"a dependency target names no issue in this workspace; nothing was created"))
|
|
return
|
|
}
|
|
s.fail(w, r, InvalidArgument("items", ReasonInvalidValue,
|
|
"an item was refused by this workspace's own validation; nothing was created"))
|
|
}
|
|
|
|
// unknownMember reports the smallest member name outside allowed, chosen
|
|
// deterministically so a client dispatching on `param` never sees it depend on
|
|
// map order.
|
|
func unknownMember(members map[string]json.RawMessage, allowed []string) (string, bool) {
|
|
var unknown []string
|
|
for name := range members {
|
|
if !slices.Contains(allowed, name) {
|
|
unknown = append(unknown, name)
|
|
}
|
|
}
|
|
if len(unknown) == 0 {
|
|
return "", false
|
|
}
|
|
return slices.Min(unknown), true
|
|
}
|
|
|
|
// decodeJSONObject reads the body as a JSON object of raw members, bounded by
|
|
// limit. It is decodeClaimBody generalized over the bound, and it makes the
|
|
// same three refusals for the same reasons.
|
|
func decodeJSONObject(w http.ResponseWriter, r *http.Request, limit int64) (map[string]json.RawMessage, *Result) {
|
|
// A body with no nameable part: `param` is documented absent on exactly
|
|
// this case and present on every other 400.
|
|
unparseable := func(detail string) *Result {
|
|
res := InvalidArgument("", ReasonInvalidValue, detail)
|
|
return &res
|
|
}
|
|
var members map[string]json.RawMessage
|
|
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, limit))
|
|
if err := dec.Decode(&members); err != nil {
|
|
var tooLarge *http.MaxBytesError
|
|
if errors.As(err, &tooLarge) {
|
|
return nil, unparseable(fmt.Sprintf("request body is larger than %d bytes", limit))
|
|
}
|
|
return nil, unparseable("request body must be a JSON object")
|
|
}
|
|
if members == nil {
|
|
// Valid JSON, but `null`: no members to read anything out of.
|
|
return nil, unparseable("request body must be a JSON object")
|
|
}
|
|
if dec.More() {
|
|
return nil, unparseable("request body must be a single JSON object")
|
|
}
|
|
return members, nil
|
|
}
|
|
|
|
func derefString(value *string) string {
|
|
if value == nil {
|
|
return ""
|
|
}
|
|
return *value
|
|
}
|