beads/cmd/bd/batch.go
Julian Knutsen d15958a5eb
fix(issueops): enforce the dotted-child dependency rule on every leg (#7424)
* fix(issueops): enforce the dotted-child dependency rule on every leg

The refusal of an explicit edge from a dotted-id child to its own ancestor
(bd-abc.1 -> bd-abc; only the parent-child edge to the immediate dotted
parent is allowed) lived only in cmd/bd (isDisallowedHierarchicalDependency).
HTTP dependencies:add, batch:apply dep_add items and `bd create --graph`'s own
edges skipped it.

It is now one library rule, issueops.CheckDottedChildDependency, returning
*issueops.DottedChildDependencyError (unwraps to ErrValidation, same message
the CLI printed):
- ValidateAddDependenciesRequest applies it, so DependencyEditor refuses on
  dolt, embedded, uow and the server behind httpapi (400 invalid_argument);
- both BatchApplier bodies (issueops and uow) apply it to each dep_add item on
  the resolved ids, so a key bound to a minted dotted child is refused too;
- the HTTP client checks it before the dial, as it does self-dependency, so
  the typed error survives the wire.

cmd/bd keeps no copy: single-edge dep add / dep --blocks rely on the role;
bd link (still on the store verb) and the --file bulk reporter call the
library rule. CLI output is byte-identical (checked against origin/main).

Conformance: RunDependencyEditorRefusesADottedChildGatedOnItsOwnParent and
RunBatchApplyRefusesADottedChildGatedOnItsOwnParent wired on dolt, embedded,
uow and http, plus a raw-wire served test past the client check and handler
mapping cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(batch): refuse a dotted child gated on its own ancestor in bd batch

Review fixup for PR #7424 (iterations 1 and 2).

Major: a bd batch script line `dep add bd-abc.1 bd-abc` still wrote the
edge this PR refuses everywhere else, on both backends. Neither batch
dispatch writes through the DependencyEditor role that enforces the
dotted-child rule: the classic arm calls storage.Transaction.AddDependency
and the proxied arm the unit of work's AddDependencies verb. Both arms now
build the edge with one helper, batchDepAddEdge, which keeps the default
type and the arity and type errors, and asks
publicops.CheckDottedChildDependency the way bd link does. A refused line
rolls the whole batch back.

Tests: TestBatchDepAddEdge (default and explicit types, arity and type
errors, three refusals typed *DottedChildDependencyError matching
ErrValidation, and the immediate parent-child edge allowed), and
TestProxiedServerBatchDepAddRefusesADottedChildGatedOnItsParent (classic
and proxied: the refusal names line 2, rolls back an earlier update line
and writes no edge, and `dep add C P parent-child` still lands exactly
one), registered in the proxied shard manifest. A mutant that drops the
rule from the shared helper fails both; one that bypasses the helper in
the proxied arm alone fails the proxied leg.

Minors:
- For a child whose parent-child edge is already stored (a
  `bd create --parent` child) the rule answers ahead of the hierarchy and
  type conflicts, so an HTTP caller now gets 400 invalid_argument where it
  got 409 dependency_cycle (blocking types) or dependency_exists (any
  other type). Documented in the DependencyEditor role doc, the CHANGELOG
  entry and the httpapi batch-apply test comment, and pinned by a
  stored-hierarchy phase in the DependencyEditor and BatchApplier
  dotted-child conformance contracts on every leg; a mutant that drops the
  rule gets the hierarchy conflict instead (409 dependency_cycle on the
  wire).
- The bd create --graph dry-run note now says the live create may also
  refuse an edge that gates a dotted-id child on its own ancestor.
- The DottedChildDependencyError comment says the message is kept verbatim
  and says "deadlock" whatever the type.

Not changed: bd relate (relates-to does not schedule) and the edges a
create writes with its issue (bd create --deps/--waits-for, and over
HTTP an explicit dotted id with a non-immediate ParentID) still write
through paths that skip the rule. The scheduling shapes need an explicit
dotted id or a grandparent target. Follow-up bd-25s1wh tracks moving the
rule down to the domain/store add verbs, with a structural bypass for
reparent and template edges.

Iteration 2 (minor): the CheckDottedChildDependency doc no longer implies
bd link and bd batch are the only writers outside the roles. It names
bd relate and the create edges as not applying the rule yet and cites
bd-25s1wh.

Complexity (cyclomatic/cognitive): runBatchOp 23/35 -> 21/31,
runBatchOpUOW 28/48 -> 26/44, batchDepAddEdge 5/4. Touching them brings
both pre-existing dispatch switches into the measured set; splitting them
is out of scope here.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(issueops,httpapi): document the dotted-child 400 and widen its gap list

Review fixup for PR #7424 (relaunch, iteration 1, minors).

- The OpenAPI spec now describes the wire change this PR makes. The
  dependencies:add and issues:batchApply 400 vocabularies name an edge
  from a dotted-id child to its own ancestor (anything but the
  parent-child edge to its immediate parent; param `edges` and `items`).
  dependencies:add says why that is request-intrinsic, and both 409s say
  it is never dependency_cycle or dependency_exists, even for a child
  whose parent-child edge is stored. Response descriptions only: the
  generated types and the wireshape digest are unchanged (make api-check).
- The CheckDottedChildDependency doc no longer reads as a complete list of
  the writers that skip the rule. Besides bd relate and the create edges
  it names bd duplicate, bd supersede, bd duplicates --auto-merge (its
  related link and its reparenting) and the tracker engine's
  createDependencies, and says the molecule, template and swarm writers
  are unaudited. bd-25s1wh's notes carry the widened scope, including
  `bd duplicate bd-a.1 --of bd-a`.

Declined (nit): unifying the root issueops import alias across the PR's
cmd/bd files. batch.go is the one file that imports both issueops
packages, so one of them needs an alias there; it uses publicops, the
repo's usual alias for the root package, and aliasing the other side
instead would rewrite base-owned lines.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:56:46 +00:00

527 lines
16 KiB
Go

package main
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"os"
"strconv"
"strings"
"github.com/spf13/cobra"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
publicops "github.com/steveyegge/beads/issueops"
)
// Scope note:
// `bd batch` is intentionally a narrow, write-oriented command runner. It does
// NOT re-use the top-level cobra handlers (close/update/create/dep) because
// those handlers read the global `store` and perform a full connection-
// per-operation workflow. Instead, batch parses a simple line-oriented grammar
// and dispatches directly against a shared storage.Transaction so the entire
// batch executes as a single dolt transaction (one DOLT_COMMIT).
//
// The supported grammar is a documented subset that matches what a downstream
// consumer's shell-script "orders" (gate-sweep.sh, spawn-storm-detect.sh,
// cross-rig-deps.sh) actually call in loops. See the Long help below for the
// exact list. Unsupported commands error out loudly.
var batchCmd = &cobra.Command{
Use: "batch",
GroupID: "maint",
Short: "Run multiple write operations in a single database transaction",
Long: `Run multiple write operations in a single database transaction.
Commands are read from stdin (one per line) or from a file via -f/--file.
All operations execute inside a single dolt transaction: on any error the
whole batch is rolled back, otherwise it is committed with one DOLT_COMMIT.
This is intended for shell scripts that currently invoke 'bd' many times in
a loop, which causes severe write amplification on a dolt sql-server backed
by btrfs+compression. Batching collapses N invocations into one transaction
and one dolt commit.
Grammar (one command per line):
close <id> [reason...]
update <id> <key>=<value> [<key>=<value> ...]
create <type> <priority> <title...>
dep add <from-id> <to-id> [type]
dep remove <from-id> <to-id>
#comment (blank lines and '# ...' comments are ignored)
Supported 'update' keys: status, priority, title, assignee, force
Supported dependency types: see 'bd dep add --help' (default: blocks)
'force' is not a field. An update whose status moves the issue into closed
(or a configured done status) is refused when it still has open children or
a live blocker, the same as 'bd close'; 'force=true' overrides that refusal.
Because the batch is one transaction, an unforced refusal rolls back EVERY
operation in the batch, not just the offending line. Note the asymmetry with
'close <id>', which does not apply that policy at all.
Tokens are whitespace-separated. Double-quoted strings ("like this") may
contain spaces; use \" to embed a quote and \\ for a backslash.
Examples:
# From a pipe
bd list --status stale -q | awk '{print "close",$1," stale"}' | bd batch
# From a file
bd batch -f operations.txt
# Inline
printf 'close bd-1 done\nupdate bd-2 status=in_progress\n' | bd batch
On success, exits 0 and prints a summary (or JSON with --json). On any error,
rolls back the entire transaction and exits non-zero with the failing line.
NOTE: This is a narrow subset. Commands like 'show', 'list', 'ready', 'sync',
complex create flows, or any flag not listed above are NOT accepted. Use
normal 'bd' subcommands for interactive/read operations.`,
Args: cobra.NoArgs,
SilenceUsage: true,
SilenceErrors: false,
RunE: func(cmd *cobra.Command, args []string) error {
CheckReadonly("batch")
evt := metrics.NewCommandEvent("batch")
defer func() {
if c := metrics.Global(); c != nil {
c.CloseEventAndAdd(evt)
}
}()
proxied := usesProxiedServer()
if !proxied && store == nil {
return fmt.Errorf("no database connection available (%s)", diagHint())
}
filePath, _ := cmd.Flags().GetString("file")
dryRun, _ := cmd.Flags().GetBool("dry-run")
commitMsg, _ := cmd.Flags().GetString("message")
var reader io.Reader
if filePath != "" {
f, err := os.Open(filePath) // #nosec G304 -- user-supplied batch file
if err != nil {
return fmt.Errorf("open batch file: %w", err)
}
defer f.Close()
reader = f
} else {
reader = cmd.InOrStdin()
}
ops, err := parseBatchScript(reader)
if err != nil {
return fmt.Errorf("parsing batch input: %w", err)
}
if dryRun {
// In dry-run mode, just echo what would run. This is helpful for
// shell script authors verifying their scripts before running.
for _, op := range ops {
fmt.Fprintf(cmd.OutOrStdout(), "line %d: %s\n", op.line, op.raw)
}
if jsonOutput {
if err := outputJSON(map[string]interface{}{
"dry_run": true,
"operations": len(ops),
}); err != nil {
return err
}
} else {
fmt.Fprintf(cmd.OutOrStdout(), "%d operations parsed (dry-run, nothing executed)\n", len(ops))
}
return nil
}
if len(ops) == 0 {
// Empty input is a no-op success, matching 'bd list | bd batch' on
// an empty list.
if jsonOutput {
if err := outputJSON(map[string]interface{}{
"operations": 0,
"status": "ok",
}); err != nil {
return err
}
} else {
fmt.Fprintln(cmd.OutOrStdout(), "batch: 0 operations (no-op)")
}
return nil
}
if strings.TrimSpace(commitMsg) == "" {
commitMsg = fmt.Sprintf("bd: batch %d ops by %s", len(ops), getActor())
}
ctx := rootCtx
if ctx == nil {
ctx = context.Background()
}
// One transaction, one commit message, whole-batch rollback on the first
// failing line — the contract is the same on both backends; only the
// transaction primitive differs (uow.RunTx there, transact here), and
// both wrap a per-op dispatch that shares this file's parser.
var results []batchOpResult
if proxied {
results, err = runBatchProxiedServer(ctx, ops, commitMsg)
} else {
results = make([]batchOpResult, 0, len(ops))
err = transact(ctx, store, commitMsg, func(tx storage.Transaction) error {
for _, op := range ops {
res, rerr := runBatchOp(ctx, tx, op)
if rerr != nil {
return fmt.Errorf("line %d (%s): %w", op.line, op.raw, rerr)
}
results = append(results, res)
}
return nil
})
}
if err != nil {
if jsonOutput {
if jerr := outputJSONError(err, "batch_error"); jerr != nil {
return errors.Join(err, jerr)
}
}
return err
}
commandDidWrite.Store(true)
if jsonOutput {
if err := outputJSON(map[string]interface{}{
"operations": len(results),
"status": "ok",
"results": results,
}); err != nil {
return err
}
} else {
fmt.Fprintf(cmd.OutOrStdout(), "batch: %d operations committed\n", len(results))
for _, r := range results {
fmt.Fprintf(cmd.OutOrStdout(), " line %d: %s %s\n", r.Line, r.Op, r.Target)
}
}
return nil
},
}
func init() {
batchCmd.Flags().StringP("file", "f", "", "Read commands from file instead of stdin")
batchCmd.Flags().Bool("dry-run", false, "Parse input and echo commands without executing")
batchCmd.Flags().StringP("message", "m", "", "DOLT_COMMIT message (default: 'bd: batch N ops by <actor>')")
rootCmd.AddCommand(batchCmd)
}
// batchOp is one parsed command line.
type batchOp struct {
line int // 1-based source line number
raw string // original source line (for error messages)
cmd string // canonical command name, e.g. "close", "update", "dep.add"
args []string // remaining tokens
}
// batchOpResult is emitted per executed op for JSON reporting.
type batchOpResult struct {
Line int `json:"line"`
Op string `json:"op"`
Target string `json:"target,omitempty"`
}
// parseBatchScript reads the whole input and tokenizes each non-empty,
// non-comment line. It rejects unknown commands immediately so a bad script
// fails before any writes.
func parseBatchScript(r io.Reader) ([]batchOp, error) {
scanner := bufio.NewScanner(r)
// Allow long lines (descriptions, multi-token updates).
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
var ops []batchOp
lineNo := 0
for scanner.Scan() {
lineNo++
raw := scanner.Text()
trimmed := strings.TrimSpace(raw)
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
continue
}
tokens, err := tokenizeBatchLine(trimmed)
if err != nil {
return nil, fmt.Errorf("line %d: %w", lineNo, err)
}
if len(tokens) == 0 {
continue
}
op := batchOp{line: lineNo, raw: trimmed}
switch tokens[0] {
case "close":
op.cmd = "close"
op.args = tokens[1:]
case "update":
op.cmd = "update"
op.args = tokens[1:]
case "create":
op.cmd = "create"
op.args = tokens[1:]
case "dep":
if len(tokens) < 2 {
return nil, fmt.Errorf("line %d: 'dep' requires a subcommand (add|remove)", lineNo)
}
switch tokens[1] {
case "add":
op.cmd = "dep.add"
case "remove", "rm":
op.cmd = "dep.remove"
default:
return nil, fmt.Errorf("line %d: unknown dep subcommand %q (want add|remove)", lineNo, tokens[1])
}
op.args = tokens[2:]
default:
return nil, fmt.Errorf("line %d: unsupported batch command %q (supported: close, update, create, dep add, dep remove)", lineNo, tokens[0])
}
ops = append(ops, op)
}
if err := scanner.Err(); err != nil {
return nil, err
}
return ops, nil
}
// tokenizeBatchLine splits a line into whitespace-separated tokens with
// support for double-quoted strings. Escape sequences inside quotes: \" and
// \\. Anything else after a backslash inside quotes is treated literally.
func tokenizeBatchLine(s string) ([]string, error) {
var tokens []string
var cur strings.Builder
inQuote := false
hasToken := false
for i := 0; i < len(s); i++ {
c := s[i]
if inQuote {
if c == '\\' && i+1 < len(s) {
next := s[i+1]
if next == '"' || next == '\\' {
cur.WriteByte(next)
i++
continue
}
cur.WriteByte(c)
continue
}
if c == '"' {
inQuote = false
continue
}
cur.WriteByte(c)
continue
}
if c == '"' {
inQuote = true
hasToken = true
continue
}
if c == ' ' || c == '\t' {
if hasToken {
tokens = append(tokens, cur.String())
cur.Reset()
hasToken = false
}
continue
}
hasToken = true
cur.WriteByte(c)
}
if inQuote {
return nil, fmt.Errorf("unterminated quoted string")
}
if hasToken {
tokens = append(tokens, cur.String())
}
return tokens, nil
}
// runBatchOp dispatches a single parsed op against the shared transaction.
// It intentionally does NOT call any of the non-tx cobra handlers; it talks
// straight to storage.Transaction so everything joins the same SQL tx.
func runBatchOp(ctx context.Context, tx storage.Transaction, op batchOp) (batchOpResult, error) {
actorName := getActor()
result := batchOpResult{Line: op.line, Op: op.cmd}
switch op.cmd {
case "close":
if len(op.args) < 1 {
return result, fmt.Errorf("close requires <id>")
}
id := op.args[0]
reason := "Closed"
if len(op.args) > 1 {
reason = strings.Join(op.args[1:], " ")
}
if err := tx.CloseIssue(ctx, id, reason, actorName, ""); err != nil {
return result, err
}
result.Target = id
return result, nil
case "update":
if len(op.args) < 2 {
return result, fmt.Errorf("update requires <id> and at least one key=value")
}
id := op.args[0]
updates, err := parseUpdateKVs(op.args[1:])
if err != nil {
return result, err
}
if err := tx.UpdateIssue(ctx, id, updates, actorName); err != nil {
return result, err
}
result.Target = id
return result, nil
case "create":
if len(op.args) < 3 {
return result, fmt.Errorf("create requires <type> <priority> <title>")
}
issueType := types.IssueType(op.args[0])
// Accept common custom types too; fall back to type validation by the
// storage layer which knows about configured custom types. We only
// reject obviously-empty input here.
if strings.TrimSpace(op.args[0]) == "" {
return result, fmt.Errorf("create: type cannot be empty")
}
priority, err := strconv.Atoi(op.args[1])
if err != nil {
return result, fmt.Errorf("create: invalid priority %q: %w", op.args[1], err)
}
title := strings.Join(op.args[2:], " ")
if strings.TrimSpace(title) == "" {
return result, fmt.Errorf("create: title cannot be empty")
}
issue := &types.Issue{
Title: title,
IssueType: issueType,
Status: types.StatusOpen,
Priority: priority,
}
if err := tx.CreateIssue(ctx, issue, actorName); err != nil {
return result, err
}
result.Target = issue.ID
return result, nil
case "dep.add":
dep, err := batchDepAddEdge(op.args)
if err != nil {
return result, err
}
if err := tx.AddDependency(ctx, dep, actorName); err != nil {
return result, err
}
result.Target = fmt.Sprintf("%s->%s", dep.IssueID, dep.DependsOnID)
return result, nil
case "dep.remove":
if len(op.args) < 2 {
return result, fmt.Errorf("dep remove requires <from-id> <to-id>")
}
from, to := op.args[0], op.args[1]
if err := tx.RemoveDependency(ctx, from, to, actorName); err != nil {
return result, err
}
result.Target = fmt.Sprintf("%s->%s", from, to)
return result, nil
}
return result, fmt.Errorf("internal: unhandled batch op %q", op.cmd)
}
// batchDepAddEdge builds the edge a `dep add <from-id> <to-id> [type]` line
// asks for. Both backends' dispatch switches call it, so the default type, the
// arity and type errors, and the dotted-id hierarchy rule cannot drift between
// them.
// Neither backend writes the edge through the DependencyEditor role that
// enforces that rule (storage.Transaction.AddDependency and the unit of work's
// AddDependencies verb do not), so batch asks the role's rule directly, the
// way bd link does: a script line `dep add bd-abc.1 bd-abc` is refused, and
// rolls the batch back, exactly as `bd dep add bd-abc.1 bd-abc` is.
func batchDepAddEdge(args []string) (*types.Dependency, error) {
if len(args) < 2 {
return nil, fmt.Errorf("dep add requires <from-id> <to-id>")
}
from, to := args[0], args[1]
depType := "blocks"
if len(args) >= 3 {
depType = args[2]
}
dt := types.DependencyType(depType)
if !dt.IsValid() {
return nil, fmt.Errorf("dep add: invalid dependency type %q", depType)
}
if err := publicops.CheckDottedChildDependency(from, to, dt); err != nil {
return nil, err
}
return &types.Dependency{
IssueID: from,
DependsOnID: to,
Type: dt,
}, nil
}
// parseUpdateKVs walks a slice of "key=value" tokens and builds the updates
// map accepted by storage.Transaction.UpdateIssue. Only a small, documented
// subset of fields is allowed — anything else is a hard error so typos in
// scripts never silently drop updates.
func parseUpdateKVs(kvs []string) (map[string]interface{}, error) {
updates := make(map[string]interface{}, len(kvs))
for _, kv := range kvs {
eq := strings.IndexByte(kv, '=')
if eq <= 0 {
return nil, fmt.Errorf("update: expected key=value, got %q", kv)
}
key := strings.TrimSpace(kv[:eq])
value := kv[eq+1:]
switch key {
case "status":
if !types.Status(value).IsValid() {
// IsValid excludes custom statuses; the transaction layer will
// re-validate. Still reject blatantly empty values here.
if strings.TrimSpace(value) == "" {
return nil, fmt.Errorf("update: status cannot be empty")
}
}
updates["status"] = value
case "priority":
p, err := strconv.Atoi(value)
if err != nil {
return nil, fmt.Errorf("update: invalid priority %q: %w", value, err)
}
updates["priority"] = p
case "title":
if strings.TrimSpace(value) == "" {
return nil, fmt.Errorf("update: title cannot be empty")
}
updates["title"] = value
case "assignee":
updates["assignee"] = value
case "force":
// Not a field: the override for close policy on a status that
// crosses into done, spelled as a token because a batch script has
// no flags. The write funnel pops it before validating fields.
force, err := strconv.ParseBool(value)
if err != nil {
return nil, fmt.Errorf("update: invalid force %q: %w", value, err)
}
updates[issueops.OpForceClosePolicy] = force
default:
return nil, fmt.Errorf("update: unsupported key %q (allowed: status, priority, title, assignee, force)", key)
}
}
return updates, nil
}