beads/docs/reference/json-schema.md
Julian Knutsen b858a29e4a
fix(contracts): JSON contract fixes before 1.3.0 freezes them — string revision tokens, --set-metadata scalar typing, create --json nano timestamps (#6053)
* fix(metadata): restore v1.2.2 scalar typing for --set-metadata values

8ebd53e88 moved the CLI's metadata-edit helpers into
internal/storage/metadata.go and, in the move, replaced v1.2.2's
toJSONValue inference with a MetadataEditValue that always json.Marshals
the value as a string. The intent was GH#4146 (a numeric-looking id or a
version like "1e3" came back as a JSON number, breaking map[string]string
Go consumers), but it silently changed the shipped CLI contract:
`bd update --set-metadata k=5` started storing "5" instead of 5. That
regression shipped only in the deprecated/superseded v1.2.1, never in
v1.2.2, and 1.3.0 would freeze it.

Restore v1.2.2's inference body-for-body inside MetadataEditValue, so the
parity claim stays checkable against `git show v1.2.2:cmd/bd/update.go`:
"null" is JSON null, "true"/"false" are booleans, a spelling that scans
as a float AND is valid JSON on its own is emitted as that literal
(5 -> 5, 1e3 -> 1e3, 0.50 -> 0.50, while 05, +5 and NaN stay strings),
and everything else is a string. Every --set-metadata caller funnels
through this one function: embedded (cmd/bd/update.go ->
storage.ApplyMetadataEdits), proxied (parseSetMetadataFlags) and the
update executor's []string path.

GH#4146's concern is served by the explicitly typed routes, which are
untouched: --metadata / --metadata-json on the CLI (MergeMetadataJSON)
and the typed set map on the HTTP/Go path (applyTypedMetadataEdits).

No data migration: a "5" written by an in-window build is
indistinguishable from a string a user set intentionally through the
typed paths, so normalizing stored metadata would corrupt intended data.
Re-running the same --set-metadata k=5 retypes any given key.

Tests: new table in internal/storage/metadata_edit_value_test.go pinning
the v1.2.2 matrix plus an ApplyMetadataEdits end-to-end; the tests that
pinned the string-always behavior are flipped (domain/db mergeops suite,
update_lifecycle patch mapping, cmd/bd toJSONValue table) and the
embedded + proxied CLI suites now assert a stored JSON number. The
contract corpus is regenerated: its update/list/ready goldens run
`--set-metadata phase=2`, so `"phase": "2"` becomes `"phase": 2` and
nothing else moves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(create): echo full-precision timestamps from bd create, not the DATETIME(0) re-read

b92442d1a routed `bd create` through issueops.Lifecycle, whose executors
build their result by re-reading the row they just wrote
(HydrateIssueOperationResult -> GetIssueInTx). created_at/updated_at are
DATETIME(0) columns, so that snapshot comes back at whole-second
precision and cmd/bd/create.go echoes it. v1.2.2 returned the caller's
own in-memory struct, which PrepareIssueForInsert stamps with
time.Now().UTC() including nanoseconds, so `bd create --json` used to
print e.g. 2026-08-28T12:34:56.789012345Z. Agents that create several
issues in a burst order them by exactly that fraction; rounded to the
second they all tie.

Restore the echo with OverlayCreateTimestamps: after hydration, put back
the created_at/updated_at (and closed_at, when the write set one) the
INSERT actually sent. Everything else on the snapshot stays DB-truth,
which is the facade's contract — only create, and only these fields.

Applied at every create body, not just the two the CLI reaches:
issueops.ExecuteCreate and ExecuteCreateBatch behind the dolt and
embedded-dolt stores, and the unit-of-work provider's own Create,
CreateBatch and batch-apply create item, which is what the proxied server
and POST /v0/beads/issues run on. Fixing only the shared executors would
have left the two bodies disagreeing about the same promise.

What is NOT changed, because it is not a regression: `bd show --json` and
`bd list --json` answered whole seconds in v1.2.2 too — they read the
column — and update/close/reopen re-read after writing there as well.
Widening the columns to DATETIME(6) would give reads sub-second times
too, but that is a schema migration on user Dolt DBs and touches paths
that deliberately compare at second precision; it is a 1.4 change.

The accepted skew, which predates this and stays: the engine ROUNDS to
the column rather than truncating, so a later read of the same row can
sit up to half a second either side of the create echo.

Tests: two new conformance cases, run on all three backends —
RunLifecycleCreateEchoesSubSecondTimestamps (a caller-supplied
nanosecond stamp must survive to the result exactly; the auto-stamped arm
stays a bounded smoke check, since requiring a non-zero fraction off the
clock is flaky by a billionth) and RunBatchCreatorEchoesSubSecondTimestamps
(per item, since both bodies build the result one snapshot at a time).
Plus a cmd/bd embedded test asserting `bd create --json` prints a
fractional part, and one httpapi assertion that the wire does not round
the role's snapshot on the way out. All four fail on the pre-fix tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(httpapi,types)!: revision / expected_version tokens are opaque JSON strings

The optimistic-concurrency token is minted by freshRowLock over the FULL
int64 range (crypto-random, non-zero) and published on the wire as a JSON
number: `revision` on responses, `expected_version` on guarded requests.
A JSON number does not survive a consumer that decodes into an IEEE-754
double — JavaScript's JSON.parse, Go's `any`, Python's `float`, jq. Live
tokens run past 5e17, where a double's ulp is already 64, so such a
client reads back a value NEAR the token, echoes that as its guard, and
earns a 409 against a row nobody else touched. The spec already carried
"DECODE IT AS A 64-BIT INTEGER" paragraphs telling clients to work around
this; that is a workaround for a wire type that should not have been a
number.

Emit and accept the token as a decimal STRING instead. It is documented
as opaque and EQUALITY-ONLY — compare it, never order or interpret it —
and a string round-trips exactly in every JSON consumer. It also leaves a
future backend free to mint a token that is not an int64 at all.

Internals are untouched: freshRowLock, the row_lock BIGINT column, its
migration, types.Issue.RowVersion and the ExpectedVersion *int64 guards
on the issueops requests all stay int64. Only the wire projection moves,
through the one spelling of the encoding — types.RevisionToken and
types.ParseRevisionToken.

STRICT STRING ONLY, with no transitional number-or-string acceptance.
This API already refuses unknown members by name and keeps one spelling
of the guard, and a number is precisely the shape a double-based client
has already rounded before it left the client — accepting it would
preserve the defect. A numeric guard is a 400 naming the member.

Chosen over clamping freshRowLock to 53 bits: a clamp does not fix rows
already minted with full-range tokens by in-window builds, so a JS
consumer reading one still rounds past 2^53 — repairing that would need a
row_lock normalization sweep over issues and wisps, a migration days
before a tag.

Surface: types.IssueDetails.Revision (what `bd show --json` prints and
GET /v0/beads/issues/{id} returns), the five hand-assigned httpapi
response fields, applyVersionGuardMember (now the reader for the batch
update item too, ahead of its typed decode, so a mistyped token still
earns a 400 naming the member), Problem.expected_version, and 14 OpenAPI
members regenerated through `make api-gen`. actual_version goes with them
— it is documented as "the `revision` the row was found holding", so
leaving it an integer would make the expected/actual pair disagree about
its own type; it is declared but never populated, so nothing changes
behaviorally.

Nothing shipped in a tagged release ever carried this member: v1.2.2 has
no internal/httpapi at all and no `revision` anywhere on the wire, so the
whole surface is in-window and can still be changed compatibly. Legacy
migration-0054 rows read as the string "0" with the same CAS semantics.

Corpus goldens are UNCHANGED: the canonicalizer already replaced any
`revision` value with the string "<REVISION>" whatever its JSON type.

Tests: types-level pins that the marshaled member matches `"revision":
"-?\d+"` and that RevisionToken/ParseRevisionToken round-trip MinInt64,
MaxInt64 and 2^53+1, plus a JS-realistic decode of 2^53+1 with a float64
control proving the token really is one a number would corrupt; httpapi
refusal tests for a numeric guard on update, close, reopen, delete and
both batch item kinds; a 409 echo asserted as the string; and the proxied
serve integration tests now splice the response's own token back in
verbatim rather than re-rendering a parsed one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 05:20:27 +00:00

261 lines
8.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
title: JSON Output Schema Contract
description: The stable JSON output contract for bd --json commands, covering the schema_version envelope, per-command fields, and consumer guidelines.
---
Last reviewed: 2026-08-07
Freshness source: `cmd/bd/output.go`, `cmd/bd/errors.go`, and
`cmd/bd/protocol/json_contract_test.go`.
All `bd` commands that support `--json` output can wrap their response in
a uniform envelope by setting `BD_JSON_ENVELOPE=1`. This will become the
default format in v2.0.
## Migration Guide
### Opt in to the envelope format
```bash
export BD_JSON_ENVELOPE=1
```
### Envelope format (BD_JSON_ENVELOPE=1, default in v2.0)
Every `--json` command wraps output as:
```json
{"schema_version": 1, "data": <original-payload>}
```
The original payload is untouched inside `.data` — no type corruption,
no field injection. Works identically for objects, arrays, and maps.
When a `--limit`-truncated listing runs in envelope mode (currently wired
for `bd ready`), the envelope also carries a `pagination` key:
```json
{"schema_version": 1, "data": [...], "pagination": {"returned": 10, "total": 42, "truncated": true}}
```
`total` is omitted when unknown; the whole `pagination` key is absent when
the result was not truncated. Legacy mode keeps the stderr text hint instead.
### Updating consumers
```bash
# Before (legacy):
bd list --json | jq '.[0].id'
bd show beads-abc --json | jq '.[0].title'
# After (envelope):
bd list --json | jq '.data[0].id'
bd show beads-abc --json | jq '.data[0].title'
# Version check (object commands, e.g. create):
bd create "Example" --json | jq '.schema_version'
```
### Timeline
- **Current release**: Legacy format is default. Set `BD_JSON_ENVELOPE=1` to opt in.
A deprecation notice is printed to stderr when `--json` is used without the env
var — but only when stderr is a terminal, and at most once per invocation, so
scripts capturing stderr will not see it.
- **v2.0**: Envelope becomes the default. `BD_JSON_ENVELOPE=0` available as
temporary escape hatch for one release cycle.
## Schema Version
Current version: **1**
The `schema_version` field is an integer that increments when:
- Fields are added, renamed, or removed
- Output structure changes (e.g., nesting depth)
- Field types change (e.g., string to integer)
Additive changes (new optional fields) do NOT bump the version.
## Output Formats
### Envelope mode (BD_JSON_ENVELOPE=1)
All commands emit a uniform envelope:
```json
{
"schema_version": 1,
"data": {
"id": "beads-abc",
"title": "Example issue",
"status": "open"
}
}
```
Arrays are wrapped the same way:
```json
{
"schema_version": 1,
"data": [
{"id": "beads-abc", "title": "First"},
{"id": "beads-def", "title": "Second"}
]
}
```
### Legacy mode (default, until v2.0)
### Object commands (create, ping, etc.)
Commands that return a single result emit a JSON object with
`schema_version` as a top-level field alongside the data:
```json
{
"schema_version": 1,
"id": "beads-abc",
"title": "Example issue",
"status": "open",
"priority": 1,
"issue_type": "task",
"created_at": "2026-04-20T12:00:00Z"
}
```
### List commands (list, ready, search, stale, show, close, update, etc.)
Commands that return one or more issues emit a raw JSON array — including
`show`, `close`, and `update`, which return one element per requested ID.
Array output carries no top-level `schema_version` field:
```json
[
{"id": "beads-abc", "title": "First", ...},
{"id": "beads-def", "title": "Second", ...}
]
```
### Error output
Errors with `--json` active emit JSON — most error paths write it to stderr,
though some command-result error paths emit the same shape to stdout:
```json
{
"schema_version": 1,
"error": "issue not found: beads-xyz",
"code": "not_found"
}
```
`code` and `hint` (a remediation suggestion) are both optional — only
`error` and `schema_version` are always present. In envelope mode
(`BD_JSON_ENVELOPE=1`) the error payload moves inside the envelope:
`{"schema_version": 1, "data": {"error": ..., "code": ..., "hint": ...}}`.
JSON-mode errors exit with code 1.
## Field Contracts by Command
### bd list --json
Required fields per item:
- `id` (string): Issue ID (e.g., "beads-abc")
- `title` (string): Issue title
- `status` (string): open, in_progress, closed, deferred
- `priority` (number): 0-4
- `issue_type` (string): bug, feature, task, epic, chore
- `created_at` (string): RFC3339 timestamp
Optional fields:
- `description`, `owner`, `updated_at`, `closed_at`
- `labels` (string[]): Attached labels
- `dependencies` (object[]): Dependency records
- `dependency_count`, `dependent_count`, `comment_count` (number)
- `parent` (string|null): Parent issue ID
### bd ready --json
Same schema as `bd list --json`. Items are filtered to unblocked issues only.
Each item includes `dependency_count`, `dependent_count`, `comment_count`,
and optional `parent` fields. In envelope mode a `--limit`-truncated result
adds the envelope-level `pagination` key (see the envelope section above).
### bd blocked --json
Returns issues that are blocked by unresolved dependencies.
Each item includes all standard issue fields plus:
- `blocked_by_count` (number): Number of blocking dependencies
- `blocked_by` (string[]): IDs of blocking issues
### bd show --json
Returns a top-level JSON array with one element per requested ID; items do
not carry `schema_version` (this shape is pinned by a contract test — a
change here is a breaking wire change). Same required fields as list
items, plus:
- `description` (string)
- `acceptance_criteria` (string)
- `revision` (string): guarded-write optimistic-concurrency token, an opaque
decimal string compared by equality and echoed back verbatim; always present,
including a legacy `"0"`
- `dependencies` (object[]): Full dependency records
- `comments` (object[]): Comment thread — present only with `--include-comments`;
the default response returns `comment_count` only (count-only, be-ijck6q)
- `comments_omitted` (boolean, optional): `true` only when `comment_count` is
nonzero and `comments` was left out of the response (no `--include-comments`).
Absent when comments were included or when there are none to omit (ga-clgh)
### `import --json`
Returns a summary object when `--json` is active:
- `source` (string): File path or "stdin"
- `created` (number): Issues created
- `updated` (number): Existing issues updated
- `unchanged` (number, optional): Rows identical to local state, untouched
- `skipped` (number): Issues skipped (stale rows + dedup)
- `dedup_skipped` (number): Issues skipped by `--dedup` title match
- `memories` (number): Memory records imported
- `ids` (string[]): IDs of created issues
- `updated_issues` (object[]): Per-issue summary of what an update changed
- `tie_kept_local_ids` (string[]): Equal-`updated_at` rows where local state won
- `stale_skipped_ids` (string[]): Rows older than the local issue, skipped
- `skipped_dependencies` (string[]): Dependency edges whose target id was absent
- `dry_run` (boolean): Whether `--dry-run` was active
### bd export --json
Outputs JSONL (one JSON object per line), not wrapped in an envelope.
Each line is a self-contained issue or memory record, discriminated by
`_type` (`"issue"` / `"memory"`). Export lines do **not** carry
`schema_version` — that field belongs to the `--json` command envelope,
not to the interchange stream. The interchange's own version marker is the
optional `_schema` header record (`{"_schema":"beads-jsonl/1"}`), which
readers skip.
Issue records carry an optional `wisp_plane` boolean: the explicit
wisps-plane marker. Export stamps it on rows that live in the wisps table
when the row flags alone cannot prove the plane (a `no_history: true` record
is otherwise ambiguous — an unpromoted no-history wisp and a promoted one
look identical). Import routes the storage plane by this marker, never by
`no_history`: marker absent means the durable issues table. The marker is a
fresh key rather than a reuse of the legacy `wisp` boolean so that older
readers, which do not know it, degrade to flag routing instead of importing
marked rows as ephemeral (purge-eligible and export-excluded). The v0.35–
v0.37 `wisp` key — those streams' spelling of `ephemeral` — is still honored
as a read-side legacy alias.
## Consumer Guidelines
1. **Check `schema_version`** on object output. If the version is
higher than expected, log a warning but attempt to parse anyway
(additive changes are backward-compatible).
2. **For list commands**, parse the output as a JSON array directly.
3. **Ignore unknown fields**. New fields may be added without bumping
the schema version.
4. **Use `--json` flag**, not `--format json`. The `--json` flag is
the stable contract; `--format` is for human-readable variants.