beads/docs/reference/json-schema.md
Steve Yegge 80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00

233 lines
7.1 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
title: JSON Output Schema Contract
description: The stable JSON output contract for bd --json commands, covering the schema_version envelope, per-command fields, and consumer guidelines.
---
Last reviewed: 2026-05-08
Freshness source: `cmd/bd/output.go`, `cmd/bd/errors.go`, and
`cmd/bd/protocol/json_contract_test.go`.
All `bd` commands that support `--json` output can wrap their response in
a uniform envelope by setting `BD_JSON_ENVELOPE=1`. This will become the
default format in v2.0.
## Migration Guide
### Opt in to the envelope format
```bash
export BD_JSON_ENVELOPE=1
```
### Envelope format (BD_JSON_ENVELOPE=1, default in v2.0)
Every `--json` command wraps output as:
```json
{"schema_version": 1, "data": <original-payload>}
```
The original payload is untouched inside `.data` — no type corruption,
no field injection. Works identically for objects, arrays, and maps.
### Updating consumers
```bash
# Before (legacy):
bd list --json | jq '.[0].id'
bd show beads-abc --json | jq '.title'
# After (envelope):
bd list --json | jq '.data[0].id'
bd show beads-abc --json | jq '.data.title'
# Version check:
bd show beads-abc --json | jq '.schema_version'
```
### Timeline
- **Current release**: Legacy format is default. Set `BD_JSON_ENVELOPE=1` to opt in.
A deprecation notice is printed to stderr when `--json` is used without the env var.
- **v2.0**: Envelope becomes the default. `BD_JSON_ENVELOPE=0` available as
temporary escape hatch for one release cycle.
## Schema Version
Current version: **1**
The `schema_version` field is an integer that increments when:
- Fields are added, renamed, or removed
- Output structure changes (e.g., nesting depth)
- Field types change (e.g., string to integer)
Additive changes (new optional fields) do NOT bump the version.
## Output Formats
### Envelope mode (BD_JSON_ENVELOPE=1)
All commands emit a uniform envelope:
```json
{
"schema_version": 1,
"data": {
"id": "beads-abc",
"title": "Example issue",
"status": "open"
}
}
```
Arrays are wrapped the same way:
```json
{
"schema_version": 1,
"data": [
{"id": "beads-abc", "title": "First"},
{"id": "beads-def", "title": "Second"}
]
}
```
### Legacy mode (default, until v2.0)
### Object commands (show, create, close, update, etc.)
Commands that return a single issue or result emit a JSON object with
`schema_version` as a top-level field alongside the data:
```json
{
"schema_version": 1,
"id": "beads-abc",
"title": "Example issue",
"status": "open",
"priority": 1,
"issue_type": "task",
"created_at": "2026-04-20T12:00:00Z"
}
```
### List commands (list, ready, search, stale, etc.)
Commands that return multiple items emit a raw JSON array:
```json
[
{"id": "beads-abc", "title": "First", ...},
{"id": "beads-def", "title": "Second", ...}
]
```
### Error output (stderr)
Errors with `--json` active emit JSON to stderr:
```json
{
"schema_version": 1,
"error": "issue not found: beads-xyz",
"code": "not_found"
}
```
## Field Contracts by Command
### bd list --json
Required fields per item:
- `id` (string): Issue ID (e.g., "beads-abc")
- `title` (string): Issue title
- `status` (string): open, in_progress, closed, deferred
- `priority` (number): 0-4
- `issue_type` (string): bug, feature, task, epic, chore
- `created_at` (string): RFC3339 timestamp
Optional fields:
- `description`, `owner`, `updated_at`, `closed_at`
- `labels` (string[]): Attached labels
- `dependencies` (object[]): Dependency records
- `dependency_count`, `dependent_count`, `comment_count` (number)
- `parent` (string|null): Parent issue ID
### bd ready --json
Same schema as `bd list --json`. Items are filtered to unblocked issues only.
Each item includes `dependency_count`, `dependent_count`, `comment_count`,
and optional `parent` fields.
### bd blocked --json
Returns issues that are blocked by unresolved dependencies.
Each item includes all standard issue fields plus:
- `blocked_by_count` (number): Number of blocking dependencies
- `blocked_by` (string[]): IDs of blocking issues
### bd show --json
Returns a single object (not wrapped in `items`). Same required fields as list
items, plus:
- `description` (string)
- `acceptance_criteria` (string)
- `dependencies` (object[]): Full dependency records
- `comments` (object[]): Comment thread — present only with `--include-comments`;
the default response returns `comment_count` only (count-only, be-ijck6q)
- `comments_omitted` (boolean, optional): `true` only when `comment_count` is
nonzero and `comments` was left out of the response (no `--include-comments`).
Absent when comments were included or when there are none to omit (ga-clgh)
### `import --json`
Returns a summary object when `--json` is active:
- `source` (string): File path or "stdin"
- `created` (number): Issues created
- `updated` (number): Existing issues updated
- `skipped` (number): Issues skipped (stale rows + dedup)
- `dedup_skipped` (number): Issues skipped by `--dedup` title match
- `memories` (number): Memory records imported
- `ids` (string[]): IDs of created issues
- `updated_issues` (object[]): Per-issue summary of what an update changed
- `tie_kept_local_ids` (string[]): Equal-`updated_at` rows where local state won
- `stale_skipped_ids` (string[]): Rows older than the local issue, skipped
- `skipped_dependencies` (string[]): Dependency edges whose target id was absent
- `dry_run` (boolean): Whether `--dry-run` was active
### bd export --json
Outputs JSONL (one JSON object per line), not wrapped in an envelope.
Each line is a self-contained issue or memory record, discriminated by
`_type` (`"issue"` / `"memory"`). Export lines do **not** carry
`schema_version` — that field belongs to the `--json` command envelope,
not to the interchange stream. The interchange's own version marker is the
optional `_schema` header record (`{"_schema":"beads-jsonl/1"}`), which
readers skip.
Issue records carry an optional `wisp_plane` boolean: the explicit
wisps-plane marker. Export stamps it on rows that live in the wisps table
when the row flags alone cannot prove the plane (a `no_history: true` record
is otherwise ambiguous — an unpromoted no-history wisp and a promoted one
look identical). Import routes the storage plane by this marker, never by
`no_history`: marker absent means the durable issues table. The marker is a
fresh key rather than a reuse of the legacy `wisp` boolean so that older
readers, which do not know it, degrade to flag routing instead of importing
marked rows as ephemeral (purge-eligible and export-excluded). The v0.35–
v0.37 `wisp` key — those streams' spelling of `ephemeral` — is still honored
as a read-side legacy alias.
## Consumer Guidelines
1. **Check `schema_version`** on object output. If the version is
higher than expected, log a warning but attempt to parse anyway
(additive changes are backward-compatible).
2. **For list commands**, parse the output as a JSON array directly.
3. **Ignore unknown fields**. New fields may be added without bumping
the schema version.
4. **Use `--json` flag**, not `--format json`. The `--json` flag is
the stable contract; `--format` is for human-readable variants.