beads/docs/reference/json-schema.md
Steve Yegge 80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00

7.1 KiB
Raw Blame History

title description
JSON Output Schema Contract The stable JSON output contract for bd --json commands, covering the schema_version envelope, per-command fields, and consumer guidelines.

Last reviewed: 2026-05-08

Freshness source: cmd/bd/output.go, cmd/bd/errors.go, and cmd/bd/protocol/json_contract_test.go.

All bd commands that support --json output can wrap their response in a uniform envelope by setting BD_JSON_ENVELOPE=1. This will become the default format in v2.0.

Migration Guide

Opt in to the envelope format

export BD_JSON_ENVELOPE=1

Envelope format (BD_JSON_ENVELOPE=1, default in v2.0)

Every --json command wraps output as:

{"schema_version": 1, "data": <original-payload>}

The original payload is untouched inside .data — no type corruption, no field injection. Works identically for objects, arrays, and maps.

Updating consumers

# Before (legacy):
bd list --json | jq '.[0].id'
bd show beads-abc --json | jq '.title'

# After (envelope):
bd list --json | jq '.data[0].id'
bd show beads-abc --json | jq '.data.title'

# Version check:
bd show beads-abc --json | jq '.schema_version'

Timeline

  • Current release: Legacy format is default. Set BD_JSON_ENVELOPE=1 to opt in. A deprecation notice is printed to stderr when --json is used without the env var.
  • v2.0: Envelope becomes the default. BD_JSON_ENVELOPE=0 available as temporary escape hatch for one release cycle.

Schema Version

Current version: 1

The schema_version field is an integer that increments when:

  • Fields are added, renamed, or removed
  • Output structure changes (e.g., nesting depth)
  • Field types change (e.g., string to integer)

Additive changes (new optional fields) do NOT bump the version.

Output Formats

Envelope mode (BD_JSON_ENVELOPE=1)

All commands emit a uniform envelope:

{
  "schema_version": 1,
  "data": {
    "id": "beads-abc",
    "title": "Example issue",
    "status": "open"
  }
}

Arrays are wrapped the same way:

{
  "schema_version": 1,
  "data": [
    {"id": "beads-abc", "title": "First"},
    {"id": "beads-def", "title": "Second"}
  ]
}

Legacy mode (default, until v2.0)

Object commands (show, create, close, update, etc.)

Commands that return a single issue or result emit a JSON object with schema_version as a top-level field alongside the data:

{
  "schema_version": 1,
  "id": "beads-abc",
  "title": "Example issue",
  "status": "open",
  "priority": 1,
  "issue_type": "task",
  "created_at": "2026-04-20T12:00:00Z"
}

List commands (list, ready, search, stale, etc.)

Commands that return multiple items emit a raw JSON array:

[
  {"id": "beads-abc", "title": "First", ...},
  {"id": "beads-def", "title": "Second", ...}
]

Error output (stderr)

Errors with --json active emit JSON to stderr:

{
  "schema_version": 1,
  "error": "issue not found: beads-xyz",
  "code": "not_found"
}

Field Contracts by Command

bd list --json

Required fields per item:

  • id (string): Issue ID (e.g., "beads-abc")
  • title (string): Issue title
  • status (string): open, in_progress, closed, deferred
  • priority (number): 0-4
  • issue_type (string): bug, feature, task, epic, chore
  • created_at (string): RFC3339 timestamp

Optional fields:

  • description, owner, updated_at, closed_at
  • labels (string[]): Attached labels
  • dependencies (object[]): Dependency records
  • dependency_count, dependent_count, comment_count (number)
  • parent (string|null): Parent issue ID

bd ready --json

Same schema as bd list --json. Items are filtered to unblocked issues only. Each item includes dependency_count, dependent_count, comment_count, and optional parent fields.

bd blocked --json

Returns issues that are blocked by unresolved dependencies. Each item includes all standard issue fields plus:

  • blocked_by_count (number): Number of blocking dependencies
  • blocked_by (string[]): IDs of blocking issues

bd show --json

Returns a single object (not wrapped in items). Same required fields as list items, plus:

  • description (string)
  • acceptance_criteria (string)
  • dependencies (object[]): Full dependency records
  • comments (object[]): Comment thread — present only with --include-comments; the default response returns comment_count only (count-only, be-ijck6q)
  • comments_omitted (boolean, optional): true only when comment_count is nonzero and comments was left out of the response (no --include-comments). Absent when comments were included or when there are none to omit (ga-clgh)

import --json

Returns a summary object when --json is active:

  • source (string): File path or "stdin"
  • created (number): Issues created
  • updated (number): Existing issues updated
  • skipped (number): Issues skipped (stale rows + dedup)
  • dedup_skipped (number): Issues skipped by --dedup title match
  • memories (number): Memory records imported
  • ids (string[]): IDs of created issues
  • updated_issues (object[]): Per-issue summary of what an update changed
  • tie_kept_local_ids (string[]): Equal-updated_at rows where local state won
  • stale_skipped_ids (string[]): Rows older than the local issue, skipped
  • skipped_dependencies (string[]): Dependency edges whose target id was absent
  • dry_run (boolean): Whether --dry-run was active

bd export --json

Outputs JSONL (one JSON object per line), not wrapped in an envelope. Each line is a self-contained issue or memory record, discriminated by _type ("issue" / "memory"). Export lines do not carry schema_version — that field belongs to the --json command envelope, not to the interchange stream. The interchange's own version marker is the optional _schema header record ({"_schema":"beads-jsonl/1"}), which readers skip.

Issue records carry an optional wisp_plane boolean: the explicit wisps-plane marker. Export stamps it on rows that live in the wisps table when the row flags alone cannot prove the plane (a no_history: true record is otherwise ambiguous — an unpromoted no-history wisp and a promoted one look identical). Import routes the storage plane by this marker, never by no_history: marker absent means the durable issues table. The marker is a fresh key rather than a reuse of the legacy wisp boolean so that older readers, which do not know it, degrade to flag routing instead of importing marked rows as ephemeral (purge-eligible and export-excluded). The v0.35– v0.37 wisp key — those streams' spelling of ephemeral — is still honored as a read-side legacy alias.

Consumer Guidelines

  1. Check schema_version on object output. If the version is higher than expected, log a warning but attempt to parse anyway (additive changes are backward-compatible).

  2. For list commands, parse the output as a JSON array directly.

  3. Ignore unknown fields. New fields may be added without bumping the schema version.

  4. Use --json flag, not --format json. The --json flag is the stable contract; --format is for human-readable variants.