Commit graph

7 commits

Author SHA1 Message Date
Julian Knutsen
b858a29e4a
fix(contracts): JSON contract fixes before 1.3.0 freezes them — string revision tokens, --set-metadata scalar typing, create --json nano timestamps (#6053)
* fix(metadata): restore v1.2.2 scalar typing for --set-metadata values

8ebd53e88 moved the CLI's metadata-edit helpers into
internal/storage/metadata.go and, in the move, replaced v1.2.2's
toJSONValue inference with a MetadataEditValue that always json.Marshals
the value as a string. The intent was GH#4146 (a numeric-looking id or a
version like "1e3" came back as a JSON number, breaking map[string]string
Go consumers), but it silently changed the shipped CLI contract:
`bd update --set-metadata k=5` started storing "5" instead of 5. That
regression shipped only in the deprecated/superseded v1.2.1, never in
v1.2.2, and 1.3.0 would freeze it.

Restore v1.2.2's inference body-for-body inside MetadataEditValue, so the
parity claim stays checkable against `git show v1.2.2:cmd/bd/update.go`:
"null" is JSON null, "true"/"false" are booleans, a spelling that scans
as a float AND is valid JSON on its own is emitted as that literal
(5 -> 5, 1e3 -> 1e3, 0.50 -> 0.50, while 05, +5 and NaN stay strings),
and everything else is a string. Every --set-metadata caller funnels
through this one function: embedded (cmd/bd/update.go ->
storage.ApplyMetadataEdits), proxied (parseSetMetadataFlags) and the
update executor's []string path.

GH#4146's concern is served by the explicitly typed routes, which are
untouched: --metadata / --metadata-json on the CLI (MergeMetadataJSON)
and the typed set map on the HTTP/Go path (applyTypedMetadataEdits).

No data migration: a "5" written by an in-window build is
indistinguishable from a string a user set intentionally through the
typed paths, so normalizing stored metadata would corrupt intended data.
Re-running the same --set-metadata k=5 retypes any given key.

Tests: new table in internal/storage/metadata_edit_value_test.go pinning
the v1.2.2 matrix plus an ApplyMetadataEdits end-to-end; the tests that
pinned the string-always behavior are flipped (domain/db mergeops suite,
update_lifecycle patch mapping, cmd/bd toJSONValue table) and the
embedded + proxied CLI suites now assert a stored JSON number. The
contract corpus is regenerated: its update/list/ready goldens run
`--set-metadata phase=2`, so `"phase": "2"` becomes `"phase": 2` and
nothing else moves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(create): echo full-precision timestamps from bd create, not the DATETIME(0) re-read

b92442d1a routed `bd create` through issueops.Lifecycle, whose executors
build their result by re-reading the row they just wrote
(HydrateIssueOperationResult -> GetIssueInTx). created_at/updated_at are
DATETIME(0) columns, so that snapshot comes back at whole-second
precision and cmd/bd/create.go echoes it. v1.2.2 returned the caller's
own in-memory struct, which PrepareIssueForInsert stamps with
time.Now().UTC() including nanoseconds, so `bd create --json` used to
print e.g. 2026-08-28T12:34:56.789012345Z. Agents that create several
issues in a burst order them by exactly that fraction; rounded to the
second they all tie.

Restore the echo with OverlayCreateTimestamps: after hydration, put back
the created_at/updated_at (and closed_at, when the write set one) the
INSERT actually sent. Everything else on the snapshot stays DB-truth,
which is the facade's contract — only create, and only these fields.

Applied at every create body, not just the two the CLI reaches:
issueops.ExecuteCreate and ExecuteCreateBatch behind the dolt and
embedded-dolt stores, and the unit-of-work provider's own Create,
CreateBatch and batch-apply create item, which is what the proxied server
and POST /v0/beads/issues run on. Fixing only the shared executors would
have left the two bodies disagreeing about the same promise.

What is NOT changed, because it is not a regression: `bd show --json` and
`bd list --json` answered whole seconds in v1.2.2 too — they read the
column — and update/close/reopen re-read after writing there as well.
Widening the columns to DATETIME(6) would give reads sub-second times
too, but that is a schema migration on user Dolt DBs and touches paths
that deliberately compare at second precision; it is a 1.4 change.

The accepted skew, which predates this and stays: the engine ROUNDS to
the column rather than truncating, so a later read of the same row can
sit up to half a second either side of the create echo.

Tests: two new conformance cases, run on all three backends —
RunLifecycleCreateEchoesSubSecondTimestamps (a caller-supplied
nanosecond stamp must survive to the result exactly; the auto-stamped arm
stays a bounded smoke check, since requiring a non-zero fraction off the
clock is flaky by a billionth) and RunBatchCreatorEchoesSubSecondTimestamps
(per item, since both bodies build the result one snapshot at a time).
Plus a cmd/bd embedded test asserting `bd create --json` prints a
fractional part, and one httpapi assertion that the wire does not round
the role's snapshot on the way out. All four fail on the pre-fix tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(httpapi,types)!: revision / expected_version tokens are opaque JSON strings

The optimistic-concurrency token is minted by freshRowLock over the FULL
int64 range (crypto-random, non-zero) and published on the wire as a JSON
number: `revision` on responses, `expected_version` on guarded requests.
A JSON number does not survive a consumer that decodes into an IEEE-754
double — JavaScript's JSON.parse, Go's `any`, Python's `float`, jq. Live
tokens run past 5e17, where a double's ulp is already 64, so such a
client reads back a value NEAR the token, echoes that as its guard, and
earns a 409 against a row nobody else touched. The spec already carried
"DECODE IT AS A 64-BIT INTEGER" paragraphs telling clients to work around
this; that is a workaround for a wire type that should not have been a
number.

Emit and accept the token as a decimal STRING instead. It is documented
as opaque and EQUALITY-ONLY — compare it, never order or interpret it —
and a string round-trips exactly in every JSON consumer. It also leaves a
future backend free to mint a token that is not an int64 at all.

Internals are untouched: freshRowLock, the row_lock BIGINT column, its
migration, types.Issue.RowVersion and the ExpectedVersion *int64 guards
on the issueops requests all stay int64. Only the wire projection moves,
through the one spelling of the encoding — types.RevisionToken and
types.ParseRevisionToken.

STRICT STRING ONLY, with no transitional number-or-string acceptance.
This API already refuses unknown members by name and keeps one spelling
of the guard, and a number is precisely the shape a double-based client
has already rounded before it left the client — accepting it would
preserve the defect. A numeric guard is a 400 naming the member.

Chosen over clamping freshRowLock to 53 bits: a clamp does not fix rows
already minted with full-range tokens by in-window builds, so a JS
consumer reading one still rounds past 2^53 — repairing that would need a
row_lock normalization sweep over issues and wisps, a migration days
before a tag.

Surface: types.IssueDetails.Revision (what `bd show --json` prints and
GET /v0/beads/issues/{id} returns), the five hand-assigned httpapi
response fields, applyVersionGuardMember (now the reader for the batch
update item too, ahead of its typed decode, so a mistyped token still
earns a 400 naming the member), Problem.expected_version, and 14 OpenAPI
members regenerated through `make api-gen`. actual_version goes with them
— it is documented as "the `revision` the row was found holding", so
leaving it an integer would make the expected/actual pair disagree about
its own type; it is declared but never populated, so nothing changes
behaviorally.

Nothing shipped in a tagged release ever carried this member: v1.2.2 has
no internal/httpapi at all and no `revision` anywhere on the wire, so the
whole surface is in-window and can still be changed compatibly. Legacy
migration-0054 rows read as the string "0" with the same CAS semantics.

Corpus goldens are UNCHANGED: the canonicalizer already replaced any
`revision` value with the string "<REVISION>" whatever its JSON type.

Tests: types-level pins that the marshaled member matches `"revision":
"-?\d+"` and that RevisionToken/ParseRevisionToken round-trip MinInt64,
MaxInt64 and 2^53+1, plus a JS-realistic decode of 2^53+1 with a float64
control proving the token really is one a number would corrupt; httpapi
refusal tests for a numeric guard on update, close, reopen, delete and
both batch item kinds; a 409 echo asserted as the string; and the proxied
serve integration tests now splice the response's own token back in
verbatim rather than re-rendering a parsed one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 05:20:27 +00:00
Steve Yegge
f215b312cf
docs: re-review and bump the three 90-day-stale freshness markers (#5394)
* docs: re-review and bump the three 90-day-stale freshness markers

azure-devops.md, json-schema.md, and otel-data-model.md all carried
Last reviewed: 2026-05-08 — exactly 91 days old today, so the doc-flags
freshness gate went red on every branch at once. Each doc was genuinely
re-reviewed against its Freshness source paths on main; drift found and
fixed:

- azure-devops.md: proxied-server mode note (ado sync/status/projects
  hard-error there); --parent and --project flags documented; duplicate
  --issues row dropped; real ado.org error text quoted; ado.pat's
  yaml-only secret storage noted. Source bug found during review filed
  separately: the cmd-level config reader cannot see a yaml-only PAT,
  so the documented Quick Start fails on main (bd-wvf77).

- json-schema.md: bd show/close/update --json return top-level ARRAYS
  (contract-test-pinned); jq recipes fixed accordingly; envelope
  pagination key documented; show's always-present revision field and
  import's unchanged counter added; error envelope (optional code/hint,
  envelope-mode shape, stdout paths, exit 1) corrected; deprecation
  notice's tty-only condition noted.

- otel-data-model.md: hard opt-in via BD_OTEL_ENABLED + standard OTel
  SDK env vars (BD_OTEL_* legacy); console-only trace export; dolt span
  attrs (shared doltSpanAttrs, dolt.remote, dolt.force_push,
  dolt.conflicts, dolt.merge_with_strategy); facade-wave storage role
  spans; seven new Dolt metrics; AI attrs (bd.ai.duration_ms, 4-attempt
  retry); appendix re-pinned to current main.

scripts/check-doc-freshness.sh passes locally.

* docs(otel): complete claim_verify_lost op label enumeration; qualify retry policy as compact-only

Review findings on #5394: op also takes guarded-update
(claim_verify.go:113) and ready-claim (issues.go:439); the retry-policy
sentence describes compact's loop, find_duplicates makes one unretried
call.
2026-08-07 02:35:11 -07:00
Steve Yegge
80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00
Athos Bernardes
3102cbd4d4
fix(show): flag comment_count as omitted when comments are not streamed (#4865)
* fix(show): flag comment_count as omitted when comments are not streamed

bd show --json's count-only default (be-ijck6q, pinned by
TestJSONContract_ShowOmitsOptInPayloadsByDefault) returns comment_count
with no comments key unless --include-comments is passed. A caller that
checks only `.comments` reads absence as "no comments" — wrong whenever
comment_count > 0. A caller that reads comment_count has no signal that
the array was left out on purpose rather than omitted by a client bug.

Add comments_omitted (bool, omitempty) to IssueDetails, set true only
when CommentCount > 0 and Comments was not populated. Leaves the
count-only default and its perf rationale untouched — this does not
revert cfcc95799/e538f2881, it makes the partial representation
self-announcing. Symmetric dependent_count/dependents case is a known,
separate gap left for a follow-up (not reported by ga-clgh, out of
scope here).

Did not bump JSONSchemaVersion / regen the golden corpus: this adds an
optional field via omitempty (no existing consumer's parse breaks,
no existing corpus fixture has comment_count > 0 for `show`), and
corpus-regen needs a live Dolt+Docker this sandbox doesn't have to
verify. Deferring that call to whoever can run `make corpus-regen` and
review the diff.

Fixes ga-clgh.

* fix(show): set comments_omitted on the proxied path too

The direct path flags comment_count as omitted when comments are not
streamed (ga-clgh); the proxied path silently left the flag unset. Mirror
the else-branch in proxiedBuildDetails, add proxied-path coverage for
present/included/zero cases, and document comments_omitted in the JSON
schema reference. No schema version bump: additive optional field already
emitted by the direct path.

Addresses the 2026-07-23 review on gastownhall/beads#4865.

Agent-Signature: claude-sonnet-5-medium on behalf of maphew

---------

Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-25 16:16:41 -07:00
Steve Yegge
a530479774 test(protocol): pin the §J interchange clauses; fix the bd import header gap (wy-ltox1)
WP2 of the beads protocol conformance corpus (PROPOSAL-beads-protocol-v0 §11).
Seven §J clauses had no test; the keystone one had no test anywhere.

cmd/bd/protocol/interchange_test.go pins:

  J6.6  export -> import into an EMPTY store -> export reproduces the stream.
        bd-go had same-DB byte-stability and per-field round-trip tests, but
        nothing asserted CROSS-store equivalence — the exact blind spot where an
        import that drops, defaults, or rewrites a field hides. The fixture
        spans P0 (the zero-value priority), every core issue_type, a closed
        issue with close_reason, labels, inline comments, both dep orientations,
        metadata, assignee, and memories.
  J1.2  unknown fields ignored — a record carrying future fields at the top
        level, inside a dependency, and inside a comment still imports cleanly.
  J1.3  `_schema` header records skipped, on both the file and stdin readers.
  J6.1  import never deletes: a partial stream does not remove the ids it omits.
  J6.3  a dependency whose target id is absent is skipped AND reported
        (skipped_dependencies in `bd import --json`).
  J6.5  `bd import -` reads a stream from stdin.
  J2.4  zero/unknown timestamps export as the Unix epoch, never year-0001
        (which json.Marshal refuses outright, failing the whole export).

J1.3 needed a code fix, not just a test. parseJSONLFile (the bootstrap reader)
skipped the header record, but the `bd import` reader loop — the one `bd import`
and `bd import -` both run through — did not: the header line fell through to
the issue path, unmarshaled into an empty Issue, and aborted the WHOLE import
with "validation failed for issue : title is required". So a canonical export
carrying a provenance header could not be imported at all. Verified against the
new test: it fails with exactly that error without the guard.

Two stale artifacts found during the extraction, fixed here:

  - TestCLI_Import_PrefixValidation_E2E asserted that import REJECTS a foreign
    id prefix without --skip-prefix-validation, a flag that no longer exists
    (every import path now passes SkipPrefixValidation). Being build-tagged out
    of CI (cgo && integration), it rotted unnoticed. Rewritten as
    TestCLI_Import_ForeignPrefix_E2E, pinning the behavior bd actually has —
    which the protocol requires anyway: §J6.6 round-trips through a store with
    its own prefix, and §J7.1 makes ids opaque.
  - docs/reference/json-schema.md claimed export emits schema_version per line.
    It does not (and must not): schema_version belongs to the --json command
    envelope; the interchange's version marker is the optional _schema header.
    The import --json summary field list was also missing five fields.

Gates: gofmt + go vet clean, go build ./... ok, full cmd/bd/protocol package
green (280s, -count=1).
2026-07-13 11:20:06 -07:00
Chris Sells
69e33099b8 docs: apply PR #4722 review feedback to user docs
- FAQ: fix bulk migration/export commands (tracker pull/push need explicit
  IDs; bulk lives on sync --pull-only / --pull / --push-only / --push) and
  correct the cross-project isolation claim (hydration + external targets)
- quickstart: replace the broken DOLT.md blob URL with site links
- installation: Quick Install Script heading matches its macOS/Linux/FreeBSD
  body
- rewrite 33 stale SCREAMING_CASE link labels to the target pages' titles
- add description frontmatter to the 55 authored pages missing it
- docs.json: wildcard redirects for the old /next/ and version-prefixed
  route spaces (1.0.0, 1.0.4, 1.0.5)
2026-07-12 11:06:57 -07:00
Chris Sells
a86a2c5dfc docs: port user docs into the Mintlify site tree under docs/
Port the website's current (non-versioned) authored pages into docs/ as
Mintlify pages: frontmatter title as H1 (body H1s removed), Docusaurus
admonitions converted to Mintlify components, root-relative extensionless
links, mermaid fences kept as-is. website/docs/** is untouched and keeps
serving the Docusaurus site until cutover.

git-mv the user-facing docs/ originals with no website twin into the new
IA (core-concepts/, architecture/, workflows/, recovery/, multi-agent/,
integrations/, reference/), converted to Mintlify form. The six files
previously mirrored by sync-website-docs.sh now live once in the site.
ERROR_HANDLING.md and CONTRIBUTOR_NAMESPACE_ISOLATION.md move to engdocs/
(contributor pattern guide and design doc per content review).

Pointer stubs remain at docs/{DOLT,STORAGE-BACKENDS,PLUGIN,RECOVERY,
JSON_SCHEMA}.md because released bd binaries print those paths; QUICKSTART
and DOLT-BACKEND stay as existing pointer stubs.

Corrected while porting: the intro's storage diagram claimed data lives at
.beads/dolt/; embedded mode (the default) uses .beads/embeddeddolt/
(internal/storage/embeddeddolt/store.go), server mode uses .beads/dolt/.
2026-07-12 10:37:07 -07:00