Commit graph

14 commits

Author SHA1 Message Date
Steve Yegge
80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00
Graeme Foster
1cf7490c71
refactor(telemetry): adopt standard OTel SDK env vars with BD_OTEL_* back-compat (#3859)
* refactor(telemetry): adopt standard OpenTelemetry SDK env vars with explicit BD_OTEL_ENABLED gate

Telemetry configuration moves from BD_OTEL_* data variables to the
standard OTEL_* environment variables. Activation now requires
BD_OTEL_ENABLED=true so a machine-global OTEL_* setting (for some other
instrumented tool) cannot silently turn bd telemetry on.

The resource picks up resource.WithFromEnv() so OTEL_SERVICE_NAME and
OTEL_RESOURCE_ATTRIBUTES take effect — users can override service.name
or stamp deployment.environment without forking.

Backwards compatibility: BD_OTEL_METRICS_URL, BD_OTEL_LOGS_URL, and
BD_OTEL_STDOUT continue to work and activate telemetry on their own.
Each translates to its OTEL_* equivalent unconditionally — a legacy
value wins over a pre-existing OTEL_* value so cross-tool environments
don't redirect bd traffic. Every invocation that sees a BD_OTEL_*
variable logs a one-line deprecation warning to stderr.

OTLP endpoint discovery is delegated to the SDK; the explicit URL
helper in internal/telemetry/otlp.go is removed.

OTEL_SDK_DISABLED=true forces telemetry off even when an opt-in is in
effect, matching SDK conventions.

Tests cover: legacy var translation + override semantics, master-switch
activation, OTEL_*-alone-doesn't-activate, OTEL_SDK_DISABLED override,
WithFromEnv resource attribute merge, deprecation warning emission, and
otlpMetricsEndpointSet detection (specific endpoint, generic endpoint,
negative).

Part of the gastownhall/beads PR #3475 split: isolates the OTEL_* env
var standardisation from the WrapStorage wiring (#3858) and the
bd.prefix attribute work (separate follow-up).

* refactor(cmd/bd): extract command-telemetry wiring into testable helpers

cmd/bd/main.go's PreRunE block contained two pieces of OTel wiring —
telemetry.Init and the bd.command.<name> root span construction — that
silently decay if a future refactor drops them. Same regression class
as the WrapStorage bug (telemetry implemented but never called) —
wiring buried in cobra PreRunE escapes review.

Extract two helpers in cmd/bd/command_telemetry.go:

  - commandSpanAttrs(cmdName, version, args): pure function returning
    the initial attribute set for the bd.command.<name> span. The
    bd.actor attribute is stamped later via SetAttributes once
    getActorWithGit() resolves.
  - startCommandTelemetry(...): composes Init + Tracer.Start. Single
    callable line in main.go.

Three tests cover the commandSpanAttrs output (full attrs and the
empty-args edge case) and the startCommandTelemetry wiring contract:
it must always return a non-nil context and span, since the
bd.command.<name> span is the parent for every storage and AI span
downstream — silently returning nil would break trace nesting across
the whole invocation. Whether the span actually records is governed
by telemetry.Init which is covered in internal/telemetry/telemetry_test.go.

main.go shrinks by 10 lines.

* fixup(cmd/bd): stage command_telemetry.go split omitted from merge commit

The previous merge commit (def9f97a4) resolved main.go's PersistentPreRunE
conflict to call initTelemetry()/startCommandSpan() but the working-tree edit
to cmd/bd/command_telemetry.go that defines those two helpers was never
staged, so the merge commit as recorded does not compile:

  cmd/bd/main.go:680:3: undefined: initTelemetry
  cmd/bd/main.go:706:26: undefined: startCommandSpan

This stages that omitted edit: command_telemetry.go now defines
initTelemetry (the Init half) and startCommandSpan (the span-start half,
using commandSpanAttrs) as separate functions, with startCommandTelemetry
kept as their combined form for callers/tests that don't need to interleave
other setup between the two steps. See the merge commit message for why the
split is necessary (main's usage-metrics bootstrap block sits between the
two steps for one internal subcommand).

Verified: CGO_ENABLED=0 go build -tags gms_pure_go ./... now succeeds.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(telemetry): gate the OTLP metric exporter on OTEL_METRICS_EXPORTER

A console-only or none selection — including the legacy BD_OTEL_STDOUT=true
translation — previously still installed an OTLP reader whenever a
machine-global OTEL_EXPORTER_OTLP_ENDPOINT existed, silently exporting bd
metrics to an endpoint configured for some other tool. The selector is now
parsed per the SDK spec (comma-separated; unset defaults to otlp; none
disables) and the OTLP reader requires the selection to include otlp.

The BD_OTEL_STDOUT translation keeps both destinations when
BD_OTEL_METRICS_URL is also set (console,otlp), preserving the existing
both-exporters behavior for that combination.

Also bound the test-cleanup Shutdown with a 2s context so suites configured
with unreachable endpoints (example.invalid) don't spend ~30s in OTLP
retries.

Both issues found by a cross-vendor (Codex) review of the rebased branch.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

---------

Co-authored-by: matt wilkie <maphew@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 00:05:06 -07:00
Kevin Glynn
f4fcc318d2
docs: remove stale 8-char prefix limit claim from rename-prefix help (GH#770) (#4813)
Commit ab3feda00 removed the len(prefix) > 8 check from validatePrefix,
but "Max length: 8 characters" survived in the command's Long help text,
the plugin skill page, and docs/reference/advanced.md. Remove from all
three sources; generated CLI docs are release-pinned and untouched.
2026-07-26 12:29:39 -07:00
Athos Bernardes
3102cbd4d4
fix(show): flag comment_count as omitted when comments are not streamed (#4865)
* fix(show): flag comment_count as omitted when comments are not streamed

bd show --json's count-only default (be-ijck6q, pinned by
TestJSONContract_ShowOmitsOptInPayloadsByDefault) returns comment_count
with no comments key unless --include-comments is passed. A caller that
checks only `.comments` reads absence as "no comments" — wrong whenever
comment_count > 0. A caller that reads comment_count has no signal that
the array was left out on purpose rather than omitted by a client bug.

Add comments_omitted (bool, omitempty) to IssueDetails, set true only
when CommentCount > 0 and Comments was not populated. Leaves the
count-only default and its perf rationale untouched — this does not
revert cfcc95799/e538f2881, it makes the partial representation
self-announcing. Symmetric dependent_count/dependents case is a known,
separate gap left for a follow-up (not reported by ga-clgh, out of
scope here).

Did not bump JSONSchemaVersion / regen the golden corpus: this adds an
optional field via omitempty (no existing consumer's parse breaks,
no existing corpus fixture has comment_count > 0 for `show`), and
corpus-regen needs a live Dolt+Docker this sandbox doesn't have to
verify. Deferring that call to whoever can run `make corpus-regen` and
review the diff.

Fixes ga-clgh.

* fix(show): set comments_omitted on the proxied path too

The direct path flags comment_count as omitted when comments are not
streamed (ga-clgh); the proxied path silently left the flag unset. Mirror
the else-branch in proxiedBuildDetails, add proxied-path coverage for
present/included/zero cases, and document comments_omitted in the JSON
schema reference. No schema version bump: additive optional field already
emitted by the direct path.

Addresses the 2026-07-23 review on gastownhall/beads#4865.

Agent-Signature: claude-sonnet-5-medium on behalf of maphew

---------

Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-25 16:16:41 -07:00
Chris Sells
8c974d8dc4
docs: align user docs and CLI reference with the v1.1.0 release (#4874)
* docs: align user docs and CLI reference with the v1.1.0 release

The Mintlify site publishes from main, so the docs had drifted ahead of the
latest release — documenting features that no released binary ships.

- Remove architecture/storage-backends: v1.1.0 supports only Dolt (the
  SQLite/Postgres/MySQL backends are post-tag work on main). Redirect the
  old routes to /architecture/dolt and drop the alternate-backend claims
  from README, core-concepts, architecture index, and FAQ.
- Stamp the docs homepage with the release: "These docs are for the 1.1.0
  release" plus a link to the v1.1.0 release notes.
- Regenerate the CLI reference (docs/cli-reference/, docs/CLI_REFERENCE.md,
  docs.json nav) from the released v1.1.0 binary via tools/docsmint: removes
  pages for post-1.1.0 commands (heartbeat, migrate-personal, reclaim,
  unclaim) and post-1.1.0 flags.
- Fix hand-written pages that referenced post-1.1.0 CLI surface, verified
  against the v1.1.0 binary: bd dep relate/unrelate (not top-level
  relate/unrelate), bd admin reset (not bd reset), bd hooks list (not
  status), bd history / bd vc status (not vc log/diff), release the claim
  via bd assign + bd update (no bd unclaim), drop bd migrate-personal and
  bd daemon mentions, and reduce the Cursor integration page to the rules
  file that v1.1.0's bd setup cursor actually installs.
- Stop documenting the BD_ALLOW_REMOTE_MIGRATE escape hatch (and the
  nonexistent-in-1.1.0 bd migrate --force): the remote-migrate gate prints
  its own operator guidance by design; docs now defer to it.

* docs: auto-regenerate CLI reference

Applied from the cli-docs-freshness-patch artifact of https://github.com/gastownhall/beads/actions/runs/29613329749
(generated with CI's canonical pinned build). See
scripts/check-cli-docs-drift.sh for how drift is attributed.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-17 14:17:28 -07:00
Julian Knutsen
0cf3b3ca09
Revert "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857) 2026-07-16 16:35:51 -07:00
Test User
98573a4983 docs(storage): explain focused Dolt and SQLite support (bd-sadcd)
Document the rollback of direct PostgreSQL and MySQL adapters because supporting additional general-purpose server databases adds operational and maintenance complexity. Our goal is to keep Beads as simple as possible and consume as few resources as possible.

Clarify that embedded Dolt, Dolt server, and SQLite remain supported, including Dolt server use of the MySQL wire protocol.

Existing PostgreSQL and MySQL databases are not opened or modified. Migration requires an export from a supporting build, reinitialization guided by bd help init-safety, and import; JSONL remains issue-level interchange rather than a complete backup.

Refresh generated CLI reference material alongside the storage guidance.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Test User
2026-07-16 03:42:17 +00:00
Steve Yegge
a530479774 test(protocol): pin the §J interchange clauses; fix the bd import header gap (wy-ltox1)
WP2 of the beads protocol conformance corpus (PROPOSAL-beads-protocol-v0 §11).
Seven §J clauses had no test; the keystone one had no test anywhere.

cmd/bd/protocol/interchange_test.go pins:

  J6.6  export -> import into an EMPTY store -> export reproduces the stream.
        bd-go had same-DB byte-stability and per-field round-trip tests, but
        nothing asserted CROSS-store equivalence — the exact blind spot where an
        import that drops, defaults, or rewrites a field hides. The fixture
        spans P0 (the zero-value priority), every core issue_type, a closed
        issue with close_reason, labels, inline comments, both dep orientations,
        metadata, assignee, and memories.
  J1.2  unknown fields ignored — a record carrying future fields at the top
        level, inside a dependency, and inside a comment still imports cleanly.
  J1.3  `_schema` header records skipped, on both the file and stdin readers.
  J6.1  import never deletes: a partial stream does not remove the ids it omits.
  J6.3  a dependency whose target id is absent is skipped AND reported
        (skipped_dependencies in `bd import --json`).
  J6.5  `bd import -` reads a stream from stdin.
  J2.4  zero/unknown timestamps export as the Unix epoch, never year-0001
        (which json.Marshal refuses outright, failing the whole export).

J1.3 needed a code fix, not just a test. parseJSONLFile (the bootstrap reader)
skipped the header record, but the `bd import` reader loop — the one `bd import`
and `bd import -` both run through — did not: the header line fell through to
the issue path, unmarshaled into an empty Issue, and aborted the WHOLE import
with "validation failed for issue : title is required". So a canonical export
carrying a provenance header could not be imported at all. Verified against the
new test: it fails with exactly that error without the guard.

Two stale artifacts found during the extraction, fixed here:

  - TestCLI_Import_PrefixValidation_E2E asserted that import REJECTS a foreign
    id prefix without --skip-prefix-validation, a flag that no longer exists
    (every import path now passes SkipPrefixValidation). Being build-tagged out
    of CI (cgo && integration), it rotted unnoticed. Rewritten as
    TestCLI_Import_ForeignPrefix_E2E, pinning the behavior bd actually has —
    which the protocol requires anyway: §J6.6 round-trips through a store with
    its own prefix, and §J7.1 makes ids opaque.
  - docs/reference/json-schema.md claimed export emits schema_version per line.
    It does not (and must not): schema_version belongs to the --json command
    envelope; the interchange's version marker is the optional _schema header.
    The import --json summary field list was also missing five fields.

Gates: gofmt + go vet clean, go build ./... ok, full cmd/bd/protocol package
green (280s, -count=1).
2026-07-13 11:20:06 -07:00
Chris Sells
69e33099b8 docs: apply PR #4722 review feedback to user docs
- FAQ: fix bulk migration/export commands (tracker pull/push need explicit
  IDs; bulk lives on sync --pull-only / --pull / --push-only / --push) and
  correct the cross-project isolation claim (hydration + external targets)
- quickstart: replace the broken DOLT.md blob URL with site links
- installation: Quick Install Script heading matches its macOS/Linux/FreeBSD
  body
- rewrite 33 stale SCREAMING_CASE link labels to the target pages' titles
- add description frontmatter to the 55 authored pages missing it
- docs.json: wildcard redirects for the old /next/ and version-prefixed
  route spaces (1.0.0, 1.0.4, 1.0.5)
2026-07-12 11:06:57 -07:00
Chris Sells
5b660081ed docs: fix fact errors found by adversarial review
Every fix verified against the binary or a traced source call path:

- How Beads Works: parent-child DOES affect ready work (a blocked parent
  blocks its children — the blocked-state SQL inherits it), hash-ID
  inputs are title/description/creator/creation-time plus a collision
  nonce (live generator is internal/idgen, not the legacy one), and
  bead gates dropped from the gates bullet (currently unresolvable).
- Graph-link type strings are hyphenated (relates-to, replies-to);
  underscore forms are rejected by the parser. Fixed corpus-wide
  including the README feature bullet.
- upgrading.md: the version-era table's boundaries were refuted by git
  archaeology (embedded Dolt shipped in v0.60.0 and always used
  .beads/embeddeddolt/; nothing changed at the claimed v0.63.3
  boundary) — replaced with a layout-based identification table.
- Phantom .beads/dolt/sql-server.log path in sync-failures and the FAQ;
  the managed server writes .beads/dolt-server.log.
- FAQ: ID growth matches DefaultAdaptiveConfig (min 3, max 8,
  probability-driven), dependency-type wording no longer claims exactly
  four types, and unmeasured performance figures are hedged instead of
  stated as facts.
- quickstart: bd migrate --cleanup doesn't exist; init installs hooks
  by default (no prompt) and has no merge-driver step; ready output
  quote and bd restore recovery source corrected.
- formulas.md: step types map to issue types (task/bug/feature/epic/
  chore; anything else falls back to task — human/gate are not step
  types), and there is no built-in formula search tier.
- todo.md: bd update takes --set-labels, not --labels.
- graph-links.md: broken #dependencies fragment → #add-dependencies.
2026-07-12 10:37:17 -07:00
Chris Sells
cf2711f13e docs(nav): nest the CLI Reference under Reference
The 113-page generated CLI Reference moves from a top-level nav group to
a nested group inside Reference — Mintlify renders nested groups
collapsed by default, so the command list no longer dominates the
sidebar. The docsmint nav splice is depth-agnostic (indentation is
derived from the pages key's line), verified by regenerating and by
re-pointing its test fixture at the nested shape. Reference overview and
the beads-docs skill IA note updated.
2026-07-12 10:37:17 -07:00
Chris Sells
5c2fe0d496 docs: loss-check merge legacy docs/ originals into the site pages
Sixteen same-ground pairs reconciled: the ported website page won and every
fact living only in the older docs/ original was merged in before deleting
it (INSTALLING, SETUP, ARCHITECTURE, CONFIG, GIT_INTEGRATION, ADVANCED,
TROUBLESHOOTING, FAQ, MOLECULES, ROUTING, MULTI_REPO_AGENTS, and the five
agent-integration guides). The two divergent FAQs are now one canonical
FAQ with the comparison-first framing leading and pre-1.0 version gating
removed.

The routing pair inverted: the ported page described a routing system that
does not exist (bd routes, bd hydrate, .beads/routes.jsonl); the page was
rebuilt from the accurate sources (ROUTING.md + MULTI_REPO_AGENTS.md)
against the real surface (routing.mode config, role detection,
repos.primary/additional hydration, --repo override).

Every claim was verified against the Go source or the generated CLI
reference. Phantom surfaces removed across the corpus: bd pin/hook/pour/
wisp/routes/hydrate/reserve/lock/merge/stats, bd hooks status, bd vc
conflicts/log, bd dolt set shared-server, fictional beads_* MCP tool names
(real names taken from beads-mcp server.py), import.orphan_handling and
export.error_policy config keys, database.wal_mode/cache_size, stale
.beads/dolt/sql-server.log paths, and SQLite-era file paths. Notable
corrections: min_hash_length default is 3, .beads/metadata.json is
git-tracked, tombstone status is removed, wisps live in the main database
under an ephemeral flag.

Freshness markers moved with their content (configuration, ide-setup) and
the freshness gate + DOC_INVENTORY ledger now point at the new locations.
Load-bearing anchors preserved: #branchless-workflows-jujutsu--jj,
#circuit-breaker-server-appears-down-failing-fast, #extensible-database,
#multi-repo-hydration. Pointer stubs remain at docs/{SETUP,INSTALLING,
GIT_INTEGRATION}.md because released binaries print those paths
(allowlisted in the docsync guard).

Gates: go test ./test/docsync PASS; generate-cli-docs.sh --check PASS;
check-doc-freshness.sh PASS; mint broken-links: no broken links; mint dev
serves all sections (11 routes spot-checked at 200).
2026-07-12 10:37:07 -07:00
Chris Sells
54a7547b0d docs: canonical concept page, section overviews, and concept diagrams
- Rewrite docs/core-concepts/index.md as 'How Beads Works' — the one
  canonical concept page (modeled on Gas City's how-gas-city-works):
  problem-first framing, the bead -> dependencies -> ready-work loop, hash
  IDs, the formula -> proto -> molecule/wisp pipeline, gates, Dolt sync,
  federation, and storage modes, led by four mermaid diagrams. Other pages
  link here instead of re-deriving the model.
- Fact fixes caught while writing (verified against source/live help):
  hash IDs are content-derived SHA-256 (internal/types/id_generator.go),
  not random; bd pour/bd wisp/bd pin/bd hook are not commands (the real
  surface is bd mol pour, bd mol wisp, bd assign, bd update --claim);
  wisps are not stored in .beads-wisp/; cross-repo dependency syntax is
  external:<project>:<capability> (cmd/bd/dep.go); recovery index claimed
  SQLite errors in a Dolt product.
- Section overview pages: rewrite workflows/index, extend recovery,
  multi-agent, and integrations indexes to list every child with one-line
  summaries; add reference/index and put it in the nav.
- New concept diagrams: sequential-vs-hash ID merge collision
  (core-concepts/hash-ids) and embedded-vs-server mode
  (architecture/dolt).
- website/README.md: real build/verify instructions and parallel-run
  status for the Docusaurus site.
- Repoint example READMEs and page labels at the real quickstart page.
2026-07-12 10:37:07 -07:00
Chris Sells
a86a2c5dfc docs: port user docs into the Mintlify site tree under docs/
Port the website's current (non-versioned) authored pages into docs/ as
Mintlify pages: frontmatter title as H1 (body H1s removed), Docusaurus
admonitions converted to Mintlify components, root-relative extensionless
links, mermaid fences kept as-is. website/docs/** is untouched and keeps
serving the Docusaurus site until cutover.

git-mv the user-facing docs/ originals with no website twin into the new
IA (core-concepts/, architecture/, workflows/, recovery/, multi-agent/,
integrations/, reference/), converted to Mintlify form. The six files
previously mirrored by sync-website-docs.sh now live once in the site.
ERROR_HANDLING.md and CONTRIBUTOR_NAMESPACE_ISOLATION.md move to engdocs/
(contributor pattern guide and design doc per content review).

Pointer stubs remain at docs/{DOLT,STORAGE-BACKENDS,PLUGIN,RECOVERY,
JSON_SCHEMA}.md because released bd binaries print those paths; QUICKSTART
and DOLT-BACKEND stay as existing pointer stubs.

Corrected while porting: the intro's storage diagram claimed data lives at
.beads/dolt/; embedded mode (the default) uses .beads/embeddeddolt/
(internal/storage/embeddeddolt/store.go), server mode uses .beads/dolt/.
2026-07-12 10:37:07 -07:00