Commit graph

228 commits

Author SHA1 Message Date
Julian Knutsen
92ad0295f1
Guard and qualify historical upgrades to v1.2 (#4907)
* Add sealed-copy SQLite migration bridge (bd-mvh)

Create a fresh current workspace from a fingerprinted copy of an authenticated historical SQLite workspace. Keep every historical executable inside an isolated destination and reject malformed or semantically lossy cutovers without touching the source.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Refuse implicit historical workspace upgrades (bd-mvh)

Detect historical SQLite and legacy Dolt layouts before config migration, version tracking, init, bootstrap, doctor, or store construction. Require the sealed-copy bridge while preserving current shared-server admission against the explicitly selected workspace.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Qualify authenticated historical upgrades (bd-mvh)

Replace generic recipe discovery with a fixed, checksum-pinned corpus spanning the reviewed SQLite, server-Dolt, and embedded-Dolt eras. Assert source isolation, representative data fidelity, schema idempotence, and rollback retention while keeping historical executables inside bounded temporary environments.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Add authenticated legacy SQLite reader (bd-tku)

Let the current binary read the exact v0.49.6/v0.50.3 SQLite contract from a sealed private DB/WAL copy and emit canonical JSONL for the normal fresh-Dolt import path. Reject schema drift and values that current import would lose, default, truncate, or deduplicate.

Qualify both authentic releases end to end, including current DATETIME(0) canonicalization and reader-to-fresh-import timestamp equality.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix empty embedded-root upgrade bypass (bd-e6m)

Require an actual embedded Dolt repository marker before current layout precedence can suppress historical SQLite or legacy Dolt refusal.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* fix(migrate): maintainer review fixups for legacy SQLite upgrade PR

- decompose the legacy-workspace guard and extract the dependency-row loader
  (behavior-preserving), and align the legacy SQLite reader projection with
  issueops.ScanIssueFrom arity after the 0060/0061 rebase, guarded by a
  projection arity test.

- assert the historical-upgrade harness schema no-op by shape
  ("Schema already at v<N>") instead of a hardcoded v59, so the migration-test
  matrix stops reddening on every schema bump (candidate now prints v61).

* Fix historical upgrade review blockers (bd-mvh)

Route direct SQLite lanes through the current authenticated reader, guard no-store commands before legacy workspace mutation, and move coarse embedded repository detection behind the storage adapter. Restore the legacy reader projection contract and verify source-preserving behavior across the repaired paths.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix historical upgrade edge cases (bd-mvh)

Refuse ambiguous local server-Dolt roots before storage opens and normalize nullable legacy SQLite sentinel fields during authenticated export.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix final historical upgrade review blockers (bd-mvh)

Make explicit server selection outrank stale embedded repository markers, so ambiguous local server roots still fail closed.

Permit zero-record cutovers only through the authenticated direct SQLite reader, verify the empty round trip against a fresh current export, and align the upgrade guidance with the enforced admission matrix.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Test User <test@test.com>
2026-08-06 18:45:04 -07:00
CI Bot
50a780647f Follow the relocated large-fixture budget in the testing.Short allowlist
CI's PR policy failed on scripts/check-testing-short.sh. The check is an
allowlist keyed on file::function, and the sweeper slice (519fb1479) deleted
cmd/bd/prune_bench_test.go, moving that budget into internal/workapi as
TestCandidateIDMatcherLargeFixture — the same skip, in the package the matcher
moved to, renamed for what it now measures. The allowlist entry became stale in
the same commit that created its replacement.

This is a RENAME of the existing exception, not a new one: the old entry is
removed, so the count of approved testing.Short() sites is unchanged and a
genuinely new skip still has to be argued for.

Also two stale counts, both the same accumulation this series has been prone
to — a wave adds a role and updates the prose in one place but not the other:

  - cmd/bd/serve.go said "the two issue roles" in two places, above a struct
    that now carries thirteen.
  - internal/httpapi/roles_test.go had THREE separate cases each introducing
    itself as "the newest role" (querier, sweeper, ready counter). At most one
    can be, and none of them is after wave 4. The general point those comments
    were each restating — that the case which matters is the newest role
    missing, because that is what a caller written against the previous release
    passes — is stated once on the test's doc comment now, so each row keeps
    only what is specific to it.
2026-08-06 05:52:52 +00:00
Julian Knutsen
1e0a7bdf59
ci: shadow PR Risk capability ownership (bd-8yjq8.1) (#5324)
Record fail-closed candidate lane selection without changing which PR Risk jobs execute. This creates replayable evidence for capability-owned admission while branch protection and historical validation are still pending.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-03 20:06:16 -07:00
Julian Knutsen
8a2748c4dd
ci: reuse built bd in macOS tests (bd-vygvv) (#5311)
Reuse the current-checkout binary that each macOS job already builds instead of letting subprocess helpers relink it serially. The workflow contract test pins provenance, ordering, matrix scope, and unchanged race-enabled test commands.

Expected hosted saving: 61.66-105.74 seconds without removing coverage or raising timeouts.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-02 21:46:29 -07:00
Julian Knutsen
a6b51bd36b
ci: let macOS caches survive test failures (bd-712nf) (#5292)
Allow trusted-main cache writers to publish content-addressed module and build caches after an ordinary test failure while still suppressing saves on cancellation. This prevents cold macOS runs from deadlocking cache bootstrap behind a successful test run.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-02 12:02:32 -07:00
Julian Knutsen
ba55d4ca78
ci: give Go caches explicit ownership (bd-712nf) (#5278) 2026-08-02 07:10:35 -07:00
matt wilkie
4983ec9c32
fix(schema,ci): prepared DML — not just prepared ALTER TABLE — silently no-ops on the Dolt CLI path (#5232)
* docs(schema)+ci: prepared DML, not just prepared ALTER TABLE, no-ops on the CLI path

cli_migrations.go's header documented the Dolt CLI batch-path limitation as
being about "some prepared ALTER TABLE statements". That understates it. Per
dolthub/dolt#11345 (verified on dolt 2.2.0 and 2.2.2), `dolt sql -q`/`-f`
silently no-ops EXECUTE of any prepared DML -- an UPDATE/INSERT/DELETE built
into a SET @sql string -- while reporting success; a prepared SELECT or a
direct statement on the same path runs correctly. Prepared ALTER TABLE is one
instance of that, not the scope of it.

Broaden the comment to say so, and add Check E to check-migration-hygiene.sh
so a NEW main-plane migration cannot quietly depend on a prepared write.
Prepared ALTER TABLE keeps its accepted-idiom status and is not flagged: only
the leading keyword inside the quoted text decides. Detection buffers a
statement across lines, because the common form here is a multi-line
`SET @sql = IF(cond, '...', '...')` whose PREPARE arrives several lines later,
so a single grep cannot correlate the two.

Deliberately new-files-only. Seven shipped migrations (0035, 0037, 0041, 0047,
0053, 0055, 0058) already PREPARE writes to real tables, and that is not a
latent bug: a shipped migration is frozen, and the only path this check is
about is the fresh-schema bundle, where an empty database makes a data
backfill a no-op whether or not the prepared write lands.
cliCompatibleMigrationSQL overrides just the subset whose prepared statements
change the committed schema shape, which is what the bundle actually promises.
The guard is there to stop the idiom spreading into a migration where the
write does matter.

Verified: hygiene script passes clean on the tree; a scratch migration
carrying a prepared ALTER TABLE, a prepared UPDATE and a multi-line prepared
INSERT flags exactly the two writes and not the ALTER.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): the prepared-DML check rejected the pattern it recommends

Cross-vendor review (codex reviewer, gpt-5.6-sol) found two false positives in
Check E, both confirmed against source:

1. It flagged prepared writes to migration-local stand-in tables — which is
   exactly the remedy its own failure message hands out. Migration 0059
   (gastownhall/beads#4877) copies into __bd_0059_* throwaway tables with
   PREPARE and then has a direct statement read them, precisely so a silent
   no-op degrades gracefully instead of corrupting state. A new migration
   following that documented pattern would have been rejected by the check
   telling it to follow that pattern.

   Fixed by recognising the tree's stand-in naming convention: a double
   underscore prefix, which no real beads table uses (__bd_0059_* and
   __temp__* are the existing instances). The multi-table DELETE form yields
   an alias rather than the table, so it stays flagged — conservative in the
   right direction.

2. It scanned .down.sql files. Only migrations/*.up.sql is embedded into the
   CLI fresh bundle (//go:embed at schema.go:218), so a down migration can
   never meet the bug this check is about; scanning it rejects valid work for
   no reason.

Verified with three scratch migrations: the __bd_-targeted INSERT is not
flagged, a real-table UPDATE still is, and a .down.sql carrying prepared DML
is not. Clean pass on the tree throughout.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): exempt a prepared statement only when EVERY DML branch is a stand-in

Second cross-vendor review round found the stand-in exemption checked only the
first DML target it matched. A single

    SET @sql = IF(cond, 'INSERT INTO __stage ...', 'UPDATE issues SET ...');

carries two branches and only one executes, so matching the stand-in INSERT
first waved through the real-table UPDATE sitting beside it — the exact
false negative the check exists to prevent, reachable by combining the
recommended pattern with an ordinary one. Reproduced before fixing.

Now extracts every INSERT/UPDATE/DELETE target in the statement and exempts
only when all of them are __-prefixed. Extracting nothing recognisable no
longer exempts either: the multi-table delete form (DELETE wd FROM ...) yields
an alias or nothing, and both stay flagged.

Verified: mixed IF flags, both-branches-stand-in passes, the 0059 shape still
passes, a plain real-table prepared write still flags, clean tree still passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): exempt only the one prepared form whose write target is unambiguous

Third review round, third defect in the same exemption, and the pattern in the
failures is the point: I was trying to identify SQL write targets with a
regex, and SQL will not be identified that way.

    UPDATE __stage s JOIN issues i ON i.id = s.id SET i.priority = 0

writes to issues. Every version of this check so far extracted __stage and
exempted it. And 'ON DUPLICATE KEY UPDATE v = 2' read as a second write verb,
so a perfectly safe stand-in upsert was rejected.

Narrowed instead of patched again. A prepared UPDATE or DELETE is now NEVER
exempted -- their multi-table forms can move the modified table behind a JOIN,
so the target is not reliably knowable here. INSERT INTO is exempted when every
target is a __ stand-in, because its target is always the token after INTO
whatever the SELECT source does. ON DUPLICATE KEY UPDATE is neutralised before
the verb scan.

The cost is that a future conditional real-table-free UPDATE gets flagged and
has to be restructured the way 0059 does, or justified. That is the right side
to err on for a check whose whole purpose is refusing to guess about writes.

Verified across seven shapes: joined UPDATE to a real table flags, joined
DELETE flags, mixed IF with one real branch flags, INSERT into a real table
from a stand-in flags, the 0059 shape passes, a stand-in upsert with ON
DUPLICATE KEY UPDATE passes, clean tree passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): match the write verb as a token anywhere, not at a fixed position

Fourth review round, and both findings are the round-4 mistake again in the
detector rather than the exemption: I was requiring a SQL construct to sit at
an exact position.

- 'SET @sql := ...' never started buffering, because the assignment regex only
  accepted bare '='. Dolt accepts both.
- The DML test required the verb immediately after the opening quote, so
  '/* conditional */ UPDATE issues ...' and 'WITH t AS (...) UPDATE issues ...'
  both sailed through — valid SQL performing exactly the prepared write this
  guard exists to reject.

Rather than add two more position cases, the verb is now matched as a whole
token anywhere in the statement. That is strictly more conservative, and the
stand-in exemption is what rescues the legitimate forms — the check no longer
depends on guessing where in a statement something will appear.

The one thing that widening could have broken is a prepared ALTER TABLE
carrying 'ON UPDATE CURRENT_TIMESTAMP', which is a column attribute and not a
write. Both that and 'ON DUPLICATE KEY UPDATE' are masked before the scan, in
one shared helper used by the detector and the exemption alike.

Verified across eight shapes: ':=' assignment, leading block comment, and
leading CTE all flag; prepared ALTER with ON UPDATE, stand-in upsert with ON
DUPLICATE KEY UPDATE, and the 0059 shape all pass; joined UPDATE to a real
table flags; clean tree passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* docs(ci): state Check E's known limits instead of implying it is a parser

Fifth review round found two more bypasses: a multi-variable
'SET @guard = 1, @sql = ...' (only the first variable is tracked) and a
PREPARE split across lines. Both are valid SQL, both real.

Not fixing them here, and saying so in the script rather than shipping a
silent gap. The reasoning:

- Neither form occurs anywhere in the current migration tree. Verified: the
  only PREPARE-without-FROM hits are comments.
- Check E is advisory hygiene on NEW files, which a human reviews anyway.
- The convergent fix is statement-level scanning rather than a line-oriented
  state machine -- it subsumes all three known bypasses at once, exactly as
  rounds 3-5 kept teaching about position-anchored matching. Attempting that
  rewrite at round 5, with no remaining review budget to validate it, would
  ship an unvalidated parser. That is worse than a documented heuristic.

So the header now carries a KNOWN LIMITS block naming all three shapes and
telling a reviewer not to trust a green check when they see one. Tracked as
mybd-q8hy2.

A check that overstates its coverage is more dangerous than one that admits
where it stops.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): cover PREPARE-from-literal, and refuse to exempt an unclassified INSERT

Sixth review round, and unlike the previous round's findings these two are not
exotica I was willing to document and defer:

- 'PREPARE stmt FROM ''UPDATE issues ...''' — the literal form, no user
  variable in play — was not checked at all. That is the most direct spelling
  of the hazard this guard exists for, and arguably the first thing an author
  writes. Now checked in place rather than through the assignment bookkeeping.

- The stand-in exemption counted classified targets but not INSERT verbs.
  MySQL and Dolt accept 'INSERT issues ...' with INTO omitted, so
  IF(c, 'INSERT INTO __stage ...', 'INSERT issues ...') produced exactly one
  stand-in target, looked unanimous, and exempted the real-table write in the
  other branch. The exemption now requires every INSERT verb to have produced
  a target, so anything it cannot classify blocks the exemption rather than
  silently missing from the tally.

That second one matters more than its severity label suggests: it is the
exemption granting safety it should not, which is worse than the detector
missing a shape. An exemption must fail closed.

The two bypasses documented last commit stand as documented -- multi-variable
SET and a PREPARE split across lines, neither of which occurs in the tree
(mybd-q8hy2). The KNOWN LIMITS block now notes that the literal PREPARE form
is covered, so the list is not read as broader than it is.

Verified across eight shapes, including both new ones and every previously
passing case: literal-PREPARE UPDATE flags, literal-PREPARE stand-in INSERT
passes, INTO-omitted mixed branch flags, and the 0059 / ON DUPLICATE KEY /
prepared-ALTER-with-ON-UPDATE cases all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* docs(ci): apply the KNOWN LIMITS note the previous commit claimed to make

9850b8efc's message says "The KNOWN LIMITS block now notes that the literal
PREPARE form is covered". It does not — the edit was scripted, its assertion
failed on a wording mismatch, and because that step was not chained to the
commit the commit went ahead anyway with a message describing a change that
was not in the diff.

Applying it for real: the block now states that
'PREPARE ... FROM ''<literal>''' IS covered, so the three listed bypasses are
not read as broader than they are, and the round count says six rather than
five.

Recording the cause rather than quietly fixing it, because the failure mode is
worth knowing: a scripted edit that fails its own precondition, in a command
whose later steps are not gated on it, produces a commit that lies. Chain the
commit to the edit.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): REPLACE is a write, and a PREPARE literal may be double-quoted

Two more from review, both ordinary MySQL rather than exotica:

- REPLACE INTO was not in the write-verb set at all, so a prepared
  'REPLACE INTO issues ...' passed the check entirely. It is now detected, and
  like UPDATE and DELETE it can never be exempted -- this function classifies
  INSERT INTO targets only, so any other write verb must block the exemption
  rather than go untallied. A REPLACE targeting a stand-in is therefore
  flagged too; that is the fail-closed direction and an author can restructure
  it as an INSERT.
- The literal-PREPARE branch matched only single-quoted strings, so
  PREPARE stmt FROM "UPDATE issues ..." slipped past.

Verified across eight shapes: double-quoted literal flags, REPLACE into a real
table flags, REPLACE into a stand-in flags (unclassified, fail-closed), and
the 0059 / literal-stand-in-INSERT / ON DUPLICATE KEY / prepared-ALTER-with-
ON-UPDATE cases all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): match the non-INSERT verb boundary the same way the detector does

The exemption's UPDATE/DELETE/REPLACE rejection required whitespace after the
verb, while prepared_has_dml used a token boundary. Dolt accepts a comment as
a token separator, so

    IF(c, 'INSERT INTO __stage ...', 'UPDATE/*x*/ issues SET priority = 3')

was seen as DML by the detector, then sailed through the exemption: the
whitespace-anchored test never noticed the UPDATE, the INSERT verb and target
counts agreed, and the real-table write passed. Reproduced, then fixed by
using the same boundary in both predicates.

Two predicates that must agree on what a verb is should not spell it two
different ways; that divergence is the whole bug.

Verified: comment-separated UPDATE now flags, double-quoted literal REPLACE
flags, and the 0059 / ON DUPLICATE KEY / prepared-ALTER-with-ON-UPDATE cases
plus the clean tree all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie
2026-08-01 23:28:45 -07:00
Julian Knutsen
b73c987302
ci: decouple artifacts from duplicate checks (bd-4b1dt) (#5264)
Build and upload reusable artifacts immediately while the existing policy and lint wrapper jobs retain independent required ownership. This releases artifact-dependent tests 4.6-6.2 minutes earlier in recent PR runs without dropping any check.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-01 22:49:45 -07:00
matt wilkie
c3e600c940
fix(pr-preflight): warn when a PR-gate workflow is red for every PR while the base shows green (#5251)
* fix(pr-preflight): sample PR-gate health, not just base-branch runs (bd-msll)

The base-branch health check reads gh run list --branch <base>, which only
sees workflows that run on the base branch itself. A job that exists only in
the PR workflow (pr.yml) never runs there, so preflight could report "base is
green" while every fresh PR was red in that job, and pr-babysit mirrors
preflight's verdict. Add a supplemental sample of recent completed
pull_request-event runs across PRs: warn (or block under
PR_PREFLIGHT_BLOCK_RED_BASE=1) only when at least 6 decisive runs span 3+
distinct branches and every one of them failed; stay silent otherwise
(some success, too few samples, or an empty/unreadable sample).

Agent-Signature: claude-sonnet-5-medium on behalf of matt wilkie

* fix(pr-preflight): per-workflow PR-gate detection, warn-only, skip on red base

Review found the lumped-sample design could never fire for a single broken
pr.yml-only job: mixing all pull_request-event runs together diluted one red
workflow against every other green PR workflow. Rework to group_by(workflowName)
and flag any single workflow whose decisive runs (>=5, >=3 distinct
headBranch values) are ALL failure-class, naming that workflow in the message.

Also: drop the PR_PREFLIGHT_BLOCK_RED_BASE coupling entirely (a [block] line
here matched neither pr-babysit's red-base nor transient-block patterns, which
would have parked every merge lane and revoked the red-base merge exception —
wiring this into the patrol is a separate change); skip the sample when the
base branch is already red (that already explains PR redness and avoids the
contradictory "base branch shows green" wording); add startup_failure to the
failure-class regex to mirror pr-babysit's probe; and guard against gh exiting
0 with non-JSON output so a malformed sample stays silent instead of aborting
under set -euo pipefail.

Tests reworked for the per-workflow design: fake gh keys the PR-gate dispatch
on the presence of --event pull_request; new cases cover a mixed-workflow
sample (only the all-red workflow fires and is named), red-base-skips-sample,
an unreadable/non-JSON sample, positive coverage of the base-branch dispatch
via BASE_RUN_LIST, and the newest-run selection. Removed the
PR_PREFLIGHT_BLOCK_RED_BASE=1 block-mode test for this detector.

Agent-Signature: claude-sonnet-5-medium on behalf of matt wilkie

* test(pr-preflight): pin the PR-gate detector's warn-only contract; harden jq guard; document

Review round: a regression test asserts the detector never emits [block]
even under PR_PREFLIGHT_BLOCK_RED_BASE=1 (downstream automation classifies
unrecognized block lines as genuine merge blockers), the group_by jq gets an
abort guard for valid-JSON-but-not-objects input, and scripts/README.md
describes the new sample.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

* fix(pr-preflight): only diagnose the PR gate against a known-green base

Codex review round: with zero decisive base runs preflight has just warned
that base health is undetermined, and the broken-gate message's 'while the
base branch shows green' clause would contradict it. Gate the sample on
base_green_count > 0; fixture cases now supply a green base sample and a new
case pins the undetermined-base silence.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

* fix(pr-preflight): group the PR-gate sample by workflowDatabaseId, not display name

Codex review round 2: workflow display names are not unique and org/ruleset
runs can lack one entirely, so name-grouping could merge distinct gates.
Group by the stable id, keep the name for display. The repo-wide sample's
base-branch conflation is documented as an accepted limitation of a
warn-only supplemental detector.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie
2026-08-01 22:46:44 -07:00
Julian Knutsen
db966433eb
test: inspect benchmark command environments without exec (bd-l7i27) (#5261)
Expose the exact runBD and runShell command construction so the environment-denylist contract is tested at the process handoff. This removes two self-reexec children and the contention-sensitive five-second test timeout while preserving production execution behavior.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-01 19:50:10 -07:00
Julian Knutsen
06b34deba0
ci: retry Dolt image pulls (bd-m0o1z) (#5228) 2026-08-01 08:18:41 -07:00
CI Bot
f373508dfd merge(bd-ilb): OpenAPI v0 spec, types-only codegen and drift gates 2026-07-31 18:34:34 +00:00
CI Bot
e988fd002d feat(httpapi): OpenAPI v0 spec, types-only codegen, and the drift gates (bd-ilb)
Lands the bd serve wire contract and the machinery that keeps it honest. No
runtime behavior: nothing routes, nothing serves, and no CLI surface changes.

internal/httpapi/spec/openapi.v0.yaml is hand-written and is the source of
truth; internal/httpapi/apigen is generated from it with oapi-codegen
types-only (no server stubs, no router), via a go.mod tool directive so
contributors need no extra install.

The decisions this slice fixes, because they are one-way doors:

- No second wire struct. Issue, IssueWithCounts, IssueDetails and
  IssueWithDependencyMetadata are pinned to the canonical Go types with
  x-go-type, so the generator emits type aliases and the CLI's --json, the
  JSONL interchange and the HTTP bodies all marshal one struct. A hand-written
  Go-to-Go mapping layer would recreate the drift disease this epic exists to
  cure. Guarded by a two-way JSON-tag bijection test, not a round-trip fixture:
  a new omitempty field absent from a fixture round-trips cleanly in both
  directions and proves nothing.
- limit=0 means unlimited, matching the CLI exactly, with defaults 50 (list)
  and 100 (ready) documented as the shared constants. The only exception --
  refusal under --allow-non-loopback -- is in the limit description.
- ready returns IssueWithCounts, not bare Issue.
- Unknown query parameters are rejected with 400 invalid_argument naming the
  parameter in a machine-readable extension member: with types-only codegen the
  decoders are hand-written, and Go's silent-ignore default would WIDEN a
  filtered result set under version skew.
- /v0/beads/issues forces (created_at DESC, id ASC) on every request, and the
  spec documents the deliberate divergence from bd list's priority ordering.
- One RFC 9457 problem+json envelope everywhere, with a frozen code vocabulary
  mapped from the sentinels via errors.Is. 5xx bodies carry a static per-code
  detail; the real error goes only to the log, because driver and dial errors
  routinely embed the DSN.
- Only the status codes the six operations need are documented. Every
  documented status+code pair is permanent wire surface.

Two traps found while building this, both now guarded:

- oapi-codegen v2.6.0 silently drops x-go-type from a component that also
  carries allOf. An allOf-composed IssueWithCounts generated
  "type IssueWithCounts = types.Issue" -- the ready endpoint would have shipped
  with the count fields gone. The pinned schemas therefore repeat their
  property lists, and internal/httpapi/pinning.go turns a regression into a
  build failure.
- context.DeadlineExceeded satisfies net.Error, so a tripped request deadline
  would have mapped to 503 db_unavailable instead of a generic 500.

make api-check is the three-part gate (regenerate, diff-or-fail, spec tests)
and runs from scripts/ci/pr-policy.sh, i.e. in a required PR job -- the Main
workflow is perpetually concurrency-cancelled, so a gate living only there
would be unenforced.

The go.mod diff is the tool directive and its closure. The protocol corpus is
untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 08:37:34 +00:00
CI Bot
e2d82a54d7 refactor(list): route both bd list paths through internal/workapi and delete list_filter.go (bd-fv4)
Second half of the extraction. cmd/bd/list_filter.go is gone; the direct
path (list.go), the proxied path (list_proxied_server.go), and the four
other consumers the file had grown (search, search_proxied_server, count,
count_proxied_server, graph_apply) call workapi.BuildListFilter /
LoadStoreListConfig / LoadUOWListConfig / ApplyStatusFilter /
ValidStatusList instead.

listInput now embeds workapi.ListParams rather than copying into it. A
50-field flags -> params mapping would be a silent-swap hazard the compiler
cannot see; embedding makes cmd/bd fill the shared struct directly, so
there is one definition of a list query and nothing to keep in sync. The
CLI-only fields (output format, pager, tree/deps rendering, effectiveLimit,
--repo) stay in listInput, which is why ListParams is a subset and not the
whole thing.

applyStatusFilter and validStatusList are exported rather than kept
unexported as the design sketch had them: search.go, search_proxied_server.go
and graph_apply.go already called them, and leaving them behind in cmd/bd
would have forked the status-parsing rules.

The --limit flag now registers workapi.DefaultListLimit. internal/config's
list.limit viper default keeps its own literal 50 - internal/config cannot
import workapi, because workapi imports internal/config for
GetCustomTypesFromYAML.

Boundary enforcement lands with the package, per the design: a
workapi-frontend-boundary depguard rule denying cobra and net/http under
internal/workapi, and a pr-policy.sh grep for the symbol-level half depguard
cannot express (config.GetDirectoryLabels, os.Getenv). Both were verified to
fail on a deliberate violation and then reverted.

Gates: protocol corpus unchanged (whole cmd/bd/protocol package green with
BEADS_PROTOCOL_REQUIRE_DOLT=1, no testdata diff); TestBuildListFilterGolden
35/35 against the filters recorded from the old builder; TestEmbeddedList
52 subtests and TestProxiedServerList 65 subtests green; count, search and
graph-apply suites green; golangci-lint clean on cmd/bd and internal/workapi
apart from the two pre-existing main.go gosec baselines.

Agent-Signature: claude-code-opus-5-unknown-reasoning on behalf of CI Bot
2026-07-31 06:01:48 +00:00
Steve Yegge
63a471a1e4
fix(schema): carry main-plane wisps ALTERs onto the ignored track for fresh clones (bd-hs7fa) (#5170)
Fresh clones materialize the clone-local (dolt_ignored) tables from the
ignored migration series with the main schema cursor already at-latest, so
a main-plane migration's DDL against one of those tables silently never
reaches them. Observed in prod on the wy-98eh5 VM re-clone: 0060 added
wisps.storage_class on the main plane only, and every shared wisp scan from
a post-0060 binary failed with Error 1054 (same mechanism as wy-pt82l's
wisps.row_lock, healed by ignored/0013).

- ignored/0020: guarded ALTER ADD wisps.storage_class, mirroring 0060.
- ignored/0021: 0049's wisps TEXT->LONGTEXT widening on the ignored track
  (fresh clones otherwise cap large-content columns at 64KB).
- New embedded convergence test diffs every clone-local table's column
  shape between the fresh-init and fresh-clone doors; it found exactly the
  two gaps above and is the permanent engine-level backstop.
- check-migration-hygiene.sh check D: a new main-plane migration with DDL
  on a clone-local table must ship an ignored-series twin in the same
  change (events is in this class since 0062).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 14:10:40 -07:00
matt wilkie
fdee94f815
fix(preflight): classify status checks latest-per-name, not raw rollup entries (#5115)
* fix(preflight): classify status checks latest-per-name, not raw rollup entries

statusCheckRollup keeps every check run on the head SHA, including runs
from cancelled or superseded check suites. A head whose earlier suite was
cancelled carries stale CANCELLED/FAILURE entries forever next to the
fresh green re-runs of the same checks, so preflight's raw count reports
'N status check(s) failed or require action' while GitHub itself says
CLEAN — permanently blocking autonomous merges of green PRs (observed on
#5073/#5074/#5076: 20 stale cancelled runs plus one stale gate failure
alongside 80 green checks; each needed a manual merge).

Collapse the rollup to the most recent entry per check name/context
(completedAt, else startedAt/createdAt) before classifying, mirroring
GitHub's own latest-per-name mergeability semantics. A genuinely failing
or pending latest run still blocks; regression-tested both ways in the
fake-gh harness, which now takes an injectable rollup via PR_ROLLUP.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P3JtQ3ukoVmfg6LCKcKBBh

* fix(preflight): keep independent checks separate; handle gh zero-time exports

Cross-vendor review findings on the latest-per-name dedupe:

- Group by (__typename, workflowName, name/context), not name alone. A
  commit status and a check run sharing a name are independent required
  gates, and identically named jobs in different workflows must not
  collapse into one entry where a green run could mask the other
  workflow's red one.
- gh exports absent CheckRun times as the Go zero time (0001-01-01...),
  which defeated the // fallback chain. Normalize zero/null times and
  pick the max of completed/started/created, so a fresh in-progress
  rerun supersedes an old completed failure and reports as pending.

Regression tests for all three cases.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P3JtQ3ukoVmfg6LCKcKBBh

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:04:38 -07:00
ecuthiell
25490efb1f
test: use native repro timeout fixtures (#5074)
Agent-Signature: codex-tui-gpt-5.6-sol-ultra on behalf of Ewen Cuthiell
2026-07-28 09:01:52 -07:00
ecuthiell
4bcea42d8a
ci: give conformance explicit timeout budgets (#5073)
The shared conformance entrypoint inherited Go's 10-minute default even though the embedded oracle regularly needs longer and dedicated jobs already budget at least 30 minutes. Give Tier 1 the maintained 30-minute budget, make Tier 2's existing 10-minute behavior explicit, and bound the outer workflow at 45 minutes.

Test the exact Go argv, failure propagation from both tiers, and the outer workflow budget without sleeping through the real deadlines.

Agent-Signature: codex-tui-gpt-5.6-sol-ultra on behalf of Ewen Cuthiell
2026-07-28 09:01:39 -07:00
Steve Yegge
48cbef2430
fix(test): prebuild bd for subprocess tests; fit cmd/bd inside the package deadline (wy-4mtr0) (#5067)
'./scripts/test.sh' with no flags could never produce a green full-suite
receipt: cmd/bd tripped the 3m per-package go test deadline (panic
'test timed out', one test running at 0s — the package alarm, not a
hang). Measured on the reporting machine class: ~494s of serial test
time across ~770 tests, dominated by (1) a dozen test helpers that each
'go build' the full cgo bd binary inside the test run and (2) subprocess
tests spawning bd + embedded Dolt per invocation.

Three-part fix, no naked timeout bump:

- scripts/test.sh now prebuilds bd once and exports BEADS_TEST_BD_BINARY
  — the same fast path CI has used all along (main.yml prebuilt
  artifact); the local runner just never got it. Binary lands in the
  test-env root so cleanup is inherited; caller-supplied values win;
  skipped when the requested packages can't include cmd/bd.
- Every in-test builder now honors BEADS_TEST_BD_BINARY (six sites
  built unconditionally before). The repo-root ./bd reuse in
  buildBDForInitTests and cli_fast_test is REMOVED: it silently ran
  subprocess tests against whatever stale binary sat in the checkout
  root — a two-day-old one produced phantom TestCreateDepsAtomicity
  failures during this investigation. Tests must exercise the
  checkout's source or an explicitly supplied binary, never an
  incidental artifact.
- Default TIMEOUT raised 3m -> 10m with the measurement recorded in the
  comment: it is cmd/bd's floor, not an arbitrary pick.


Claude-Session: https://claude.ai/code/session_01EF8QbCbF6JnTuoCKvu9xiG

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 15:10:21 -07:00
matt wilkie
f8b847f781
fix(release): regenerate and gate the MCP uv.lock on version bumps (#5082)
The release workflow's MCP package gate runs 'uv sync --locked', but
update-versions.sh bumped integrations/beads-mcp/pyproject.toml without
regenerating uv.lock, and no pre-tag gate checked the lock. The stale
lock therefore only surfaced in the tag-triggered release run — after
the tag existed and could no longer be rewritten. This cost v1.1.0 and
v1.1.2 their first release runs and burned the v1.1.1 tag outright.

Two-part fix:
- update-versions.sh runs 'uv lock' as part of the bump when uv is
  available, warning loudly (without aborting the bump half-applied)
  when uv is missing or the lock fails.
- check-versions.sh gains a dependency-free check that the beads-mcp
  version pinned in uv.lock matches the canonical version (PEP 440
  normalization applied for -rc.N prereleases), plus a fuller
  'uv lock --check' freshness pass when uv is available. Both the
  release-tag pre-push hook and CI's verify-version-consistency job
  run this script, so a stale lock is now refused before a tag exists.

Agent-Signature: claude-fable-5-xhigh on behalf of matt wilkie

Claude-Session: https://claude.ai/code/session_01HmGSvX14yJcKD8Ezw7qZwd

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 12:53:58 -07:00
matt wilkie
46417b5837
ci(lint): cross-lint Windows-tagged Go, fix fmt-check exit status, pin golangci-lint (core of #5075) (#5083)
Closes #4991; extracted from PR #5075 per its review (split-merge). The
full PR bundled this fix with substantial CI toolchain-identity/host-binding
machinery; this commit keeps only the reviewed core:

- scripts/ci/pr-lint.sh: after the native golangci-lint pass, add a second
  pass with GOOS=windows GOARCH=amd64 CGO_ENABLED=0 GOWORK=off (skipped if
  the native target already matches), so files guarded by
  //go:build windows && !cgo get linted.
- Makefile fmt-check: stop swallowing gofmt's own exit status - a gofmt
  failure (not just "found unformatted files") now fails the check.
- Pin golangci-lint-action from version: latest to v2.10.1 in
  .github/workflows/pr.yml and .github/workflows/main.yml, and harden the
  lint invocations with --config=.golangci.yml,
  --modules-download-mode=readonly, and GOWORK=off.

.github/workflows/ci-measurements.yml already pinned golangci-lint to
v2.10.1 via `go install ...@v2.10.1`, so no change was needed there.


Agent-Signature: claude-fable-5-medium on behalf of maphew

Co-authored-by: Ewen Cuthiell <ewencuthiell@gmail.com>
2026-07-26 12:41:18 -07:00
ecuthiell
bbda695454
fix(scripts): remove Python from doc freshness checks (#4978) (#4979)
Use strict pure-Bash proleptic-Gregorian validation and day numbering so the full 0001-01-01 through 9999-12-31 domain is independent of platform date parsing. Keep native date only as the default TODAY provider, with distinct provider-versus-override diagnostics.

Mark the real-process suite integration-only, harden Git Bash discovery and its child environment, and require native Linux, macOS, and Windows boundaries in PR CI.

Agent-Signature: codex-tui-gpt-5.6-sol-ultra on behalf of Ewen Cuthiell

Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-25 22:53:47 -07:00
Steve Yegge
ff1d36fc1f sync: name a resolve command on conflict, fix -q asymmetry, close doc-flags hole, cover runSyncCommand (wy-j6q2z)
Bundled MINOR/NOTE polish from the wy-jpd3.4 adversarial pass:

1. syncConflictMessage now names `bd conflicts list`/`show`/`resolve` on
   every conflict halt instead of only on the generic error path.
2. check-doc-flags.sh Check 1 no longer excludes the "removed in v0.51"
   wording, closing the one phrasing a stale doc could use undetected.
3. sync_test.go gains runSyncCommand-level coverage via a fake
   storage.DoltStorage: the no-remote exit-0 gate, the noPush ->
   PushSkipped correction, --attempts 0 rejection, the JSON envelope per
   status, and exitError -> process-exit mapping.
4. TestIsNoRemoteConfiguredErrSupersetOfRemoteNotFound no longer asserts a
   condition that was unsatisfiable by construction; it now checks the
   widening is real in both directions.
6. `bd sync -q` now silences the success-path prints (printNoRemoteGuidance,
   "Sync complete.") that ops.progress was already gated on, so quiet means
   quiet on a solo rig too. Conflict/retry reports stay unsilenced -q is
   "errors only".
7. A pull error unrelated to a live conflict (e.g. a transport failure
   racing another writer's already-conflicted state on a shared
   sql-server) is no longer silently discarded when the run reports the
   conflict: syncOutcome.DiscardedPullError carries it through to the
   halt message.

Finding 5 (sync's default-remote path never adopts a git origin as Dolt
remote, unlike `bd dolt push`) is confirmed real by code reading but left
unfixed here; filed as wy-gpzg7.

GATE-RECEIPT: go test ./cmd/bd/ -count=1 -timeout 40m rc=0
2026-07-25 19:28:31 -07:00
ecuthiell
8137ed4092
test(preflight): inject fake gh in Bash (#4967) (#4968)
Execute the production script as a real Bash process whose controlled BASH_ENV defines the fake gh function. Curate shell, Git, GitHub, and path state; poison inherited startup/configuration in the fixture; and require the process boundary on Linux, macOS, and Windows.

Agent-Signature: codex-tui-gpt-5.6-sol-ultra on behalf of Ewen Cuthiell
2026-07-25 16:14:33 -07:00
Steve Yegge
f94ea5bfe5 feat(sync): bd sync — the federation loop as one verb (wy-jpd3.4)
Every multi-machine beads deployment hand-rolls the same shell loop around
pull/push; this ships it as `bd sync [--remote <name>] [--attempts N]`.

Three properties are the point of the verb, and each is a lesson paid for by
the reference deployment (a two-machine federation on a 60s timer):

- Conflicts are detected POSITIVELY, from the merge's own captured conflict
  rows and from dolt_conflicts — never inferred from the pull's exit status.
  A pull fails for reasons that are not conflicts, and a settled merge that IS
  conflicted aborts and leaves dolt_conflicts empty, so an exit-status guess
  both invents phantom conflicts and misses real ones. A pre-flight check also
  halts on conflicts left live by an earlier halted sync, which Dolt otherwise
  refuses to merge over with an opaque transport error.
- A conflict sync cannot settle is NEVER resolved by picking a side: it halts
  before recomputing or pushing, with no --strategy-style override. A timer
  that silently picks a side loses work on a schedule. The halt message
  reports whether the conflict rows are live or were aborted away, read from
  WHICH detection source fired rather than assumed — the SQL pull route aborts
  and restores the working set, the CLI/git-protocol route deliberately leaves
  the rows live, and an operator told the wrong one looks in the wrong place.
- The is_blocked recompute runs unconditionally, on every attempt. That is
  deliberate and load-bearing: RecomputeAllBlocked is specifically the repair
  that does NOT depend on a merge advancing HEAD (bd-6dnrw.37), so it is what
  recovers a column left stale by a conflicted pull an operator resolved by
  hand — a state this verb manufactures itself by exiting 2. Gating it on "did
  this tick merge anything" re-imposes the condition it exists to escape, and
  from then on no tick merges anything, so the repair would never run again
  while every tick reported success. HEAD is the wrong instrument for that
  question anyway: the pull's own pre-merge auto-commit (GH#2474) moves it for
  purely local dirty state, and an auto-resolved merge can land in the working
  set without moving it at all (bd-6dnrw.39).

Exit codes are the machine contract so a timer can branch without parsing
output: 0 synced, 1 error, 2 merge conflict (halted, nothing pushed), 3
push-race retries exhausted (transient). --json reports the same outcome.

Push-race classification is deliberately narrow — race signatures (behind /
non-fast-forward / a lost --force-with-lease: stale info, fetch first, "contains
work that you do not have"), not the word "rejected". A protected branch or a
declining pre-receive hook also rejects, permanently; treating that as a race
burns the whole attempt budget every tick and reports exit 3 ("retry next tick")
forever instead of surfacing the failure.

Also: hasNoRemoteConfigured split out of isConfirmedNoRemote so sync can reuse
the structural no-remote proof with its own (broader) error hint without
loosening it — a solo rig must not fail every tick, but a typoed or broken
remote must still exit non-zero. The command doc and the doc-flags gate are
updated: `bd sync` is live again, so the stale reference to hunt is docs still
calling it removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-24 18:15:13 -07:00
Jim Wordelman
ae48b0a090
perf(dolt): eliminate 12s slow path on bd commands when .local_version is stale (#3710)
* perf(dolt): be-1he eliminate 12s `dolt remote -v` against multi-db server roots

Three layers of defense for the bd 12s slow-path bug:

- Layer 1: migrateServerRootRemotes skips when rootDir/.dolt/repo_state.json
  is missing. Multi-database server roots have only sql-server.info; running
  `dolt remote -v` there takes ~12s to fail.
- Layer 2: ListCLIRemotes wraps the dolt subprocess in a 2s context timeout.
  Backstops Layer 1 against any future variant of the broken-parent-dir case.
- Layer 3: autoMigrateOnVersionBump does a read-only probe of bd_version
  before opening writeable. Read-only stores skip syncCLIRemotesToSQL, so a
  stale .local_version (e.g. from a multi-binary install) no longer triggers
  the slow path on every command.

End-user impact pre-fix: gc mail inbox 60-80s, gc status 30s+ hangs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for be-1he (3-layer slow-path fix)

* docs(repro): add be-1he slow-path repro script (be-vxngmn)

Adds scripts/repro-be-1he-slow-path/repro.sh + REPRO.md demonstrating
the 12-second `dolt remote -v` slow path on multi-DB server roots.

Satisfies coffeegoddd's review request: "We need a beads-only repro of
this to prove its a bug."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(cli): regenerate CLI reference after slow-path fix rebase

Output of ./scripts/generate-cli-docs.sh ./bd and ./scripts/generate-llms-full.sh.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(cli): regenerate CLI reference + llms-full with pure-go bd

The previous regen (0aeb5934e) was built with a CGO-enabled bd, which
includes the CGO-only `bd federation` subcommands. CI builds the docs
binary with CGO_ENABLED=0 (scripts/ci/pr-policy.sh build_docs_binary),
so check-doc-flags rejected the extra federation subcommand pages.

Regenerated docs/CLI_REFERENCE.md, the federation cli-reference pages
(live + versioned 1.0.0/1.0.4/1.0.5), and website/static/llms-full.txt
with a CGO_ENABLED=0 binary. check-doc-flags + check-doc-freshness now
pass locally under LC_ALL=C.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(release-gate): correct be-1he layer description to match shipped diff

The PR body and this gate doc both described a "Layer 1" repo_state.json
sentinel in internal/storage/dolt/federation.go, but the actual diff never
touched that file - only cmd/bd/version_tracking.go (Layer 3) and
internal/storage/doltutil/remotes.go (Layer 2) are changed. Correct the
gate doc's commit table, acceptance-criteria row, and follow-up/out-of-scope
notes to describe only what ships. No code or test changes; Layer 2 and
Layer 3 remain independently verified as still-needed on current main
(ListCLIRemotes has no timeout, no equivalent sentinel exists).

Agent-Signature: claude-sonnet-medium on behalf of maphew

* fix(dolt): scope ListCLIRemotes timeout, fix repro.sh, purge Layer 1 narrative

Addresses the cross-vendor review's four should-fixes plus the comment nit
(gastownhall/beads#3710 review, 2026-07-24):

1. ListCLIRemotes' 2s cap could turn a slow-but-valid dolt remote -v into
   "remote absent": FindCLIRemote folds any error (including a timeout) into
   "", and EnsureCLIRemote then blind-adds on that signal, which hard-fails
   if the remote actually exists. Chose the "scope the cap" fix over
   "propagate DeadlineExceeded distinctly": it's fully contained inside
   ListCLIRemotes (no changes needed at the three call sites - remotes.go:192
   ensure-path, store.go:1958 URL match via hasMatchingCLIRemote, and
   doctor/federation.go:127, which already treats errors distinctly from "no
   remotes"), whereas threading a sentinel error type through FindCLIRemote's
   string-returning signature and its callers touches more surface for the
   same outcome. ListCLIRemotes now stats dbPath/.dolt/repo_state.json: 2s
   cap when absent (the known broken multi-DB server-root case - there's
   never a real answer coming from there), 30s cap otherwise (a real repo's
   `dolt remote -v` is ~130ms even loaded, so 30s only bites a genuinely hung
   subprocess). Added TestListCLIRemotesTimeout covering both branches,
   following the t.TempDir() pattern already used in
   persisted_remotes_test.go.

2. repro.sh's baseline `dolt remote -v` ran in the invocation directory, not
   $SERVER_ROOT, so the central "slow" measurement never exercised the
   constructed broken root. Fixed to `(cd "$SERVER_ROOT" && dolt remote -v
   2>&1) || true`, matching the correct subshell pattern already used later
   in the script. Also relabeled the "PARTIAL FIX ... Layer 2 timeout capped
   it" branch: the 2s cap lives in bd's Go code, not dolt, and this script
   calls the raw dolt binary directly - it's never capped. Relabeled to
   describe the underlying dolt slowness the script demonstrates instead of
   implying bd's timeout is in play here.

3. `dolt init --quiet` isn't supported by dolt 2.2.x and set -e killed the
   script before it reached the summary. Redirected output instead
   (`dolt init >/dev/null 2>&1`). Ran repro.sh end-to-end locally (dolt
   2.2.0): completes with exit 0, and the fixed subshell now shows real
   timing/output from inside $SERVER_ROOT.

4. Purged the unshipped "Layer 1" (federation.go sentinel) narrative from
   repro.sh and REPRO.md - same reconciliation already done for the release
   gate doc and PR body. Both files now describe only the two layers that
   ship (remotes.go timeout, version_tracking.go read-only probe), with a
   short note that an earlier draft described a Layer 1 that never landed.

5. cmd/bd/version_tracking.go:195-197 comment cited syncCLIRemotesToSQL,
   which doesn't exist anywhere in the tree, and overstated the read-only
   probe's benefit. On current main the writeable-open gate reads remotes
   via doltutil.PersistedRemotes (a fast on-disk repo_state.json probe, no
   dolt subprocess), so the probe saves an initSchema round-trip, not a 12s
   hang. Corrected.

Agent-Signature: claude-sonnet-medium on behalf of maphew

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-24 00:53:06 -07:00
coffeegoddd☕️✨
6389546c56 test(proxied): split the six heaviest suites so shards balance under the timeout
The proxied cmd lane is init-bound: each subtest's bd init runs the full
migration chain, so a shard's wall-time tracks its total init count. The six
heaviest suites (Update 56 inits, Show 47, Create/Close 41, Dep 36, Ready 34)
were each a single top-level function, and a function is the atomic unit of
sharding — so no shard split could pull the worst shard below Update's ~169s
(local), which is what kept brushing the 15m budget on CI.

Split each of the six into 2-3 top-level functions (Update/Show ->3, the rest
->2), balanced by init count, by inserting function boundaries between the
already-independent per-subtest blocks — no test logic moves, every t.Run set
is byte-identical to before. Regenerate the 15-shard manifest by greedily
bin-packing every TestProxiedServer* function by init count
(scripts/ci/gen_proxied_shard_manifest.py); the heaviest shard drops from ~56
to ~31 estimated inits, roughly halving the long pole.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 15:55:17 -07:00
Julian Knutsen
8ebd53e882
fix(update): resolve metadata/note merges inside the mutation transaction (concurrent lost-update) (#4732)
* fix(update): resolve metadata/note merges inside the mutation transaction

bd update --metadata/--set-metadata/--unset-metadata/--append-notes and
bd note resolved their merges client-side: read the issue in one
transaction, merge in process memory, write the merged blob back in a
later transaction. Two concurrent writers touching DIFFERENT keys of
the SAME issue both exited 0, but the later commit silently erased the
earlier writer's committed keys. The multiprocess-safety audit proved
the loss empirically: 7 of 200 exit-0 --set-metadata writes vanished in
an 8-writer same-issue hammer against the stock binary.

The first version of this fix moved the merge into the mutation
transaction on the direct path only; red-team review blocked it because
the proxied-server route still snapshot-merged in
buildUpdateSpecForIssue — and worse: on a Dolt serialization rollback,
uow.CommitWithRetries retried only the COMMIT, re-committing the
rolled-back (now empty) session. Dolt answered "nothing to commit",
isDoltNothingToCommit swallowed it, and bd printed "Updated" and exited
0 for a write that never landed.

This revision closes both paths:

- Merge-shaped edits travel as operation keys (issueops.OpMergeMetadata,
  OpSetMetadata, OpUnsetMetadata, OpAppendNotes) and are resolved
  against the row re-read INSIDE the mutation transaction
  (issueops.ResolveMergeOps) — in both the embedded path
  (updateIssueInTx) and the domain/db repository used by proxied-server
  mode. CLI paths (bd update, bd note) no longer pre-merge; the shared
  merge helpers live in internal/storage.
- The in-transaction read takes a row write lock
  (GetIssueForUpdateInTx, SELECT ... FOR UPDATE) so the read itself
  serializes on Postgres (READ COMMITTED) and MySQL (REPEATABLE READ).
  sqlitedialect strips the clause — safe because _txlock=immediate
  already takes the database write lock at BEGIN. Dolt parses FOR
  UPDATE as a no-op and relies on commit-time conflict detection plus
  the whole-attempt retry below.
- The proxied server redoes the WHOLE read-merge-write attempt in a
  fresh unit of work on a serialization failure (new exported
  uow.IsSerializationError; exponential backoff, 15s budget), commits
  each attempt exactly once, and fails loudly to stderr when retries
  exhaust instead of exiting 0.

Verification: cross-backend regression body in
internal/storage/conformance/lostupdate.go (two independent store
handles plus a lock-holding third pool) wired into the sqlite, dolt
(embedded and sql-server), mysql, and postgres suites; cmd/bd
multi-process race tests for --set-metadata and bd note; proxied-server
attempt/retry unit tests. Re-running the audit hammer against the
patched binary: 0 lost of 200 exit-0 writes across natural and
lock-widened runs (stock binary lost writes in both), with the one
contention exhaustion surfacing as a nonzero exit instead of a silent
loss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(update): string --set-metadata, proxied partial-failure exit, drop dead locking scaffolding

Maintainer adoption-review fixes for PR #4732 (review iterations 1 and 3):

- metadata: MetadataEditValue stores --set-metadata values as JSON strings
  again (GH#4146) instead of inferring numbers/booleans/null from content,
  which silently broke map[string]string round-trips. Typed values stay
  available via --metadata / --metadata-json. Domain merge-op tests now
  expect string values for --set-metadata.
- cmd/bd: restore the proxied multi-ID partial-failure accumulator so a batch
  with any failed ID exits non-zero (had regressed #4726), and derive success
  output from the resolved jsonOutput signal so --format json and config
  json:true are honored on the proxied path.
- storage: remove the dead internal/storage/sqlitedialect package (509 LOC,
  zero importers) and the FOR UPDATE / GetIssueForUpdateInTx row-locking
  scaffolding. SQLite/Postgres/MySQL are gone and Dolt treats FOR UPDATE as a
  parse-only no-op; the read-merge-write is made safe by commit-time conflict
  detection plus the whole-attempt retry. Also drop the uncalled MVCC-only
  conformance helper. updateIssueInTx returns to baseline complexity via a
  small read-and-resolve helper.
- tests: remove the SQLite-backend race tests (the removed backend can't run
  them); Dolt concurrency coverage in uow/embeddeddolt/domain-db is
  load-bearing.
- readability: split ResolveMergeOps into resolveMetadataMergeOps and
  resolveNotesAppendOp (with an isMergeOpKey helper) so each diff-authored
  merge-op resolver stays below the review complexity gate (cyclomatic <21 /
  cognitive <26). Pure structure change: same resolution order, error strings,
  schema validation, and copy-not-mutate semantics.
- policy: allowlist TestUOW_ConcurrentMergeOps_NoLostUpdate in
  scripts/check-testing-short.sh. Its testing.Short() skip is a legitimate
  runtime/stress skip (it starts a Dolt container and a dbproxy subprocess),
  matching the other allowlisted Dolt concurrency stress tests; without the
  entry the ci-pr-policy check fails on the adopted head.

- tests: remove the two obsolete cmd/bd multi-ID tests
  (TestMultiIDUpdatePartialMetadataMergeFailure and
  TestMultiIDUpdatePartialMetadataEditFailureJSON) and their now-unused
  seedNonObjectMetadata/showMultiIDUpdateMetadataValue helpers. They seeded a
  non-object array via `bd update --metadata "[]"` to force a later per-ID
  metadata-merge failure, but the new in-transaction merge correctly rejects
  non-object metadata ("new metadata is not a JSON object"), so that seeding
  path is unreachable and the tests now fail at setup. The four #4726 per-ID
  exit-code tests in the same file are retained.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Test User <test@test.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 05:59:31 -07:00
Test User
6fd9dbda7d refactor(storage)!: remove the SQLite backend and dialect
Extend #4847's rollback to the last alternative backend: bd now ships a
single storage engine (Dolt, embedded and server mode).

- Delete internal/storage/{sqlite,sqlitedialect} and the packages the
  removal orphans: sqlkit (sole importer was the sqlite store),
  unsupportedgen (sole go:generate consumer), the NonCommitGraphBackend
  marker and its two dead consumer checks, and the domain-db parity test
  that used sqlite as its harness. Drop modernc.org/sqlite from go.mod
  and regenerate the nix vendorHash.
- Tombstone backend="sqlite" exactly like postgres/mysql: recognized in
  metadata (never silently aliased to a fresh Dolt DB), rejected before
  any open with its own rationale and the export -> re-init -> import
  guidance. The fail-closed guarantee now also covers never creating the
  beads.db file the removed backend would have provisioned.
- bd init: --backend=sqlite reaches removed-backend guidance;
  --sqlite-path becomes a hidden parser tombstone like the pg/mysql
  flags; help text and supported-backend enumerations say Dolt only.
- Doctor: sqlite workspaces fail closed like the other tombstones; the
  modern sqlite stub payloads are gone. The legacy pre-Dolt surface
  (artifact scans, migration validation, migrate-sqlite-to-current.sh,
  the migration-test harness) is untouched; genuine pre-Dolt workspaces
  carry no backend field and never hit the tombstone.
- Conformance: drop the sqlite profile; the CLI differential runner
  idles (skip, not fail) until another backend profile is registered,
  and the export/import round-trip keeps running against the embedded
  Dolt reference. Tier-1 oracle legs are unchanged.

Backend selection has never shipped in a release (v1.1.0 predates it),
so released workflows are unaffected and the pinned CLI docs already
describe the resulting behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 02:43:56 +00:00
Test User
0d7cf0022d Reapply "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857)
This reverts commit 0cf3b3ca09.
2026-07-18 02:43:56 +00:00
Chris Sells
d2f87d50de
docs(pipeline): pin the CLI docs to the released bd; repair main's broken CLI reference (#4876)
* docs(pipeline): pin the generated CLI docs to the released bd; repair main's CLI reference

The #4874 squash-merge raced the docs-autofix bot: the bot had pushed a
regeneration from main's source onto the PR branch, so the squash landed a
contradictory mix — the release-aligned hand-written docs plus a CLI
reference re-documenting unreleased commands and flags, with docs.json
referencing four pages (heartbeat, migrate-personal, reclaim, unclaim)
whose files do not exist. That broken nav fails the docsync guard on main
and on every branch cut from it. This restores the intended end state of
#4874 and lands the pin that prevents the race from recurring.

- Regenerate the CLI reference from a bd built from the v1.1.0 tag (CI's
  canonical pure-Go build), removing the unreleased commands/flags and
  fixing the dangling nav entries.
- docs/cli-docs.pin names the release tag; scripts/resolve-docs-bd.sh
  builds and caches a canonical pure-Go bd from that tag, fetching the tag
  first in shallow CI checkouts.
- generate-cli-docs.sh, check-doc-flags.sh, and check-cli-docs-drift.sh
  resolve bd through the pin and ignore a supplied binary while pinned
  (BD_DOCS_IGNORE_PIN=1 bypasses), so the drift gates and the autofix bot
  now enforce docs-match-release instead of docs-match-main.
- tools/docsmint detects the pre-overhaul Docusaurus staging layout that
  bd <= v1.1.0 emits and converts it to the generic form before its normal
  Mintlify transform (tested); removable once the pin reaches a release
  whose emitter writes build/cli-docs.
- check-doc-flags.sh: run without timeout(1) on macOS instead of failing
  Check 4.
- Decision record: engdocs/decisions/2026-07-17-docs-release-pin.md; the
  beads-docs skill §8 updated to match.

* lint: annotate gosec G705 false positive in the docsmint legacy bridge

Same class as the existing nav-splice annotation: a developer tool writing
repo Markdown, not a web-output sink.
2026-07-17 18:03:28 -07:00
Julian Knutsen
0cf3b3ca09
Revert "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857) 2026-07-16 16:35:51 -07:00
Test User
cafb6888bd refactor(storage): remove direct PostgreSQL and MySQL adapters (bd-sadcd)
Roll back the direct adapters, dialect layers, credential handling, dependencies, CI services, and conformance profiles for general-purpose server databases. That support adds operational and testing complexity at odds with our goal of keeping Beads as simple as possible and consuming as few resources as possible.

Keep the Storage and DoltStorage interfaces unchanged. Embedded Dolt, Dolt server, and SQLite remain supported; the MySQL wire driver remains because Dolt server uses it.

Retain legacy metadata identifiers and hidden init flags only as compatibility tombstones. Removed, unknown, or corrupt backend metadata now fails safely across CLI, SDK, bootstrap, doctor, factories, and Dolt administration before storage is opened or modified, with guidance to export using a supporting build and reinitialize via bd help init-safety.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Test User
2026-07-16 03:35:18 +00:00
coffeegoddd☕️✨
e79ea8e65d /{.github,docs,engdocs,go,internal,scripts}: bump dolt 2026-07-15 13:14:34 -07:00
Julian Knutsen
96c098db14
fix: support PR preflight on gh 2.45 (bd-of1cd) (#4737)
Query closing-issue presence through GraphQL nodes instead of the unsupported gh pr view field. Reject malformed and partial responses before applying contributor policy.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Test User

Co-authored-by: Test User <test@test.com>
2026-07-12 20:19:32 -07:00
matt wilkie
f6bd99333d Merge upstream/main into docs/mintlify-overhaul
Conflict: .githooks/pre-push — keep main's stdin capture + chained beads
hook integration (96ec4db4e), apply this branch's Docusaurus purge to the
release-tag block (version-only wording, no snapshot-docs references).

Agent-Signature: claude-fable-5-high on behalf of maphew
2026-07-12 19:11:43 -07:00
Julian Knutsen
ddca3e3ef1
fix(security,ci): redact all DSN password forms in telemetry; require pg/mysql conformance (post-merge #4601) (#4712)
* fix(security,ci): redact all DSN password forms in telemetry; require pg/mysql conformance

Post-merge remediation for #4601 (choose-your-own-backend), reviewed landed
range 812ad6d..3f0474a. Addresses the two gating findings from the post-merge
review.

Security (telemetry DSN leak): scrubArgsForTelemetry only redacted
user:PASS@host userinfo, so `bd init --pg-url` URL `?password=`/`?sslpassword=`
query params and libpq `password=`/`sslpassword=` keyword/value forms leaked
into the bd.args telemetry span before init reached the parser-backed
redaction path. Route --pg-url/--mysql-url values (both `--flag=value` and
`--flag value`) through the parser-backed pgdialect scrubber via a new exported
pgdialect.ScrubDSNString; every other arg keeps the narrow userinfo scrub, so
ordinary text is never over-redacted.

CI (conformance anti-skip): scripts/conformance.sh ran the pg/mysql Tier-1
suites with no skip/PASS detector, so a missing BEADS_*_TEST_URL self-skipped
to green and silently narrowed backend coverage. Add require_backend +
run_sql_conformance mirroring the embedded-Dolt reference guard, and set
BEADS_CONFORMANCE_REQUIRE=postgres,mysql in conformance.yml (which also arms
the existing Tier-2 e2e hard-fail).

Tests: pgdialect TestScrubDSNString; cmd/bd TestScrubArgsForTelemetry (all DSN
password forms x both spellings) and TestScrubArgsForTelemetryLeavesOrdinaryArgs
(no over-redaction + userinfo defense-in-depth). Bash guards unit- and
integration-tested against real `go test` self-skip/required paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): redact federation add-peer --password/-p from telemetry args

The bd.args telemetry root span scrubbed --pg-url/--mysql-url DSN passwords
but not federation add-peer's --password/-p, so `bd federation add-peer
<name> <url> --user u --password <secret>` leaked the plaintext SQL
password verbatim into the exported bd.args attribute and trace logs — the
same sink and harm class the DSN scrub already closed.

Redact by flag identity: secretFlagTokens resolves the secret flag's actual
--long/-short tokens from the running command, so -p is redacted only on the
command that binds it to --password (federation add-peer) and never on the
many commands that bind -p to --priority/--prefix/--parallel. The value is
redacted wholesale across every pflag spelling: --password <v>,
--password=<v>, -p <v>, -p=<v>, and the concatenated -p<v> shorthand.

Tests cover all five spellings, the command-aware token resolution, and the
no-over-redaction guarantee for the overloaded -p on non-secret commands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): redact raw encoded URL-userinfo passwords in DSN scrubber

Post-merge remediation follow-up review (attempt-4) for #4601: the shared
pgdialect DSN redactor missed percent-encoded URL-userinfo passwords when
called directly — ScrubDSNError in driver.go, and any telemetry/log sink that
scrubs a raw DSN through ScrubDSNString.

dsnPasswordValues added only url.User.Password(), which url.Parse returns
percent-DECODED, so ScrubDSNString("postgres://u:SUPER%2ASECRET@h/db", same)
left the raw "SUPER%2ASECRET" in error text and logs. The query-param branch
already adds both the raw and decoded forms; the userinfo branch now does too,
extracting the raw as-written password structurally (rawURLUserinfoPassword)
so it matches regardless of encoding form (mixed case, over-encoding).

Tests: add encoded-userinfo cases to TestScrubDSNError and TestScrubDSNString
(the latter now checks every leak form, matching TestScrubDSNError). Both new
cases fail on the pre-fix redactor and pass after.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(security): close five telemetry DSN-redaction bypasses found in review

Test-first fixes, each reproduced against pgx.ParseConfig before fixing:

- percent-encoded query KEYS (pass%77ord=, %70assword=) are live password
  params after url.Query() decoding; compare unescaped keys too
- pflag boolean-shorthand clusters (-qpSECRET parses as -q -p SECRET) leaked
  the raw token; walk the leading letter run to the first secret shorthand
- backslash-escaped whitespace in unquoted libpq values (password=SUPER\ SECRET)
  leaked the tail; value pattern now consumes escaped sequences
- pgx treats \v as DSN whitespace but Go regexp \s does not; spell the
  whitespace class out explicitly
- overlapping secrets (one a prefix of another) leaked the longer tail;
  replace longest-first in ScrubDSNString

Found by paired cross-vendor review (claude fable-5 + codex gpt-5.6-sol) on
PR 4712.

Agent-Signature: claude-fable-5-high on behalf of maphew

---------

Co-authored-by: Eddie the Engineer <test@test.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: matt wilkie <maphew@gmail.com>
2026-07-12 13:44:35 -07:00
Chris Sells
789044b83e chore: purge deleted docs-pipeline references from release tooling, CI, and repo plumbing
- beads-release formula: drop the snapshot-docs step (ran the deleted
  scripts/snapshot-release-docs.sh on the critical needs-chain); stamp-
  changelog now needs update-vendorhash
- update-versions.sh: remove the mangled --skip-docs/docs-snapshot logic
  and garbled usage text left by mid-line deletions
- pre-push hook: drop the snapshot-script remediation hint
- release.yml: collapse the prerelease branch; BEADS_REQUIRE_RELEASE_DOCS
  has no consumer since check-docs-version.sh was deleted
- RELEASING.md / scripts/docs.md: describe the current pipeline (no
  --versioned, no website/ outputs, no check-docs-version.sh)
- docs-mintlify.yml: paths filter now includes engdocs/** and the curated
  root markdown files the docsync guard validates
- docs-render-check.sh: pin mint@4.2.687, fail closed when mint dies
  without producing a broken-links report, parse report lines only, drop
  dead extract_page_links(), fix the stale jq requirement; mint.sh pins
  the same version
- nightly.yml: remove the deleted website measurement suite option
- CODEOWNERS: init-safety ADR gate follows the file to engdocs/adr/
- beads-docs skill: align with settled decision 6 — no pointer stubs,
  moved pages get docs.json redirects; bd's printed paths are fixed at
  the source
- CHANGELOG (edited line), FEDERATION-SETUP, README, examples, oracle-a,
  .buildflags, gh-issue-to-pr formula: retarget moved-file references;
  anchor the build/ gitignore pattern
2026-07-12 11:07:26 -07:00
Chris Sells
39d07f8442 fix(docs-tooling): harden the CLI-docs pipeline per PR #4722 review
- docsmint: all page rewrites (HTML-comment, relative-link, ESM) now skip
  fenced code blocks; fence contents render verbatim
- docsmint: spliceCLINav fails closed when the "pages" key found after
  "group": "CLI Reference" is not a sibling key of the same object, and
  validates the spliced docs.json parses before writing
- docsmint: neutralize MDX ESM hazards in docs/CLI_REFERENCE.md too (bd
  emits it without the per-page transforms, but Mintlify serves it as a
  hidden page); regenerated docs/CLI_REFERENCE.md accordingly
- help_all: fail loudly when two commands collide on the same doc page ID
  instead of letting the later write win; repair a garbled comment
- generate-cli-docs.sh: --check no longer trusts a repo-root ./bd it was
  not explicitly given (the CGO guard detects CGO-ness, not staleness)
- docsync: validate the docs.json redirects array (well-formed, no dup
  sources, destinations resolve)
- pour: runtime strings match the corrected Long text (Dolt sync, issue
  database — not "sync to git" / "persistent in .beads/")
- testdata: formula_help.txt asserts the new docs/workflows/formulas.md path
2026-07-12 11:07:12 -07:00
Chris Sells
abb0357f77 ci: satisfy build-tag policy and gosec in the new docs tooling
check-build-tags requires every go test/go run in tracked scripts,
workflows, and the Makefile to carry -tags=gms_pure_go (ICU-linkage
guard): added to the docsync invocations in docs-mintlify.yml and
make check-docs, and to the docsmint stage of generate-cli-docs.sh
(the flagged workflow job name no longer embeds a bare invocation).
docsmint's two variable-path os.ReadFile sites get justified
#nosec G304 annotations — both read only repo-owned files derived
from the tool's repo-root argument. golangci-lint: 0 issues.
2026-07-12 10:37:17 -07:00
Chris Sells
de0200ff77 docs(emitter): Cobra-accurate usage lines, published global flags, MDX ESM guard
Three generated-content defects from the adversarial review, all
structurally invisible to the drift gate (generated-vs-generated):

- Usage lines now mirror Cobra's help template: a runnable command shows
  its UseLine, a command with subcommands shows '<path> [command]', both
  when both apply. Previously bare UseLine() stamped 'bd mol [flags]' on
  30 parent pages — a line the binary never prints.
- The generated CLI index publishes the Global Flags block; the binary
  prints it on every --help but no page carried it.
- docsmint neutralizes prose lines that Mintlify's prebuild would dedent
  into MDX ESM blocks (a line beginning 'export'/'import' outside a
  fence gets wrapped in a code span). This is why /cli-reference/mail
  returned HTTP 200 while rendering only a parse-error placeholder —
  reproduced, fixed, and covered by a planted-hazard test; fenced
  content stays verbatim.

Also rewords check-cli-docs-drift.sh's early-PASS message, which
asserted 'the supplied binary differs' as the cause when uncommitted
local edits produce the same result.

All 113 pages + docs/CLI_REFERENCE.md regenerated through the pipeline;
TDD'd in cmd/bd/help_all_test.go and tools/docsmint/docsmint_test.go.
Verified live: /cli-reference/mail renders real content, zero parse
errors; generate-cli-docs.sh --check, docsync, freshness, and
mint broken-links all pass on this tree.
2026-07-12 10:37:17 -07:00
Chris Sells
52aea3f7e3 docs: go all-in on Mintlify — delete the Docusaurus site and its machinery
Per the decision record: no parallel run and no cutover PR. Removes
website/ (including the four versioned doc snapshots — current-only
versioning; snapshots remain in git history), deploy-docs.yml,
generate-llms-full.sh (Mintlify serves llms.txt natively), the ci-website
package gate (Makefile target, scripts/ci/website.sh, the package-website
jobs and their aggregate-gate entries in pr.yml/main.yml/release.yml, the
ci-measurements suite, detect-package-gates wiring), and the release-time
docs snapshot flow (snapshot-release-docs.sh, check-docs-version.sh, their
call sites in update-versions.sh/check-versions.sh, RELEASING.md section
rewritten for the Mintlify reality).

Drift machinery now covers exactly the Mintlify artifacts:
check-cli-docs-drift.sh pathspecs/fingerprint, docs-autofix-push.sh
allowlist, and check-doc-flags.sh scans drop the website/llms paths. The
beads-docs skill's generated-content and verification sections updated to
the bd-generic + docsmint pipeline.

NOTE for merge time: if branch protection requires the 'Package Gate
(website)' check by name (rather than the aggregate gate), drop it from
the required checks or PRs will wait forever.

Gates: docsync PASS, generate-cli-docs.sh --check PASS, freshness PASS.
2026-07-12 10:37:17 -07:00
Chris Sells
73a4d95fbc docs(emitter): bd emits generic Markdown only; Mintlify targeting moves to tools/docsmint
Per the decision record (engdocs/decisions/2026-07-10-mintlify-docs-overhaul.md):
bd's doc generator is vendor-neutral. `bd help --docs-root` now writes
docs/CLI_REFERENCE.md plus a generic per-command tree (title/description
frontmatter, portable CommonMark, HTML-comment markers) into build/cli-docs/
— uncommitted staging, gitignored. All site-generator-specific output leaves
the binary: Docusaurus frontmatter, sidebarPositionMap, the website/ writers,
the Mintlify writers, the docs.json nav splice, and the --docs-version
snapshot flag are gone from cmd/bd.

tools/docsmint (Go, unit-tested) is the Mintlify post-processor: it turns
the staging tree into the committed docs/cli-reference/ pages (JSX comment
markers, extensionless route links) and splices the CLI Reference pages
array in docs/docs.json. scripts/generate-cli-docs.sh runs both stages;
--check seeds the committed artifacts into a scratch root, regenerates, and
diffs, exactly as before.

Regenerated output: the 113 pages are byte-identical through the new
pipeline; docs.json nav ordering is now plain filename sort;
docs/CLI_REFERENCE.md drops the removed --docs-version flag.

`bd help --doc <cmd>` stdout is likewise generic now (no Docusaurus
id/slug/sidebar_position).
2026-07-12 10:37:07 -07:00
Chris Sells
5c2fe0d496 docs: loss-check merge legacy docs/ originals into the site pages
Sixteen same-ground pairs reconciled: the ported website page won and every
fact living only in the older docs/ original was merged in before deleting
it (INSTALLING, SETUP, ARCHITECTURE, CONFIG, GIT_INTEGRATION, ADVANCED,
TROUBLESHOOTING, FAQ, MOLECULES, ROUTING, MULTI_REPO_AGENTS, and the five
agent-integration guides). The two divergent FAQs are now one canonical
FAQ with the comparison-first framing leading and pre-1.0 version gating
removed.

The routing pair inverted: the ported page described a routing system that
does not exist (bd routes, bd hydrate, .beads/routes.jsonl); the page was
rebuilt from the accurate sources (ROUTING.md + MULTI_REPO_AGENTS.md)
against the real surface (routing.mode config, role detection,
repos.primary/additional hydration, --repo override).

Every claim was verified against the Go source or the generated CLI
reference. Phantom surfaces removed across the corpus: bd pin/hook/pour/
wisp/routes/hydrate/reserve/lock/merge/stats, bd hooks status, bd vc
conflicts/log, bd dolt set shared-server, fictional beads_* MCP tool names
(real names taken from beads-mcp server.py), import.orphan_handling and
export.error_policy config keys, database.wal_mode/cache_size, stale
.beads/dolt/sql-server.log paths, and SQLite-era file paths. Notable
corrections: min_hash_length default is 3, .beads/metadata.json is
git-tracked, tombstone status is removed, wisps live in the main database
under an ephemeral flag.

Freshness markers moved with their content (configuration, ide-setup) and
the freshness gate + DOC_INVENTORY ledger now point at the new locations.
Load-bearing anchors preserved: #branchless-workflows-jujutsu--jj,
#circuit-breaker-server-appears-down-failing-fast, #extensible-database,
#multi-repo-hydration. Pointer stubs remain at docs/{SETUP,INSTALLING,
GIT_INTEGRATION}.md because released binaries print those paths
(allowlisted in the docsync guard).

Gates: go test ./test/docsync PASS; generate-cli-docs.sh --check PASS;
check-doc-freshness.sh PASS; mint broken-links: no broken links; mint dev
serves all sections (11 routes spot-checked at 200).
2026-07-12 10:37:07 -07:00
Chris Sells
3198395a32 docs(ci): add Mintlify preview, docsync guard, and broken-links workflow
- mint.sh + make docs-dev: local preview of the docs/ Mintlify site
  (Node 22 handling modeled on the Gas City wrapper).
- test/docsync: Go guard keeping docs/ and docs.json in exact
  correspondence — every nav entry has a file, every page is in the nav
  (tiny stub allowlist for paths released bd binaries print), published
  pages use root-relative extensionless links that resolve, and engdocs/
  plus curated root markdown keep exact GitHub-style paths. Wired into
  make check-docs.
- .github/workflows/docs-mintlify.yml: docsync + baseline-aware
  mint broken-links check on PRs touching docs/**.
- Retire scripts/sync-website-docs.sh and its mirror-drift gate: the six
  mirrored files now live once in the Mintlify site; the committed
  Docusaurus mirrors stay frozen so the parallel-run site keeps building.
- Repoint inbound links to moved docs across README, guides, examples,
  npm-package, plugin resources, and Go comments; add docs/SYNC_CONCEPTS.md
  pointer stub (bd's agent templates print that path). Runtime strings in
  cmd/bd (prime.go, store_factory_nocgo.go, init_git_hooks.go) still print
  old paths — covered by stubs, flagged in the PR.
2026-07-12 10:37:07 -07:00
Chris Sells
4ea6fa3ac0 docs(emitter): emit Mintlify CLI reference and generated docs.json nav
bd help --docs-root now additionally writes docs/cli-reference/ in Mintlify
form (frontmatter title as H1, extensionless root-relative links, JSX
auto-generated markers since Mintlify parses .md through MDX) and rewrites
the CLI Reference pages array inside docs/docs.json via a textual splice
that leaves the rest of the file byte-identical — the ~113 nav entries are
never hand-maintained. The Docusaurus outputs (docs/CLI_REFERENCE.md,
website/docs/cli-reference/, versioned snapshots) are unchanged.

generate-cli-docs.sh --check seeds docs/docs.json and docs/cli-reference/
into its scratch root and diffs both; check-cli-docs-drift.sh adds them to
GEN_PATHSPECS and the regeneration fingerprint; docs-autofix-push.sh
allowlists the new generated paths; check-doc-flags.sh scans the new
docs/ subdirectories and requires command coverage in docs/cli-reference/.

Includes the initial docs/docs.json scaffold (nav mirrors the Docusaurus
sidebar IA) and the generated docs/cli-reference/ pages.

TDD: new emitter behavior covered in cmd/bd/help_all_test.go
(Mintlify page format, nav splice correctness + idempotence, bare-root
generation without docs.json).
2026-07-12 10:37:07 -07:00
Chris Sells
a86a2c5dfc docs: port user docs into the Mintlify site tree under docs/
Port the website's current (non-versioned) authored pages into docs/ as
Mintlify pages: frontmatter title as H1 (body H1s removed), Docusaurus
admonitions converted to Mintlify components, root-relative extensionless
links, mermaid fences kept as-is. website/docs/** is untouched and keeps
serving the Docusaurus site until cutover.

git-mv the user-facing docs/ originals with no website twin into the new
IA (core-concepts/, architecture/, workflows/, recovery/, multi-agent/,
integrations/, reference/), converted to Mintlify form. The six files
previously mirrored by sync-website-docs.sh now live once in the site.
ERROR_HANDLING.md and CONTRIBUTOR_NAMESPACE_ISOLATION.md move to engdocs/
(contributor pattern guide and design doc per content review).

Pointer stubs remain at docs/{DOLT,STORAGE-BACKENDS,PLUGIN,RECOVERY,
JSON_SCHEMA}.md because released bd binaries print those paths; QUICKSTART
and DOLT-BACKEND stay as existing pointer stubs.

Corrected while porting: the intro's storage diagram claimed data lives at
.beads/dolt/; embedded mode (the default) uses .beads/embeddeddolt/
(internal/storage/embeddeddolt/store.go), server mode uses .beads/dolt/.
2026-07-12 10:37:07 -07:00
Chris Sells
0e017821f4 docs: move internal/maintainer docs from docs/ to engdocs/
Relocate contributor- and maintainer-facing material out of docs/ so the
directory can become the user-facing Mintlify site. Moves preserve subpaths
(adr/, design/, staged-for-removal/ intact). messaging.md, UI_PHILOSOPHY.md,
and AGENT_SIGNING.md move per their DOC_INVENTORY dispositions (design doc,
design philosophy, maintainer convention).

Inbound links updated repo-wide: root guides, CONTRIBUTING, AGENTS,
workflows, lint config, scripts (check-doc-freshness now resolves the
inventory at engdocs/DOC_INVENTORY.md), Go comments, and the synced website
metadata page (regenerated via sync-website-docs.sh + generate-llms-full.sh).

The emitted help text in cmd/bd/init_safety_help.go still references
docs/adr/0002-init-safety-invariants.md; changing it would change bd's
runtime output, which is out of scope for this branch (flagged in the PR).
2026-07-12 10:37:07 -07:00
Eddie the Engineer
c79bb32bca fix(backends): maintainer fixup — CI repair + adoption review fixes
Maintainer fixup for the choose-your-own-storage-backend PR, amended across the
pre-review CI-repair and the review-loop apply-fixes iterations so contributor
commits stay intact under one maintainer commit.

CI repair (pre-review):
- default.nix: bump vendorHash for the added Postgres/MySQL driver deps.
- conformance.yml: provision the Postgres service DB with a C-locale cluster so
  text ORDER BY sorts by code point, matching Dolt's utf8mb4_0900_bin oracle.

Review fixes — blockers:
- uow/doltserver_tx: on a non-serialization DOLT_COMMIT failure, ROLLBACK on the
  pinned session before releasing it, and poison (pool-discard via driver.ErrBadConn)
  the connection when the rollback also fails, so a later borrower can't inherit an
  open transaction and implicitly commit orphaned writes. Satisfies the existing
  doltserver_tx_test rollback/poison cases.
- sqlitedialect: translate GREATEST/LEAST to SQLite scalar max/min so the
  ReconcileChildCounters upsert (GREATEST(last_child, ?)) no longer aborts creation
  or import of hierarchical dotted issue IDs (e.g. x-1.2) on the SQLite backend.

Review fixes — majors:
- conformance: add a portable dotted hierarchical child-ID scenario (x-1, x-1.1,
  x-1.2, x-1.2.1) exercising the ReconcileChildCounters GREATEST path on every
  backend — the coverage hole that let the SQLite blocker ship green. Verified
  against the embedded-Dolt oracle and the real SQLite driver.
- postgres: warn at provision/open when the database collation is not code-point
  ordered (C/POSIX/C.*), since text ORDER BY otherwise diverges from Dolt silently.
- sqlite: pin the pool to one connection and add busy_timeout + WAL to the DSN so
  overlapping operations wait instead of surfacing a raw "database is locked"
  (matches the Dolt backend's single-connection posture). Adds a concurrency guard.
- cmd/bd: gate explicit `bd dolt commit` / `bd vc commit` with a typed
  storage.ErrUnsupported on non-Dolt backends instead of the no-op Commit printing
  "Committed."; internal opportunistic auto-commit is untouched.

Review fixes — attempt 2 (stale init backend-flag test contract):
- cmd/bd/init_test, init_embedded_test: the init backend-flag tests still asserted
  the pre-#3151 one-backend contract (--backend=sqlite must fail "SQLite backend
  has been removed"; --backend=postgres is an "unknown backend"), so
  `go test ./cmd/bd -run '^TestInitBackendFlag$'` failed in a clean environment now
  that sqlite/postgres/mysql are supported pluggable backends. Rewrite
  TestInitBackendFlag and the cgo TestEmbeddedInit sibling (rejected_backends →
  sql_backend_flags) for the new contract: --backend=sqlite initializes and
  persists backend=sqlite; --backend=postgres is recognized and fails only on
  missing connection config (not "unknown backend"); a genuinely unknown backend
  is still rejected. Pin BEADS_DIR and scrub BEADS_*/BD_* so the subtests are
  hermetic against an ambient .beads or Dolt server.

Cleanups (non-gating):
- cmd/bd/init: extract checkExistingSQLBackend so the added SQL-backend branches
  don't inflate checkExistingBeadsDataAt's complexity, and load config once.
- configfile: correct the stale "Deprecated: always dolt" Backend field comment.

Review fixes -- attempt 3 (SQLite datetime parity blocker + conformance/rewrite
hardening of the storage-parity gate):
- sqlite/dsn: add _time_format=datetime. Without it modernc binds time.Time via
  t.String() ("2026-07-09 12:34:56.123 +0000 UTC"), a shape SQLite's date functions
  cannot parse, so the counts mega-query's DATE_FORMAT(created_at,...) (strftime on
  SQLite) returns NULL and deps_json exposes a zero created_at while Dolt emits the
  real timestamp. datetime stores whole-second "2026-07-09 12:34:56", matching
  Dolt/MySQL datetime(0). Verified end-to-end against the real modernc driver.
- conformance: add ReadyWorkDepCreatedAtParity -- imports a dependency with a known
  created_at (the bd import / CreateIssuesWithFullOptions bound-time.Time path) and
  asserts the counts-rendered deps_json created_at equals it, closing the blind spot
  that let the SQLite blocker ship green (the counts tests only checked DependencyCount).
  Proven to fail on the pre-fix DSN and pass on both the fixed SQLite backend and the
  embedded-Dolt oracle.
- conformance: add ClaimReadyIssueConcurrentExclusivity -- races 24 workers over 8
  ready issues and asserts each is claimed exactly once, exercising the SQL backends'
  multi-writer claim property that was previously only asserted, never raced. Gated to
  the multi-connection dialects (postgres/mysql) via DialectName; SQLite (single-conn
  pool) and embedded-Dolt skip.
- conformance.sh: run the embedded-Dolt reference conformance with
  BEADS_TEST_EMBEDDED_DOLT=1 and fail loudly if it skips. It self-skips without the env
  var, so the parity gate was reporting success while never running the reference oracle.
- mysqldialect: drive the is_blocked-recompute error-1093 rewrite test from the real
  issueops templates (new issueops.SelfReferentialRecomputeTemplates) instead of a
  hand-copied fixture, so template drift fails a non-gated test rather than only the
  env-gated MySQL conformance suite. All four mark/unmark x issue/wisp templates
  verified rewritten into the materialized derived-table form.

Note: the review re-flagged TestInitBackendFlag as failing, but it passes in a clean run
(plain and gms_pure_go, matching the scorecard's own rerun); the only cmd/bd failure in
this sandbox is an unrelated pre-existing embedded-Dolt/CGO test not in this PR's diff.

Review fixes -- attempt 4 (conformance-gate self-brick + all-wisps inline-dependency
drop blockers; Postgres error-log password-leak major):
- conformance.sh: anchor the embedded-Dolt reference skip/pass guards to top-level
  result lines (^--- SKIP/PASS: TestConformance). The attempt-3 loud-fail guard used
  an unanchored grep, so the indented, legitimate subtest skip of
  ClaimReadyIssueConcurrentExclusivity (embedded-Dolt is single-writer) matched it and
  false-failed the entire storage-parity gate on every run even when the top-level
  suite passed. Whole-suite-skip protection is preserved.
- sqlkit/portable + embeddeddolt/create_issue: the all-wisps arm looped the
  single-issue issueops.CreateIssueInTx, which runs no batch dependency-persist pass,
  so an all-wisps batch carrying inline dependencies (e.g. importing no-history beads
  with inter-bead edges -- export DOES include no-history beads, GH#2619) silently
  dropped every edge (not even reported via OnSkippedDependency) and, one transaction
  per issue, forfeited batch atomicity. Both now mark ephemeral, then run the whole
  batch in ONE transaction through issueops.CreateIssuesInTx, matching the server-mode
  DoltStore all-wisps arm. The embedded-Dolt reference oracle had the same latent drop,
  so the oracle and the new SQL backends were corrected together to keep conformance
  parity (the finding compared sqlkit against the server-mode DoltStore, which already
  persisted these edges).
- conformance: add CreateAllWispsInlineDependencies -- an all-wisp batch with one valid
  inter-wisp edge and one unresolvable edge; asserts the valid edge persists and the
  unresolvable one is reported via OnSkippedDependency. Fails on the pre-fix per-issue
  loop, passes on the batch path; verified on SQLite and the embedded-Dolt oracle, with
  a full embedded-Dolt conformance re-run clean.
- pgdialect: scrub query-param/sslpassword secrets from pgx ParseConfig errors in Open
  and OpenRaw (new ScrubDSNError). pgx's ParseConfigError redacts only URL userinfo, so
  a postgres://u@h/db?password=SECRET DSN (the shape bd init connects with) leaked the
  cleartext secret through the wrapped parse/TLS-config error into logs. Connect-time
  errors were verified not to carry the password. Adds redact_error_test coverage for
  the query-param, sslpassword, userinfo, and libpq-keyword shapes plus the real
  Open/OpenRaw parse-failure path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 17:44:37 +00:00
Test User
1fc38ba778 feat: choose your own storage backend (Postgres / MySQL / SQLite) — rebased on main
Rebase of PR #4601 (feat/choose-your-own-backend) onto current origin/main,
squashed to a single commit because the original 73-commit history built and then
removed the uowStore derisk spike incrementally, which does not replay cleanly
over the 125 commits main has since advanced (notably #4679, the ready-work
counts fix, which merged to main and is now inherited rather than re-applied).
The full pre-rebase history remains on the original branch tip for reference.

Contents (multi-backend feature + the six review follow-ups):
- bd init --backend=postgres|mysql|sqlite provisioning behind the Dolt-parity
  seam; the three SQL backends share *sqlkit.Store over pluggable dialects and
  carry a generated typed-unsupported shell for the non-core surface.
- Passwords in init DSNs are redacted before persisting to metadata.json (driver
  parser + re-parse verification), with a negative-test matrix proving no secret
  survives (pgdialect + mysql).
- The uowStore derisk spike and its report artifacts are removed; the shell
  generator now lives in internal/storage/unsupportedgen (neutral name).
- Compile-time guard `var _ storage.DoltStorage = (*Store)(nil)` on each backend
  plus a regen-idempotence test keep the generated skip lists honest.
- sqlkit exposes CountReadyWork so every SQL backend satisfies
  storage.ReadyWorkCounter and inherits the shared page-pushed ready-counts path.
- Commercial/product names scrubbed from the feature's own code for vendor
  neutrality.

Rebase reconciliations: adopted main's metadata-load hardening in the store
factories alongside the backend dispatch; combined the config interface
(GetMetadata + ReconcileVersion) and the conformance registrations (ReadyCounts +
Claim/lease); took main's versions of the shared-infra concurrency (uow
doltserver_tx) and delete-cascade (domain/issue_delete) evolutions; implemented
the interface's new UnclaimIssue on sqlkit (delegating to issueops) and added it
to the skip lists.

Verified: go build (cgo + CGO_ENABLED=0) and go vet clean; unit tests
(sqlbuild/issueops/pgdialect/mysql/unsupportedgen) and the SQLite conformance
suite green (RunAll, incl. ReadyCounts equivalence and Claim/lease on the real
sqlkit backend).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 17:44:37 +00:00