Commit graph

262 commits

Author SHA1 Message Date
Karel Bourgois
a4afdef458
fix(scripts): update-nix-vendorhash writes a stray default.nix'' on BSD sed (#6096)
`SED_INPLACE="sed -i ''"` cannot carry an empty argument. After word-splitting,
the two quote characters survive as a literal two-character argument, so BSD sed
reads '' as the backup SUFFIX rather than as "no suffix" and writes
`default.nix''` beside the real file on every macOS run.

Reproduction (bash, macOS):

    echo 'vendorHash = "old";' > default.nix
    SED_INPLACE="sed -i ''"
    $SED_INPLACE "s|old|new|" default.nix
    ls
    # default.nix   default.nix''

The stray is not ignored by .gitignore and carries the placeholder
sha256-AAAA... hash the script sets in step 1, so a `git add -A` after running
the script commits a file whose vendorHash can never validate. I hit exactly
that on a resync.

Fixed by branching at the call site in a function, where an empty argument is
passed as a real argument instead of surviving word-splitting as text. Same
detection, same two call sites, no behavior change on GNU sed.
2026-09-02 17:33:58 -07:00
postoso
4c4be78b5c
fix(scripts): make check-doc-flags portable to BSD grep and fail loud on grep errors (#6026)
Co-authored-by: postoso <postoso@users.noreply.github.com>
2026-09-02 17:25:53 -07:00
Bee
fa53c7e2a6 ci(main): bound the Test matrix's per-package deadline (wy-5b5fbl)
Both legs of main.yml's `test` matrix run every package in one `./...`
invocation with no `-timeout`, so they rode go test's DEFAULT 10m
per-package deadline. cmd/bd has outgrown it: scripts/test.sh already
documents the floor for local runs (~1090s of test time / ~18min wall for
cmd/bd, which is why TEST_TIMEOUT defaults to 25m there), and on
2026-09-01 main run 33497003321 failed twice with
`panic: test timed out after 10m0s` in cmd/bd AND, as collateral in the
same parallel coverage pass, internal/storage/uow — with DIFFERENT tests
only 1-12s in flight on each attempt (a package-wide crawl on the 2-core
coverage-instrumented runner, not a hang), while a rerun of the identical
tree passed.

Give both legs the repo's own 25m per-package deadline and keep it inside
one finite job budget (`timeout-minutes: 45`), the shape
test-embedded-conformance already documents: the go deadline fires first
and prints goroutine stacks, so a REAL hang stays diagnosable instead of
being cut off by a bare job kill.

The ubuntu leg's `matrix.test-flags` is dead config — its coverage step
hardcodes its own flags — so the ubuntu deadline goes on the gotestsum
line and the macOS one on `test-flags`, the only leg that reads it.
scripts/ci_workflow_test.go pins that macOS string exactly, so the
ratchet moves with it (adversarial review F1: without this the diff is a
deterministic red on main and on every PR).

Verified: go test -count=1 ./scripts rc=0 (the ci_workflow ratchet over
the patched main.yml); actionlint .github/workflows/main.yml rc=0.
2026-09-01 05:23:00 -07:00
Julian Knutsen
d530cddfa6
ci: add advisory cyclomatic complexity reporting (#6081)
* ci: add advisory cyclomatic complexity reporting (bd-7g21)

Add a pinned gocyclo report, checked-in baseline, stable-key PR diff, and opt-in guard for shipped Go code. Run the report on every PR as a non-required artifact while maintainers measure signal quality; preserve existing complexity as grandfathered debt.

Agent-Signature: unknown-model/unknown-reasoning

* test(protocol): refresh dependency removal corpus
2026-08-30 19:05:17 -07:00
Julian Knutsen
e7894732e0
fix(ci): make a failed pinned-dolt download actually fail the job (#5935)
* fix(ci): make a failed pinned-dolt download actually fail the job

scripts/ci/install-dolt.sh retried the download with

    if curl -fsSL -o "$workdir/$asset" "$url"; then
      break
    fi
    status=$?

`status=$?` there reads the status of the *if compound*, not of curl. A
compound whose condition fails and which has no else branch is itself
successful, so `status` was 0 on every failed attempt, and the
retries-exhausted branch ran `exit 0`. A download that never succeeded
reported success.

That defeats the entire point of the script. The step goes green, the
version-drift guard at the bottom never runs, and the job proceeds on
whatever `dolt` the runner image already had on PATH -- the unpinned
binary the pin exists to keep out, and the one that made
TestFreshBootstrapHealIncarnation flake when releases/latest moved to
2.3.x.

Capture curl's status directly instead, and include it in the
retries-exhausted message so the log says which failure it was.

Guarded by TestInstallDoltPropagatesDownloadFailure, which runs the real
script against stub curl/sleep/tar/sudo/dolt/uname on PATH. Against the
unfixed script exactly one subtest reddens ("exhausted retries": exit 0,
want 42); "immediate success", "eventual success" and "installed version
drifts" stay green, so the harness demonstrably observes success and the
new signal is specific to this bug rather than to everything failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(ci): pin what install-dolt.sh actually downloads and installs

The review council mutated the script and found two changes the new test
waved through: swapping the pinned release URL for releases/latest, and
installing mode 0644 into /opt/dolt instead of 0755 into /usr/local/bin.
Both are exactly the drift this script exists to prevent, and both left
all four subtests green because the test only counted calls. The sibling
pull_dolt_image_test.go already asserts its docker argv; this one now
does the same for curl and sudo.

The council also flagged the drift guard itself: `[[ $installed !=
*"$version"* ]]` accepts any version containing the pin as a substring,
so 12.2.0 and 2.2.0-rc1 both pass. Compare the version token exactly.

Red before green, by real on-disk swaps (a -overlay cannot reach a file
the test execs):

  releases/latest URL          all 5 subtests red (was: all green)
  install -m 0644 /opt/dolt    4 of 5 red -- exhausted_retries stays
                               green because it never reaches sudo
  substring drift guard        only installed_version_merely_contains
                               _the_pin reddens

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 16:58:46 -07:00
Julian Knutsen
dbb6df7e9f
fix(schema): CLI bundle never got migration 0065's wisp_comments widening (ga-61ruw) (#5910)
* fix(schema): CLI bundle never got migration 0065's wisp_comments widening (ga-61ruw)

Second and last live instance of the class #5903 fixed for 0060: the Dolt CLI
batch path silently no-ops a PREPARE'd ALTER TABLE while EXECUTE reports
success (dolthub/dolt#11345, fixed in 2.3.0 — but the CLI is pinned to 2.2.0
by #5894 because 2.3.0 broke DOLT_RESET). So 0065's wisp_comments.text
TEXT->LONGTEXT widening never reached the fresh-schema bundle.

Measured, not argued. Applying AllMigrationsSQL() through dolt 2.1.8, 2.2.0
(the pinned CLI) and 2.3.1 and diffing information_schema — with wisp_ tables
INCLUDED, unlike the parity oracle — produced exactly one difference on main
@825cd4c9:

    column|wisp_comments|004|text|text|NO      (2.1.8, 2.2.0)
    column|wisp_comments|004|text|longtext|NO  (2.3.1)

After this change all three agree on all 504 lines. 2.1.8 and 2.2.0 are
byte-identical to each other, so the host CLI is a faithful stand-in for the
pinned one.

WHY THE OBVIOUS FIX FAILED, AND WHAT REPLACES IT. A bare direct MODIFY was
tried before and backed out: it reproduces as
`error ... ALTER TABLE wisp_comments MODIFY COLUMN text LONGTEXT NOT NULL:
table not found: wisp_comments` inside
TestFullChainFromPreWispsAndMissingDependenciesIDConvergesThroughDoltCLI,
which replays the substitution over a database that never synced the
clone-local wisp tables (#4695/#4176) and whose 0047 repair recreates only
wisps and wisp_dependencies.

The substitution surface now says so out loud. cliSubstituteAssumesWispTables
names the substitutes that presume the wisp tables exist — 0053, which the
full-chain test already special-cased by version number, and now 0065. Replay
callers use the frozen source text, whose own PREPARE guard no-ops correctly
because an INFORMATION_SCHEMA probe for a missing table yields NULL and
IF(NULL = 1, ...) takes the 'SELECT 1' branch.
TestMigration0065NoopsWithoutWispCommentsThroughDoltCLI pins that, with the
direct substitute failing on the same database as its control.

CLOSING THE CLASS. 0060 and 0065 were both found by accident, six days apart,
because nothing looks for this shape: check-migration-hygiene.sh check E flags
prepared DML, not prepared ALTER, and the parity oracle excludes wisp_ tables
by name. internal/storage/schema/cli_prepared_ddl.go adds
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified: it scans the
substituted bundle text for prepared ALTER TABLE and requires each hit to have
either a direct-DDL override or an entry in preparedALTERSafeOnFreshBundle
with a written reason. No Dolt binary is involved, so a CLI upgrade cannot
mask it the way it masked the oracle.

Mutation-verified end to end. A synthetic 0066 with a prepared ALTER and no
override makes the guard fail with a message naming the file and statement;
the same bundle really does drift (issues.storage_class varchar(16) on 2.2.0
vs varchar(32) on 2.3.1); adding an override makes it pass; removing the
migration returns green. Removing 0065's override reds both the new guard and
the #5903-style string assertion.

The seven grandfathered entries were measured the same way — build the bundle
truncated to just before migration N, apply N's frozen text on 2.3.1 where
prepared DDL always executes, diff. Six change nothing (their guards do not
fire on a fresh database); 0050's guard does fire, but ALTER COLUMN ... DROP
DEFAULT is one of the forms that executes on 2.2.0. 0065's own text moves that
diff, which is the control proving the six empty diffs mean something.

Probed subcommand split on 2.1.8, with a direct ALTER as the must-land
control: ADD COLUMN, MODIFY COLUMN, DROP COLUMN and ADD PRIMARY KEY vanish;
DROP DEFAULT, RENAME COLUMN, ADD/DROP INDEX, ADD CONSTRAINT, CREATE INDEX,
RENAME TABLE and CALL all execute. The guard flags every prepared ALTER TABLE
anyway — over-flagging costs one justified inventory entry and avoids guessing
about forms nobody probed.

NOT WIDENING THE PARITY ORACLE, and here is the evidence rather than a
shrug. AllMigrationsSQL() walks the main series only, and there is no
ignored-series bundle to pair with it. Applying bundle-only vs
bundle-plus-ignored-series through 2.3.1, the ignored plane owns
wisps.is_blocked (+ two indexes), the dropped uuid() defaults on
wisp_comments/wisp_dependencies/wisp_events .id, wisp_events.old_value and
new_value as LONGTEXT, leases.granted_node and ignored_schema_migrations
itself. Every one would read as a spurious "only in runtime" line, so covering
the ephemeral plane means building an ignored-series bundle, not deleting a
predicate. That analysis is now recorded in the oracle's own comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(dolt): stop the argv-limit test depending on the migration bundle's size

TestRunDoltSQLHandlesLargeScript asserted that schema.AllMigrationsSQL()
exceeds Linux's MAX_ARG_STRLEN (131072 bytes), so that the script it pipes
really does exercise the argv path. That precondition is the right idea and
the wrong measurement: the bundle's size is an accident of migration content.
Registering 0065's direct-DDL override in the previous commit REPLACED a
~2.4KB guarded PREPARE block with a 62-byte statement and took the bundle from
132601 to 130281 bytes, so the test went red on Linux and macOS for a reason
with nothing to do with argv. Every future override shrinks it again; every
future migration grows it back.

Build a script that is definitely oversized instead of hoping. padPastArgvLimit
appends SELECT 1; statements — which the CLI still has to parse — until the
script clears the limit, and leaves an already-oversized script alone. The
precondition assertion stays, now covering the padded script, so a broken
helper cannot quietly leave the test passing on a script that would have fit
in argv.

TestPadPastArgvLimit is the control for that helper, across empty, just-under,
exactly-at and already-over inputs. Mutation-verified: stubbing the padding out
reds it on every under-limit case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 23:38:48 -07:00
Julian Knutsen
825cd4c937
fix(ci): pin the Dolt CLI — 2.3.0 broke DOLT_RESET on fresh databases (ga-m0dlk) (#5894)
* fix(dolt): git-remote tests need a sql-server that shares their filesystem

The four git-remote tests that push through the store API fail on every run
of the new test-server-storage-full lane (#5836), which is what has the
merge pipeline stalled: shards 1, 9, 10 and 15 all die with

  the remote: origin 'git+file:///tmp/embedded-git-remote-test-*/remote.git'
  could not be accessed; fatal: ... does not appear to be a git repository

setupEmbeddedGitRemote dates from when this package had an in-process
embedded engine. store.go now hardcodes serverMode:true, so New() connects
to the suite's shared Dolt server — which TestMain runs in a Docker
container whose only mount is the image's own /var/lib/dolt. CALL DOLT_PUSH
executes inside that container, so the host bare repo the test just created
does not exist for it, and neither does the store's own Path (which is why
TestGitRemotePushSkipsUserPrePushHook cannot find its cache mirror and
TestGitRemoteSyncRoundTrip's clone comes up empty). The dolt credential
hints in the log are generic advice, not a credential problem — nothing
here needs a credential.

Give these tests their own local dolt sql-server rooted at the store's data
directory, the arrangement TestGitRemoteExternalServerRouting,
TestSQLRemotePersistsAcrossExternalServerRestart and
TestCredentialCLIRoutingE2E already use. The server is spawned with
doltserver.ServerSpawnEnv(), the same environment bd gives the server it
starts: a `dolt` CLI command run against a directory a sql-server already
serves is proxied to that server, so the git subprocess is spawned by the
server and the CLI-side guards (git tracing, core.hooksPath) only take
effect if the server carries them too.

Two consequences fall out and are fixed here:

  - The setup now takes a genesis CommitAll. Commit() deliberately skips
    config (GH#2455), so issue_prefix stayed dirty forever: Pull() refuses
    to auto-commit a dirty internal config key, and a peer cloning the
    database got no prefix at all. The CLI sibling setupGitRemote has
    always done this with DOLT_COMMIT('-Am', 'Genesis: schema and config').

  - The gate is testutil.RequireDoltBinary rather than skipIfNoDolt: these
    tests no longer need the shared container, only dolt and git, so
    requiring the container would be a skip for no reason.

Setup asserts <Path>/<db>/.dolt exists on this filesystem, so if the store
ever drifts back onto a containerized server these tests fail loudly
instead of quietly testing nothing.

Verified against the real shard script on a live Dolt server: shard 1 goes
from 1 failure to 0, shard 9 from 1 to 0.

* test(dolt): pin what TestCreateIssueAfterPull's Pull actually delivered

Shard 1 still fails in CI after the topology fix, now at the last assertion:
'GetIssue(ai-clone-001) failed: not found'. Deterministic across two CI runs,
not reproducible locally on dolt 2.1.8 or 2.3.1 (the whole shard is green
here). The end-of-test assertion cannot distinguish 'Pull() reported success
but merged nothing' from 'Pull() delivered the row and the post-pull
CreateIssue dropped it', which are different bugs in different places.

Assert immediately after Pull, before the post-pull write, so the CI failure
names which one it is.

* fix(ci): pin the Dolt CLI — 2.3.0 broke DOLT_RESET on fresh databases

Test (Server Dolt Full Suite 12/16) has been failing on a coin flip with

  TestFreshBootstrapHealIncarnation/exact_creator_heals_its_interrupted_bootstrap
  creator-authorized migration: schema migration: pending schema migrations
  alter pre-existing dirty tables: issues; run 'bd dolt commit' ... (#4566)
      schema: fresh-bootstrap reset: Error 1105 (HY000): context canceled

That second line is the whole story, and it is not a beads bug. The heal was
authorized and its capability consumed; what failed is the
CALL DOLT_RESET('--hard') in schema.MigrateUpWithLock. Dolt 2.3.0 regressed
that procedure: roughly one freshly created database in twenty comes up with
it permanently unusable, answering "Error 1105 (HY000): context canceled" to
every session, from any connection, for the life of the server process. It is
not transient and not load-related — a beads-free probe that only does
CREATE DATABASE followed immediately by the procedure reproduces it:

  dolt 2.1.8   0/40 databases broken
  dolt 2.2.0   0/60
  dolt 2.3.0   3/60
  dolt 2.3.1   3/100

Every workflow installed the CLI by piping dolthub/dolt's releases/latest
install.sh, so the binary under test changed the moment 2.3.x shipped —
while internal/testutil/testdoltcommon.go had the sql-server container pinned
at 2.2.0 the whole time. Server-mode tests run both halves at once against the
same databases, so the suite was exercising a Dolt pair no release ever
shipped. Pin the CLI to the container's release and keep the two locked
together:

- scripts/ci/install-dolt.sh installs a pinned Dolt, verifies `dolt version`
  on PATH actually resolves to it, and records the per-version measurements
  and the criterion for raising the pin.
- All 25 install sites across six workflows route through it.
- TestWorkflowsInstallPinnedDolt rejects any reintroduction of
  releases/latest; TestPinnedDoltCLIMatchesContainerImage fails when the CLI
  pin, DoltDockerImage and pull-dolt-image.sh disagree.

The test itself is unchanged in what it asserts. It gains one preflight probe:
before exercising the heal it checks that this server can hard-reset the
database it just created, so a broken Dolt is named as such instead of
surfacing as the generic #4566 refusal — which reads like a beads defect and
sends the reader to "run 'bd dolt commit'", advice that cannot work when the
reset the heal needs is the thing that is broken. On a pristine database the
probe is a no-op, and over 60 fresh databases a successful first reset never
broke a later one.

Verification: 48 runs of the test (6 workers x 8) and two full shard-12 runs
on the pinned 2.2.0, zero failures. Same binary on 2.3.1: 3 of 24 runs still
fail, now in 0.04s with the explicit diagnosis instead of 1s in with the
misleading one.

Beyond this test: bd dolt compact, bd dolt flatten and mergesettle's rollback
all issue DOLT_RESET('--hard'), so they are unreliable for real users on any
Dolt 2.3.x server. That needs an upstream report; pinning CI does not fix it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:59:27 -07:00
Jim Wordelman
5941096915
Fix: check-testing-short.sh flags comments and misnames the offending function (round 2) (#5809)
* test(feat): red — Fix: check-testing-short.sh flags comments and misnames the offending function (refs be-0v13)

* feat: green — Fix: check-testing-short.sh flags comments and misnames the offending function (refs be-0v13)

Skip comment-only mentions of testing.Short() before the allowlist
check: read the exact matched line and treat a line whose first
non-whitespace characters are // as non-offending. Existing
func-attribution logic already reports "unknown" for a hit above the
first func in a file, so no change was needed there.

Also fixes two test-fixture bugs found while confirming RED: the RED
test file's own source contained the literal substring testing.Short()
(now built via string concatenation into a shortCall const so the
checker doesn't flag this file when it scans the real repo tree), and
single-file temp-dir fixtures omitted the grep filename prefix
(now every fixture dir gets a companion .go file so find+grep always
sees 2+ filenames, matching real repo-root invocations).

* test(scripts): red — check-testing-short.sh must not attribute a hit between two functions to the preceding one (refs be-r56j)

* fix(scripts): green — resolve testing.Short() hits to the containing function, not the nearest preceding one (refs be-r56j)

* chore: release gate PASS for be-053u-check-testing-short-round2

* ci: retrigger — unrelated Ryuk reaper flake in storage/uow/tracker job

The "Test (storage domain + uow)" job failed on an infrastructure flake
unrelated to this PR's diff. That job runs three package trees in one
go test invocation (.github/workflows/pr.yml:676), so Go runs their test
binaries in parallel and each starts its own Dolt testcontainer. The
binaries share a single testcontainers Ryuk reaper; in this run the
internal/tracker binary reused a sibling's reaper and its handshake was
reset:

  23:12:21 Reaper obtained from Docker for this test session 6a74346c
  23:12:21 Reaper handshake failed: read ack: ... connection reset by peer

With no live connection holding the session open, Ryuk reaped the
session's containers. The tracker package's Dolt container went down
mid-suite and every subsequent store-backed test failed with
"connection refused" on 127.0.0.1:32772.

This is be-2on, root-caused on a different PR (#5339) with the identical
signature. The durable fix is PR #5770 (disable Ryuk on the Dolt
testcontainer steps), which is open and awaiting upstream merge.

This PR touches only scripts/check-testing-short.sh, its test, and a
release-gate record — nothing in internal/tracker or the Dolt path.
Empty commit to re-trigger CI.
2026-08-19 21:01:15 -07:00
Jim Wordelman
30f30ca95b
ci: PR-CI lane runs the full internal/storage/dolt suite against a live Dolt server (#5836)
* test(ci): red — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

* feat(ci): green — gastownhall/beads: no PR-CI lane runs TestBenchDBPurgeDoesNotLeak against a live Dolt server (checkDolt SKIP) (refs be-aiy5)

Add test-server-storage-full as a sibling job to test-server-storage in
pr-risk.yml. build-embedded already compiles every top-level test in
internal/storage/dolt into one binary (embedded-test-binaries); the
existing job only ever runs -test.run '^TestConformance$' against it.
The new job reuses that same artifact and live-server setup with no new
build step, and runs everything else via -test.skip '^TestConformance$'
so previously-stranded tests (TestCreateGuard_*, federation/concurrent
suites, and diff-owned tests such as TestBenchDBPurgeDoesNotLeak on PR
#5792) get a real PASS/FAIL instead of a silent SKIP. Sets
BEADS_TEST_ENV_RUN_DOLT=1 so federation_test.go's own live-server check
fails loudly rather than skipping if this job's setup ever regresses.
Wired into ci-gate: added to needs, CI_GATE_REQUIRED, and the
tier-gated skipped_ok fallback (mirrors test-server-storage's existing
treatment in each).

Also updates two pre-existing meta-tests over pr-risk.yml that pinned
exact snapshots of the file this job necessarily changes:
scripts/ci_capability_selector_test.go's ci-gate authority snapshot,
and scripts/pull_dolt_image_test.go's retry-helper call count
(pr-risk.yml: 2 -> 3, the new job's own Dolt-image-pull step).

* chore: release gate PASS for be-pp7e (PR-CI lane for TestBenchDBPurgeDoesNotLeak)

Deploy gate record for be-pp7e, gating be-aiy5's PR-CI lane addition
(runs TestBenchDBPurgeDoesNotLeak against a live Dolt server instead of
letting checkDolt SKIP it). Review: be-w90n, verdict pass. All 7 gate
criteria PASS; two pre-existing/unrelated failures encountered during
evaluation (be-jy56, be-vf95) were attributed and excluded from the
verdict.

* test(ci): red — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

TestPRRiskGateReachesFullServerDoltStorageSuite now asserts
pr-risk.yml's test-server-storage-full job is a fail-fast:false matrix
over shard 1..16, with the Test step invoking
server-storage-test-shard.sh, instead of pinning the old single-job
unsharded command. Fails against the current workflow, which still
runs all 1126 tests unsharded in one 15m job (be-extn).

* feat(ci): green — shard test-server-storage-full across 16 matrix jobs (refs be-yy25)

Adds .github/scripts/server-storage-test-shard.sh (mirrors
embedded-storage-test-shard.sh: discovers internal/storage/dolt/*_test.go,
excludes TestConformance, manifest-plus-hash-fallback shard assignment) and
its shard manifest .github/scripts/server-storage-test-shards.txt, pinning
TestCloudAuthCLIRouting (~9.5m) alone on shard 1 and spreading the other
known 10-20s-heavy tests across the rest.

pr-risk.yml's test-server-storage-full job becomes a fail-fast:false matrix
over shard 1..16, Test step now runs server-storage-test-shard.sh instead of
the old unsharded single-job command that died at 256/1126 tests after
15m28s with zero failures (job 95616282508).

Documents the new lane in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md alongside the
existing embedded matrix description.

TestPRRiskGateReachesFullServerDoltStorageSuite now passes. Offline shard
partition verified disjoint and complete across all 1125 discovered tests
(BEADS_TEST_SHARD_LIST_ONLY=1 for i=1..16). (be-extn)

* test(feat): red — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* feat: green — Fix: sharded lane runs prebuilt test binary from repo root, breaking ../schema/migrations and ../issueops (6 tests) (refs be-gepv)

* chore: release gate PASS for be-23f3

Extends deploy/be-pp7e-gate (PR gastownhall/beads#5836) with be-gepv's
fix rather than cutting a fresh isolated branch: be-gepv's own bead
text and be-extn's precedent both establish this fix belongs on the
already-open PR. All 7 gate criteria PASS; two repo-wide lint/policy
failures attributed to already-tracked pre-existing issues (be-vf95,
be-jy56), confirmed untouched by this PR lineage.

be-gepv, be-6yo1, be-23f3

* fix(dolt): federation test isolation + panic (be-3c78s)

TestFederationDatabaseIsolation and TestFederationVersionControlAPIs are
federation-prototype tests that predate the suite's shared-server,
branch-per-test isolation architecture (setupTestStore/StartTestBranch)
and were never updated to account for it.

- setupFederationStore hardcoded Database: "beads" for both towns. New()
  always connects in server mode against one shared Dolt server for the
  whole test binary (TestMain), so Path alone doesn't isolate two stores
  — Database does. Both towns silently shared one database, so Alpha
  could read Beta's issues and vice versa. Give each town its own
  database (beads_test_federation_<prefix>, matching the beads_test AD-01
  cleanup-firewall prefix).

- TestFederationDatabaseIsolation's not-found assertions assumed GetIssue
  returns (nil, nil) on a miss. It never does — issueops.GetIssueInTx
  always returns a wrapped storage.ErrNotFound. Switched to
  errors.Is(err, storage.ErrNotFound).

- TestFederationVersionControlAPIs hardcoded Checkout(ctx, "main") to
  return to its "starting" branch, but setupTestStore already checks out
  a private branch (test-<name>-<hex>) — "main" is a branch shared by
  every parallel test in the binary and never has vc-001. That mismatch
  is what produced the nil-pointer panic (GetIssue's error was discarded
  via `_`, and the resulting nil issue was dereferenced). Capture the
  real starting branch via CurrentBranch() right after setupTestStore()
  and check both previously-discarded GetIssue errors before
  dereferencing.

Neither bug is a production security issue: New(), Checkout(), and
GetIssue() all behave exactly as specified. Both bugs are confined to
this prototype test file's own setup/assertions, which hadn't caught up
to the isolation model the rest of the suite already uses correctly.

Verified via `go build`/`go vet`/`go test -c` (all clean) and running the
compiled binary, which now reaches and cleanly SKIPs both tests (no
Docker locally) instead of the previous isolation failure / panic
surfacing at compile-adjacent runtime. Full GREEN confirmation is via
CI, which has a working Dolt/Docker environment.

Refs be-3c78s

* chore: release gate PASS for be-42tlj
2026-08-19 20:39:38 -07:00
Jim Wordelman
6ec78f3a2d
fix: bound and retry apt-get calls in CI to avoid unbounded 15-minute job timeout (#5810)
* test(feat): red — Fix: unbounded apt-get in CI burns the whole 15m job timeout (migration-test.yml, release.yml) (refs be-na2q)

* fix(feat): green — Fix: unbounded apt-get in CI burns the whole 15m job timeout (migration-test.yml, release.yml) (refs be-na2q)

* fix(ci): make apt-get update retry exhaustion deliberate, not a silent fall-through

After three failed `apt-get update` attempts the loop simply ended and the
step went on to `apt-get install` against a stale index, with nothing in the
log saying the index was never refreshed. That contradicted the PR body's
claim that the step "fails fast with a clear error".

Proceeding best-effort is the right behavior -- the runner image's cached
index installs these packages far more often than not, and failing here would
turn a transient mirror outage into a red build -- so make it an explicit,
logged decision instead of a fall-through:

  - track whether any attempt succeeded and emit
    `:⚠️:apt-get update failed after 3 attempts; proceeding with
    cached index` when none did
  - drop "; retrying" from the per-attempt warning, which was wrong on the
    final attempt
  - skip the 10s backoff after the last attempt, where there is nothing left
    to wait for

The workflow tests pin both new strings, so a revert to the silent
fall-through goes red.

* fix(ci): raise apt step timeout-minutes above the retry budget they backstop

The step-level caps were tighter than the inner `timeout` calls they were
meant to backstop, so they reproduced the mid-flight kill this hardening
exists to remove -- just at 6 and 8 minutes instead of 15:

  migration-test.yml  worst case 3x120s + 2x10s + 180s = 560s vs cap 360s
  release.yml         worst case 3x120s + 2x10s + 300s = 680s vs cap 480s

Raise them to 11m and 13m, each ~100s above its own worst case, and record
the arithmetic next to the value so the next edit re-derives it. 11m also
stays under the historical-upgrades job's own 15m budget, so a stuck step
fails as a step rather than cancelling the job with no apt-specific signal.
The goreleaser job sets no job-level timeout-minutes at all, which makes its
step cap the only bound on a stuck apt there.

The workflow tests now assert the relations instead of only pinning the
minute counts:

  - step timeout-minutes must exceed the step's own worst-case budget,
    computed from the attempt count, per-attempt timeout, backoff and
    install timeout that the same table pins against the YAML
  - it must stay below the job's timeout-minutes where the job sets one

Both read the value off the workflow rather than off the table's own
constant, so they stay honest if the YAML drifts.
2026-08-19 11:47:22 -07:00
Julian Knutsen
ed382cbdb8
Pin authenticated historical release catalog (#5100)
* Pin authenticated historical release catalog (bd-9q3.1)

Establish the exact 122-version SumDB-backed source universe and record proxy prereleases, archived repository-only tags, current repository-tag drift, and optional release asset provenance. Offline validation pins the complete reviewed catalog identity.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Simplify validateCatalog into phase helpers

Extract validateCatalog's validation phases into validateVersionEntries,
validateEntryProvenance, validateTagDrift, validateSourceRelations, and
validateExclusions, dropping its cyclomatic/cognitive complexity from
43/43 to 11/8. Behavior and error paths are unchanged: the pinned catalog
digest and offline hermetic validation still pass.

Also derive the proxy @v/list URL from modulePath so version listing and
module downloads cannot drift to different module identities.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(review): dedupe bd init legacy-upgrade guard; document release catalog anchor

Maintainer review fixups for PR #5100 (adopt-pr review loop, attempt 1).

cmd/bd/init.go: remove the duplicate guardLegacyUpgradeWorkspace block in bd init. After the rebase onto feature/v12-u0-history-20260719 the base already carried the guard, so the PR's added block was a byte-identical second call that re-ran the read-only workspace classification twice and misrepresented the diff intent. A single guard call before the existing-workspace safety checks is retained; behavior is unchanged (reviewer-confirmed idempotent).

scripts/migration-test/catalog/README.md: document that the pinned release catalog is currently a self-validated reference anchor (pinned SHA-256 identity + strict offline validator + go test ./... digest gate) whose intended consumer is the stacked v1.2 upgrade-census / runtime-schema coverage work, since no migration-test harness consumes it yet.

Addresses the two minor findings from the Claude + Codex review synthesis (decision: approve; scorecard 971/1000). Contributor commits preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(migration-test/catalog): correct semver prerelease ordering

versionCompare tie-broke same-core versions with strings.Compare, so
v1.2.3-rc.10 sorted before v1.2.3-rc.2 and a stable release sorted before
its own prereleases. generate and validate share this comparator, so a
wrong canonical order self-validates against the pinned digest and the
anti-tamper check cannot catch it. Replace the lexicographic tiebreak with
a SemVer 2.0.0 section 11 prerelease comparator (numeric-aware identifier
comparison, numeric < alphanumeric, longer set wins, stable > prerelease)
and add regression coverage including rc.2 vs rc.10. The checked catalog's
canonical order and pinned digest are unchanged: its only same-core group,
v1.1.0-rc.1/-rc.2, sorts identically.

Maintainer adoption fixups for PR #5100 review:
- tighten origin-ref provenance from any refs/ prefix to the exact
  refs/tags/<version> or refs/heads/main shapes actually used
- key the digest-pinned Catalog/Entry/SourceZip/AssetRecord composite
  literals to avoid silent field-shift on a future struct reorder
- drop the inert GONOSUMDB= env entry and print the recomputed SHA-256
  from generate to make deliberate digest updates less error-prone

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-18 12:25:44 -07:00
matt wilkie
935afe2466
fix(examples): tidy both example modules and build them in CI (#5229)
* fix(examples): tidy both example modules and build them in CI

Both Go modules under examples/ fail a plain `go build` on current main:

    go: updates to go.mod needed; to update it:
            go mod tidy

They are separate modules that reach the parent through
`replace github.com/steveyegge/beads => ../..`, so their go.mod and go.sum
record the parent's entire dependency graph. Every root dependency change
therefore invalidates them — and nothing in CI ever compiled them, so the drift
accumulated silently. examples/ is the first code a new user copies, which makes
this a bad first five minutes rather than a cosmetic wart.

Two parts:

1. `go mod tidy` in examples/bd-example-extension-go and examples/library-usage.
   The extension example carries most of the churn (~1.6k lines of go.sum),
   which is inherent to recording the parent graph through the replace directive
   and is why an earlier fix (#4942, the Go 1.26.5 bump) deliberately left it
   out. Both modules now build clean with the project's canonical
   `-tags gms_pure_go`.

2. scripts/build-examples.sh plus a `build-examples` job in the PR workflow, so
   the same drift cannot recur unnoticed. The script discovers example modules
   from `git ls-files 'examples/*/go.mod'` (no hardcoded list), sources
   .buildflags for the canonical CGO/tag settings, builds into a scratch
   directory so it leaves no untracked binaries, reports every failing module
   rather than stopping at the first, and prints the exact `go mod tidy`
   command to run.

The job is deliberately NOT added to ci-gate's required list. Because of the
replace directive it would fail on any root go.mod change not mirrored into the
examples, so making it blocking imposes a "tidy the examples too" step on every
dependency bump. That is a maintainer call about contributor friction; the job
is visible on the checks list either way, and pr.yml records how to promote it.

Verified locally: both modules build with -tags gms_pure_go; the script exits 1
and names the module when an example go.mod is reverted to its pre-tidy state;
scripts/check-build-tags.sh stays clean (the script sources .buildflags).

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(examples): address dual-vendor review — vet not build, advisory job, portability

Reviewed by a claude reviewer and scripts/codex-agent reviewer (gpt-5.6-sol).
Four fixes, two of them correctness.

1. `go build -o <dir>/ ./...` was a FALSE GREEN: with -o naming a directory, Go
   compiles only the MAIN packages and silently skips every library package.
   Reproduced in an isolated module with a good main package plus a library
   package containing a type error — `go build -o dir/ ./...` exits 0 while
   `go build ./...` and `go vet ./...` both exit 1. The mirror-image bug: a
   library-only example module fails `go build` with "no main packages to
   build", a false red. Switched to `go vet ./...`, which type-checks every
   package INCLUDING test files (examples/library-usage/main_test.go exercises
   a lot of live API, and `go mod tidy` counts its imports, so a build that
   never compiles it left part of the recorded graph unverified), writes no
   artifacts, and still fails on the stale-go.mod condition this exists for.
   That also deletes the mktemp/trap/scratch-dir machinery entirely.

2. "Not in ci-gate" is NOT "non-blocking", which the previous comment claimed.
   Verified in source: pr-preflight.sh gates on every FAILURE in the raw
   statusCheckRollup and calls block(), and pr-babysit requires all rollup
   entries SUCCESS/NEUTRAL/SKIPPED before merging. Both ignore ci-gate
   membership. So the previous state was the one posture that stalls the merge
   patrol repo-wide after any un-mirrored dependency bump while advertising
   itself as optional. The job is now continue-on-error: true — genuinely
   advisory — and the comment records how to promote it to a real gate. Whether
   it SHOULD be a gate remains the maintainers' call.

3. Portability and robustness in the script, all reproduced by the reviewers:
   - `mapfile` does not exist in bash 3.2 (stock macOS) and `xargs -r` is
     GNU-only, so the advertised local check could not run on macOS. Replaced
     with a NUL-delimited read loop, which also fixes module paths containing
     whitespace (`git ls-files | xargs -n1 dirname` turned "examples/has
     space/go.mod" into two bogus entries).
   - `source ./.buildflags` was unguarded under `set -uo pipefail`; a failure
     continued with GOFLAGS unset and would type-check the ICU path while
     check-build-tags.sh still passed, since that only greps for the literal
     string. Now a hard exit.
   - Finding zero modules exited 0. A job that checks nothing must not report
     success; it is now an error.

Verified: both modules vet clean; reverting an example go.mod to its pre-tidy
state still exits 1 with the exact `go mod tidy` command; a broken library
package now fails where it previously passed; check-build-tags.sh clean
(97 files); shellcheck clean.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(examples): drop GNU-only sort -z from the module discovery pipeline

BSD sort has no -z, so on stock macOS — the exact platform the Bash-3.2
compatibility block targets — the sort stage emptied the pipeline and the
script exited claiming 'found no example modules'. git ls-files output is
already sorted, so the stage bought nothing. Found by cross-vendor review
(codex gpt-5.6-sol) of this branch.

Agent-Signature: claude-fable-5-high on behalf of maphew

* fix(ci,examples): disable setup-go cache in build-examples; re-tidy examples after merge

TestGoCacheOwnershipTopology requires every setup-go step in pr.yml to set
cache: false (caching is owned by explicit restore/save steps); the new
build-examples job predates that policy landing on main. Also re-run
go mod tidy in both example modules so their recorded dependency graphs
match the merged main — exactly the drift this PR's CI job exists to catch.

Agent-Signature: claude-fable-5-high on behalf of maphew

* ci(examples): bound build-examples with timeout-minutes: 10

continue-on-error keeps a red result advisory, but a hung job at the
6-hour default timeout holds the check pending, stalling merge-lane
consumers the advisory posture was meant to protect. Matches
pr-preflight-platforms' bound.

Agent-Signature: claude-fable-5-high on behalf of maphew

* ci(examples): make the advisory posture real - step-level continue-on-error

Job-level continue-on-error still reports the check run as FAILURE in the
PR rollup (only the workflow-run conclusion flips), and pr-preflight /
pr-babysit gate on per-check-run conclusions - so the job as written was
de facto blocking while advertising itself advisory. Follow the cygwin-leg
precedent: continue-on-error on the build step, outcome-guarded ::warning
annotation on failure. Also restore the repo-wide Go module cache (the
examples' graph is the parent's via the replace directive) so the job
stops cold-downloading the full dependency graph on every PR.

Agent-Signature: claude-fable-5-high on behalf of maphew

* test: register build-examples' module-cache restore in the cache topology registry

TestGoCacheOwnershipTopology keeps an explicit inventory of every cache
step per job; the advisory-lane fix added a restore step to build-examples
without registering it, so the whole scripts package went red on this
head. Register the job (inventory, managed map, ordering, setup-go id)
and drop the step's unused 'restore-cache' id — the registry compares
ids, and nothing references it.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019CwiLhLbAGZdJtKYT76CPp

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 21:16:27 -07:00
Steve Yegge
ce20a64bb4 fix(release): version bumps now own the tracked .githooks markers
The v1.2.0 bump (fa3d8cc5c) reddened main: the tracked .githooks/* managed
sections embed the binary Version in their BEGIN/END markers, and
TestTrackedManagedHookSectionsMatchGenerator (added with the sections in
#5522) holds them byte-equal to the cmd/bd/hooks.go generator — but
update-versions.sh's hook step was a stale comment saying there were no
hook files to update (true of the old cmd/bd/templates/hooks/*, not of the
tracked rendered copies). update-versions.sh now rewrites the marker
version wholesale (not from CURRENT_VERSION — the markers were already
drifted at v1.1.0 when the 1.2.0 bump ran), and check-versions.sh gates
the markers so a bump that skips them is refused before the push instead
of reddening main after it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 14:06:28 -07:00
ecuthiell
0498b22a75
fix(scripts): scrub Windows env keys case-insensitively (#5534) (#5609)
Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

Co-authored-by: Steve Yegge <steve.yegge@gmail.com>
2026-08-10 23:45:33 -07:00
ecuthiell
1b67d0b1fb
fix(ci): remove recursive Make from lint formatting (#5302) (#5320)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 21:43:59 -07:00
ecuthiell
9628c0189f
ci: simplify js/wasm boundary proof (saa-cgh.63) (#5612)
Use Go's supported wasm executor wrapper directly in the required CI lane. Preserve non-vacuous exact-test outcome checks and aggregate-gate propagation without runner host-attestation ceremony.

Fixes #5491

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 21:02:35 -07:00
ecuthiell
49a28af801
Validate doc freshness max age (#5524) (#5606)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 21:02:23 -07:00
ecuthiell
bf930078e8
test(scripts): report helper-only probe as skipped (#5599)
Mark the launchability probe skipped in ordinary package runs so a helper-only guard cannot be mistaken for executed coverage.

Signed-off-by: Ewen Cuthiell <ewencuthiell@gmail.com>

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 20:58:58 -07:00
ecuthiell
6748ef650c
build: make repository text EOL deterministic (#5319)
* build: make repository text EOL deterministic

Apply one repository-wide LF policy to detected text so Windows checkout settings cannot turn ordinary Go source or future text formats into CRLF. Preserve concise rationale for LF-sensitive hook templates, embedded Markdown, migration hashes, and generated-doc comparisons; keep the tracked JPEG explicitly binary and remove redundant path selectors.

Exercise a real isolated Git checkout with core.autocrlf=true through a narrow test package. Bind the existing required Ubuntu, macOS, and Windows matrix to exact expected GOOS values, fail closed on missing or wrong host identity and skipped or zero EOL coverage, and preserve the original doc-freshness command as a separate step.

Resolve and positively identify one canonical Git executable, reuse it under a curated environment with repo-local hooks, and protect the workflow topology with regression tests. Bind the required aggregate to startup-clean absolute Linux Bash and uname identities, require the exact POSIX checkout workspace, and re-enter privileged profile-free Bash with exec so ambient functions or startup state cannot turn a gate failure into success.

Exercise the aggregate shell boundary in subprocess regressions with hostile exported source and exit functions, poisoned startup options, paired clean and failing fixtures, wrong-host falsification, and invalid-workspace cases.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* build: split CI-gate hardening from EOL policy

Remove the profile-free CI-gate host-binding chain and its structural and subprocess tests from PR #5319 so the final diff matches the accepted repository EOL motivation. Preserve the required three-host EOL policy lane and its CI Gate propagation. The merge-gate hardening will be reconstructed from current main under its own threat model and review vehicle.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 20:48:09 -07:00
ecuthiell
766e6c17f2
fix: reject incompatible timeout in generated hooks (#5522)
Select timeout or gtimeout only after a successful GNU coreutils identity probe, validate BEADS_HOOK_TIMEOUT as positive whole seconds, and preserve the Perl and warned direct fallbacks under inherited shell options.

Keep GNU and Perl deadlines soft and backend-scoped, preserve natural status 137, regenerate every tracked hook artifact, and exercise the real process boundary through the required three-host preflight.

Fixes #5503

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 20:47:53 -07:00
Julian Knutsen
ff75b45c91
docs: the ext-api-store documentation tranche — post-#5516 auth sweep, wave doc coherence, role checklist refresh (#5545)
Three beads (ga-uwix0.1, ga-2yaqp.5, ga-2yaqp.6) plus a fable review round.
Docs and spec prose only: all 19 touched .go files are gofmt-normalized,
comment-stripped byte-identical to main. The one exception is deliberate and
reviewed — scripts/check_doc_freshness_test.go gains a data row, because the
delegated gate ruling required it.

ga-uwix0.1 — #5516 gave bd serve optional bearer auth and made it MANDATORY
beyond loopback; fifteen prose sites still said the opposite. The three the
council named are reworded to the stronger true form (the actor is not the
authenticated principal even where a bearer is required; a shared token admits
a client to the WHOLE surface and grants no narrower right). The sweep for a
fourth found eight more in five spellings the audit's grep could not reach,
including claim.go's "any client can claim as any name", release.go's
no-AUTHORIZATION variant, and six memory/settings-plane sites where the true
statement is sharper: a bearer decides WHO may call, never what a caller who
is let in may read. The review round found four more in the CHANGELOG's
UNRELEASED section, which round one had misclassified as released history —
they would have shipped false in the same release as the auth entry.

The serve runbook's "Binding beyond loopback" section was actionably wrong: it
said the flag "prints a warning and changes nothing else", and its refusals
table quoted a string not in the binary. Rewritten against a build, with every
flag, error string, event name and reason value produced by running the
command. It gains the operational trap nobody had written down: /healthz is
auth-exempt and GET /v0/beads/ready is not, so an unauthenticated readiness
probe earns a 401 forever while liveness stays green. The runbook is now IN
scripts/check-doc-freshness.sh, pinned to the four files it is read against.

ga-2yaqp.5 — nine prose defects, every count re-derived rather than trusted;
three of the audit's own numbers had moved again. 38 route rows and 36
capability-bearing ones; thirteen pinned schemas, not twelve or seven; six
missing entries in the no-query-parameter list, not five; twenty-eight role
accessors, not twenty-seven; nineteen patch members, not thirteen. Two prose
"code counts" were counting paragraphs. CodeAlreadyClaimed claimed the holder
is always attached when only one of four producers always attaches it and
releaseIssue never does. MetadataCAS's leaf contradicted its own shipped wire.
The path/operationId inversion is now recorded where a spec author looks.

ga-2yaqp.6 — the role checklist refreshed against what this wave's additions
actually cost, traced to real commits. Step 10 sent people to a name list
#5460 deleted. The "four more edits" claim was stated twice and undercounted
both times: the two freshest wire slices share a nine-file core plus
roles_test.go. The step-with-no-number gained the fact that explains its
instances — write-role decorators wrap, read-role decorators recurse, so the
call lands on the stub, and both real instances are reads. The wave got its
first CHANGELOG entry: six new public operations, not three, and zero new role
accessors in the window.

Review verdict: FIX-REQUIRED then all items addressed. The reviewer
live-falsified the strong auth form and failed to break it, and confirmed
every recount re-derives exact.
2026-08-10 08:05:07 -07:00
ecuthiell
07d679ea51
fix: export the native prebuilt test binary path (#5512)
Derive the complete test-runner output with go env GOEXE before both
building and exporting it. Add a fast process regression for launchability
and caller precedence, maintained by the existing three-host preflight lane.

Fixes #5502

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 05:31:23 -07:00
ecuthiell
345dbe8878
test(ci): derive storage timeout nesting (saa-cgh.64) (#5498)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-10 05:29:58 -07:00
Julian Knutsen
cff62bd631
Make the pre-commit lint gate trustworthy, and stop one main-side lint word from reddening every PR (#5538)
* fix(githooks): build golangci-lint with the repo's Go, not golangci-lint's

The pre-commit hook ran `go run golangci-lint@v2.10.1`. For a pkg@version
build the go command honours the TARGET module's toolchain directive, so
GOTOOLCHAIN=auto downgraded to go1.25.12 — and golangci-lint built by
go1.25 refuses this repo's `go 1.26.5` at config load:

    go: github.com/golangci/golangci-lint/v2@v2.10.1 requires go >= 1.25.0; switching to go1.25.12
    Error: can't load config: the Go language version (go1.25) used to build golangci-lint is lower than the targeted Go version (1.26.5)

Every Go commit failed there, so the hook taught six-plus contributors to
reach for --no-verify — the one habit a lint gate must never train.

CI never hit it because actions/setup-go installs the go.mod version and
exports GOTOOLCHAIN=local, so its `go install ...@v2.10.1` compiles the
linter with go1.26.5. Pinning GOTOOLCHAIN to the go directive reproduces
that resolution on any host, including one whose base toolchain is older
than the repo's target (where a bare GOTOOLCHAIN=local would pin the
linter to that older toolchain instead).

Verified on a real staged Go change: red before with the message above,
`0 issues.` after, and still exit 1 on a planted errcheck violation.

Bead: ga-2ltro.13

* ci: lint PRs for what they introduce, sweep the whole tree on main

Both PR lint gates linted the whole tree: the `lint` job left
only-new-issues at its false default, and pr-lint-wrapper ran
`make ci-pr-lint` over ./... . So a single lint word landing on main —
twice this wave, once a prose misspell — turned EVERY open PR red for a
change it did not make, and each author had to either fix someone else's
line or wait.

The tradeoff, recorded beside the job: a pre-existing issue in untouched
code no longer blocks a PR, and because "new" is measured off the diff, a
violation carried in by a MOVE or a rename does. What it buys is that a
main-side word now reds main's own run, where it belongs. main.yml
already runs both jobs unscoped on every push, so whole-tree coverage —
including the GOOS=windows cross-lint — is unchanged; only its blast
radius moves.

The wrapper takes its scope from BD_LINT_NEW_FROM_MERGE_BASE, set only
by pr.yml, so `make ci-pr-lint` run by hand still sweeps the tree. That
makes the local wrapper STRICTER than the PR gate rather than equal to
it, which is what LINTING.md and CI_CLEANUP_PLAN.md said before and no
longer would: both now state the two-lane contract and say the local
run's extra strictness is deliberate. Their freshness markers move with
the workflow and wrapper files they are pinned to.

Bead: ga-uwix0.2

---------

Co-authored-by: Julian Knutsen <ci@beads.test>
2026-08-10 04:55:55 -07:00
ecuthiell
c469a4bc64
fix(hooks): make js/wasm execution explicitly unsupported (#5478)
* fix(hooks): make js/wasm execution explicitly unsupported (saa-cgh.14)

Provide the runHook implementation required by the shared hook runner on js/wasm. Keep process execution out of the WebAssembly target and return a precise unsupported-platform error when the platform-specific boundary is reached.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* ci: enforce js/wasm hook boundary (saa-cgh.14)

Exercise the exact unsupported-execution contract under Go js/wasm and pinned Node in an existing required leaf job. Guard the host identity, exact test selection, and required-gate reachability against false-green drift.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* ci: bind js/wasm proof authority (saa-cgh.14)

Launch the required boundary proof through startup-clean privileged Bash, bind its Go, GOROOT, and Node identities, and curate inherited runtime controls. Parse the exact test result with Bash builtins so ambient helpers cannot counterfeit execution counts.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell

* ci: follow up js/wasm review (saa-cgh.14)

Keep go.mod as the sole Go-version authority for the required wasm lane, centralize pinned setup action identities, and document the singleton test selector's maintenance contract.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-09 11:49:54 -07:00
ecuthiell
bad15e7959
ci: give storage jobs explicit timeout budgets (saa-cgh.60) (#5488)
The race-enabled UOW package now runs close enough to Go's implicit 10-minute deadline that ordinary CI variance can fail unrelated changes. Give the storage tier 15 minutes, preserve the doctor/fix tier's existing 10-minute behavior explicitly, and bound their shared job at 30 minutes. Mirror the policy in PR and main workflows and pin the nested budgets plus required-gate wiring in a structural test.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-09 09:13:09 -07:00
ecuthiell
a2cc86ee3a
test(scripts): stabilize Git Bash shim paths (saa-cgh.55) (#5465)
Resolve shim and state paths under the same minimal Bash environment used by
the test commands so Git for Windows cannot remap /tmp between setup and
execution.

Reset the effective command PATH inside profile-free Bash and prove docker,
sleep, and go resolve to the generated shims before either target script runs.
This keeps ambient host tools and real retry sleeps out of the harnesses while
preserving their exact argv, call-count, and exit-status assertions.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-09 01:05:13 -07:00
CI Bot
0a5fcad975 Merge remote-tracking branch 'origin/main' into report/events-journal 2026-08-09 02:28:04 +00:00
CI Bot
fffddc67cd feat(events): clone-local events journal tables and the DML scanner (bd-opisf)
Add the storage substrate the durable events journal is built on, ahead of the
engine that writes to it.

bd_events_journal records every committed bead mutation; bd_events_seq is the
single-row counter its seq is drawn from. Both are clone-local: registered in
doltIgnorePatterns before the migration runs, so the CREATE never lands as
tracked-at-HEAD in the first place. That placement is the whole design. A
journal row per mutation on the versioned plane would restore the history churn
0062 just removed, and a per-clone counter on a replicated table would conflict
on every merge — so the journal describes THIS clone's working set, and nothing
else. seq is deliberately not AUTO_INCREMENT: that assigns at INSERT rather than
at commit, so under concurrent transactions a lower seq can commit after a
higher one is already visible and a consumer tailing WHERE seq > cursor skips it
permanently. Drawing seq from a shared counter row inside the mutation's own
transaction converts that silent skip into a detected conflict.

The tracked 0064 and the ignored-series twin 0022 create the identical shape, so
the fresh-init door and the fresh-clone door converge; both tables join the
per-table shape diff in migrate_ignored_plane_shape_test.go, which is what pins
that. The hygiene script's clone-local table list is extended to match
doltIgnorePatterns, its documented mirror.

journalscan holds the AST primitives the completeness guards will share:
per-function bead-table DML detection (literal, %s, and %[1]s table names) and a
call-graph fixpoint. Detecting mutators by the SQL they execute rather than by
name is what stops a mutator named off-pattern from shipping un-journaled, so
the detector itself is unit-tested — a false negative there silently disarms
every guard built on it.

lock_test's dirty-guard shape pinned its mocked cursor to LatestVersion()-2 to
keep 0062 (the migration that touches `events`) among the pending set. That
offset stops covering 0062 the moment anything newer lands, which 0063 and this
change do; it is now pinned to the flip migration's own version.

Re-port of OSS PR #4916 (head 3666e5026) onto current main, folding in the
3bfed2431 hardening generation. Migrations renumbered from that lineage's
0056/ignored-0014.

Co-authored-by: CI Bot <ci@beads.test>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of julianknutsen
2026-08-09 01:58:17 +00:00
Steve Yegge
cad9344acd
Merge pull request #5420 from gastownhall/dependabot/github_actions/actions-1537e80d38
chore(deps): bump the actions group with 5 updates
2026-08-08 15:56:34 -07:00
Steve Yegge
891e8d5d4e test(ci): track actions/cache v6.1.0 in the cache-ownership ratchet
TestGoCacheOwnershipTopology pins the released actions/cache SHA; the
actions-group bump moved cache/restore and cache/save from v5.1.0
(caa2961) to v6.1.0 (55cc834), so the ratchet constant moves with it.
Verified 55cc834 is the tagged v6.1.0/v6 release commit upstream.
2026-08-08 15:39:11 -07:00
Julian Knutsen
869020c221
Fix a SIGPIPE in the mutation harness's worktree guard (#5434)
Reported by the first caller to use scripts/mutation-equivalence.sh at scale: it
killed 5 of 9 mutation runs, presenting as "not a worktree of $REPO" for a path
that plainly was one.

grep -q exits on the first match, which SIGPIPEs the git worktree list feeding
it. Under set -o pipefail the pipeline is then nonzero and the negation takes
the failure branch, rejecting a legitimate worktree. It only bites once the
listing is large enough that git is still writing when grep exits — this repo
has ~370 worktrees and a 53 KB listing — so it reads as intermittent and
machine-specific.

Captured into a variable instead of piped. Verified both directions: a real
worktree is now recognised and falls through to the branch-checked-out guard as
intended.

The guard this broke was added two commits earlier, in response to a review that
found the tool would hard-reset a live checkout. Tightening it introduced a
failure mode that made it refuse legitimate ones. Worth recording, because the
reflex when a guard misfires is to loosen the guard rather than look for the bug
in it — and the caller who hit this worked around the symptom rather than
reporting it, which is how a tool acquires a reputation for flakiness.
2026-08-08 08:28:08 -07:00
Julian Knutsen
e29b5eadb4
Write down how to retire a test against a contract, and ship the tool for it (#5425)
Follow-up to the four-slice deduplication pass (#5398, #5399, #5405, #5406).
No behaviour change.

That pass retired ~700 lines net, and its reviews found about a dozen coverage
losses and six tests that could not fail — every one of them green under
go build, go vet, go test and golangci-lint. The reasoning that caught them
lived in review transcripts and a scratch directory. Written down nowhere, the
next pass rediscovers it by shipping the loss first.

The engdoc section states each rule as the check that would have caught the
failure earlier: a mutation verdict is only true of the body you mutated; a
wrapper is a composition the contract structurally cannot see; write a residue
from the decision surface rather than the mutants you happened to run, since
half of what one such residue missed were POSITIVE assertions; ask what the
fixture makes unobservable, because every fixture-defect case had correct
assertions; never let a role-answer assertion replace a raw-row one; cite by
symbol, not line.

scripts/mutation-equivalence.sh is the tool. It ran the entire pass from a
scratch directory, which means it would not have survived it.

A fact-check review then found more wrong with this change than with anything it
describes. The section cited `runTransaction`, which does not exist in the dolt
package — while teaching symbol citation. It claimed no contract case reads
is_blocked; there are 45 references across ten files, several added by the pass
it describes. Three of its counts did not reconcile with the merged record, and
one was curated to make a sentence read true. All corrected, with the counts now
qualified and the pull requests named as the record.

The same review demonstrated three false-REDUNDANT routes in the script — the
one wrong answer that costs coverage — plus a guard that refused the repo only
by accident, and would have hard-reset the caller's own checkout on a slightly
different setup. Build failures and timeouts are now reported UNUSABLE rather
than read as catches; checkout and reset failures are fatal rather than
measuring a different commit; off-file mutations are fatal; A == B is refused;
and the worktree must be a detached scratch worktree of the repo.

A seventh guard came from using it: it measures HEAD, not the working tree, so
uncommitted work is invisible and the run reads INCONCLUSIVE. It now warns.
2026-08-07 18:58:48 -07:00
ecuthiell
80b13798dc
docs(lint): align contributor guidance with CI (saa-90w) (#5317)
Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Ewen Cuthiell
2026-08-07 05:37:12 -07:00
Julian Knutsen
92ad0295f1
Guard and qualify historical upgrades to v1.2 (#4907)
* Add sealed-copy SQLite migration bridge (bd-mvh)

Create a fresh current workspace from a fingerprinted copy of an authenticated historical SQLite workspace. Keep every historical executable inside an isolated destination and reject malformed or semantically lossy cutovers without touching the source.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Refuse implicit historical workspace upgrades (bd-mvh)

Detect historical SQLite and legacy Dolt layouts before config migration, version tracking, init, bootstrap, doctor, or store construction. Require the sealed-copy bridge while preserving current shared-server admission against the explicitly selected workspace.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Qualify authenticated historical upgrades (bd-mvh)

Replace generic recipe discovery with a fixed, checksum-pinned corpus spanning the reviewed SQLite, server-Dolt, and embedded-Dolt eras. Assert source isolation, representative data fidelity, schema idempotence, and rollback retention while keeping historical executables inside bounded temporary environments.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Test User <test@test.com>
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Add authenticated legacy SQLite reader (bd-tku)

Let the current binary read the exact v0.49.6/v0.50.3 SQLite contract from a sealed private DB/WAL copy and emit canonical JSONL for the normal fresh-Dolt import path. Reject schema drift and values that current import would lose, default, truncate, or deduplicate.

Qualify both authentic releases end to end, including current DATETIME(0) canonicalization and reader-to-fresh-import timestamp equality.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix empty embedded-root upgrade bypass (bd-e6m)

Require an actual embedded Dolt repository marker before current layout precedence can suppress historical SQLite or legacy Dolt refusal.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* fix(migrate): maintainer review fixups for legacy SQLite upgrade PR

- decompose the legacy-workspace guard and extract the dependency-row loader
  (behavior-preserving), and align the legacy SQLite reader projection with
  issueops.ScanIssueFrom arity after the 0060/0061 rebase, guarded by a
  projection arity test.

- assert the historical-upgrade harness schema no-op by shape
  ("Schema already at v<N>") instead of a hardcoded v59, so the migration-test
  matrix stops reddening on every schema bump (candidate now prints v61).

* Fix historical upgrade review blockers (bd-mvh)

Route direct SQLite lanes through the current authenticated reader, guard no-store commands before legacy workspace mutation, and move coarse embedded repository detection behind the storage adapter. Restore the legacy reader projection contract and verify source-preserving behavior across the repaired paths.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix historical upgrade edge cases (bd-mvh)

Refuse ambiguous local server-Dolt roots before storage opens and normalize nullable legacy SQLite sentinel fields during authenticated export.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

* Fix final historical upgrade review blockers (bd-mvh)

Make explicit server selection outrank stale embedded repository markers, so ambiguous local server roots still fail closed.

Permit zero-record cutovers only through the authenticated direct SQLite reader, verify the empty round trip against a fresh current export, and align the upgrade guidance with the enforced admission matrix.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Test User <test@test.com>
2026-08-06 18:45:04 -07:00
CI Bot
50a780647f Follow the relocated large-fixture budget in the testing.Short allowlist
CI's PR policy failed on scripts/check-testing-short.sh. The check is an
allowlist keyed on file::function, and the sweeper slice (519fb1479) deleted
cmd/bd/prune_bench_test.go, moving that budget into internal/workapi as
TestCandidateIDMatcherLargeFixture — the same skip, in the package the matcher
moved to, renamed for what it now measures. The allowlist entry became stale in
the same commit that created its replacement.

This is a RENAME of the existing exception, not a new one: the old entry is
removed, so the count of approved testing.Short() sites is unchanged and a
genuinely new skip still has to be argued for.

Also two stale counts, both the same accumulation this series has been prone
to — a wave adds a role and updates the prose in one place but not the other:

  - cmd/bd/serve.go said "the two issue roles" in two places, above a struct
    that now carries thirteen.
  - internal/httpapi/roles_test.go had THREE separate cases each introducing
    itself as "the newest role" (querier, sweeper, ready counter). At most one
    can be, and none of them is after wave 4. The general point those comments
    were each restating — that the case which matters is the newest role
    missing, because that is what a caller written against the previous release
    passes — is stated once on the test's doc comment now, so each row keeps
    only what is specific to it.
2026-08-06 05:52:52 +00:00
Julian Knutsen
1e0a7bdf59
ci: shadow PR Risk capability ownership (bd-8yjq8.1) (#5324)
Record fail-closed candidate lane selection without changing which PR Risk jobs execute. This creates replayable evidence for capability-owned admission while branch protection and historical validation are still pending.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-03 20:06:16 -07:00
Julian Knutsen
8a2748c4dd
ci: reuse built bd in macOS tests (bd-vygvv) (#5311)
Reuse the current-checkout binary that each macOS job already builds instead of letting subprocess helpers relink it serially. The workflow contract test pins provenance, ordering, matrix scope, and unchanged race-enabled test commands.

Expected hosted saving: 61.66-105.74 seconds without removing coverage or raising timeouts.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-02 21:46:29 -07:00
Julian Knutsen
a6b51bd36b
ci: let macOS caches survive test failures (bd-712nf) (#5292)
Allow trusted-main cache writers to publish content-addressed module and build caches after an ordinary test failure while still suppressing saves on cancellation. This prevents cold macOS runs from deadlocking cache bootstrap behind a successful test run.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-02 12:02:32 -07:00
Julian Knutsen
ba55d4ca78
ci: give Go caches explicit ownership (bd-712nf) (#5278) 2026-08-02 07:10:35 -07:00
matt wilkie
4983ec9c32
fix(schema,ci): prepared DML — not just prepared ALTER TABLE — silently no-ops on the Dolt CLI path (#5232)
* docs(schema)+ci: prepared DML, not just prepared ALTER TABLE, no-ops on the CLI path

cli_migrations.go's header documented the Dolt CLI batch-path limitation as
being about "some prepared ALTER TABLE statements". That understates it. Per
dolthub/dolt#11345 (verified on dolt 2.2.0 and 2.2.2), `dolt sql -q`/`-f`
silently no-ops EXECUTE of any prepared DML -- an UPDATE/INSERT/DELETE built
into a SET @sql string -- while reporting success; a prepared SELECT or a
direct statement on the same path runs correctly. Prepared ALTER TABLE is one
instance of that, not the scope of it.

Broaden the comment to say so, and add Check E to check-migration-hygiene.sh
so a NEW main-plane migration cannot quietly depend on a prepared write.
Prepared ALTER TABLE keeps its accepted-idiom status and is not flagged: only
the leading keyword inside the quoted text decides. Detection buffers a
statement across lines, because the common form here is a multi-line
`SET @sql = IF(cond, '...', '...')` whose PREPARE arrives several lines later,
so a single grep cannot correlate the two.

Deliberately new-files-only. Seven shipped migrations (0035, 0037, 0041, 0047,
0053, 0055, 0058) already PREPARE writes to real tables, and that is not a
latent bug: a shipped migration is frozen, and the only path this check is
about is the fresh-schema bundle, where an empty database makes a data
backfill a no-op whether or not the prepared write lands.
cliCompatibleMigrationSQL overrides just the subset whose prepared statements
change the committed schema shape, which is what the bundle actually promises.
The guard is there to stop the idiom spreading into a migration where the
write does matter.

Verified: hygiene script passes clean on the tree; a scratch migration
carrying a prepared ALTER TABLE, a prepared UPDATE and a multi-line prepared
INSERT flags exactly the two writes and not the ALTER.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): the prepared-DML check rejected the pattern it recommends

Cross-vendor review (codex reviewer, gpt-5.6-sol) found two false positives in
Check E, both confirmed against source:

1. It flagged prepared writes to migration-local stand-in tables — which is
   exactly the remedy its own failure message hands out. Migration 0059
   (gastownhall/beads#4877) copies into __bd_0059_* throwaway tables with
   PREPARE and then has a direct statement read them, precisely so a silent
   no-op degrades gracefully instead of corrupting state. A new migration
   following that documented pattern would have been rejected by the check
   telling it to follow that pattern.

   Fixed by recognising the tree's stand-in naming convention: a double
   underscore prefix, which no real beads table uses (__bd_0059_* and
   __temp__* are the existing instances). The multi-table DELETE form yields
   an alias rather than the table, so it stays flagged — conservative in the
   right direction.

2. It scanned .down.sql files. Only migrations/*.up.sql is embedded into the
   CLI fresh bundle (//go:embed at schema.go:218), so a down migration can
   never meet the bug this check is about; scanning it rejects valid work for
   no reason.

Verified with three scratch migrations: the __bd_-targeted INSERT is not
flagged, a real-table UPDATE still is, and a .down.sql carrying prepared DML
is not. Clean pass on the tree throughout.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): exempt a prepared statement only when EVERY DML branch is a stand-in

Second cross-vendor review round found the stand-in exemption checked only the
first DML target it matched. A single

    SET @sql = IF(cond, 'INSERT INTO __stage ...', 'UPDATE issues SET ...');

carries two branches and only one executes, so matching the stand-in INSERT
first waved through the real-table UPDATE sitting beside it — the exact
false negative the check exists to prevent, reachable by combining the
recommended pattern with an ordinary one. Reproduced before fixing.

Now extracts every INSERT/UPDATE/DELETE target in the statement and exempts
only when all of them are __-prefixed. Extracting nothing recognisable no
longer exempts either: the multi-table delete form (DELETE wd FROM ...) yields
an alias or nothing, and both stay flagged.

Verified: mixed IF flags, both-branches-stand-in passes, the 0059 shape still
passes, a plain real-table prepared write still flags, clean tree still passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): exempt only the one prepared form whose write target is unambiguous

Third review round, third defect in the same exemption, and the pattern in the
failures is the point: I was trying to identify SQL write targets with a
regex, and SQL will not be identified that way.

    UPDATE __stage s JOIN issues i ON i.id = s.id SET i.priority = 0

writes to issues. Every version of this check so far extracted __stage and
exempted it. And 'ON DUPLICATE KEY UPDATE v = 2' read as a second write verb,
so a perfectly safe stand-in upsert was rejected.

Narrowed instead of patched again. A prepared UPDATE or DELETE is now NEVER
exempted -- their multi-table forms can move the modified table behind a JOIN,
so the target is not reliably knowable here. INSERT INTO is exempted when every
target is a __ stand-in, because its target is always the token after INTO
whatever the SELECT source does. ON DUPLICATE KEY UPDATE is neutralised before
the verb scan.

The cost is that a future conditional real-table-free UPDATE gets flagged and
has to be restructured the way 0059 does, or justified. That is the right side
to err on for a check whose whole purpose is refusing to guess about writes.

Verified across seven shapes: joined UPDATE to a real table flags, joined
DELETE flags, mixed IF with one real branch flags, INSERT into a real table
from a stand-in flags, the 0059 shape passes, a stand-in upsert with ON
DUPLICATE KEY UPDATE passes, clean tree passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): match the write verb as a token anywhere, not at a fixed position

Fourth review round, and both findings are the round-4 mistake again in the
detector rather than the exemption: I was requiring a SQL construct to sit at
an exact position.

- 'SET @sql := ...' never started buffering, because the assignment regex only
  accepted bare '='. Dolt accepts both.
- The DML test required the verb immediately after the opening quote, so
  '/* conditional */ UPDATE issues ...' and 'WITH t AS (...) UPDATE issues ...'
  both sailed through — valid SQL performing exactly the prepared write this
  guard exists to reject.

Rather than add two more position cases, the verb is now matched as a whole
token anywhere in the statement. That is strictly more conservative, and the
stand-in exemption is what rescues the legitimate forms — the check no longer
depends on guessing where in a statement something will appear.

The one thing that widening could have broken is a prepared ALTER TABLE
carrying 'ON UPDATE CURRENT_TIMESTAMP', which is a column attribute and not a
write. Both that and 'ON DUPLICATE KEY UPDATE' are masked before the scan, in
one shared helper used by the detector and the exemption alike.

Verified across eight shapes: ':=' assignment, leading block comment, and
leading CTE all flag; prepared ALTER with ON UPDATE, stand-in upsert with ON
DUPLICATE KEY UPDATE, and the 0059 shape all pass; joined UPDATE to a real
table flags; clean tree passes.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* docs(ci): state Check E's known limits instead of implying it is a parser

Fifth review round found two more bypasses: a multi-variable
'SET @guard = 1, @sql = ...' (only the first variable is tracked) and a
PREPARE split across lines. Both are valid SQL, both real.

Not fixing them here, and saying so in the script rather than shipping a
silent gap. The reasoning:

- Neither form occurs anywhere in the current migration tree. Verified: the
  only PREPARE-without-FROM hits are comments.
- Check E is advisory hygiene on NEW files, which a human reviews anyway.
- The convergent fix is statement-level scanning rather than a line-oriented
  state machine -- it subsumes all three known bypasses at once, exactly as
  rounds 3-5 kept teaching about position-anchored matching. Attempting that
  rewrite at round 5, with no remaining review budget to validate it, would
  ship an unvalidated parser. That is worse than a documented heuristic.

So the header now carries a KNOWN LIMITS block naming all three shapes and
telling a reviewer not to trust a green check when they see one. Tracked as
mybd-q8hy2.

A check that overstates its coverage is more dangerous than one that admits
where it stops.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): cover PREPARE-from-literal, and refuse to exempt an unclassified INSERT

Sixth review round, and unlike the previous round's findings these two are not
exotica I was willing to document and defer:

- 'PREPARE stmt FROM ''UPDATE issues ...''' — the literal form, no user
  variable in play — was not checked at all. That is the most direct spelling
  of the hazard this guard exists for, and arguably the first thing an author
  writes. Now checked in place rather than through the assignment bookkeeping.

- The stand-in exemption counted classified targets but not INSERT verbs.
  MySQL and Dolt accept 'INSERT issues ...' with INTO omitted, so
  IF(c, 'INSERT INTO __stage ...', 'INSERT issues ...') produced exactly one
  stand-in target, looked unanimous, and exempted the real-table write in the
  other branch. The exemption now requires every INSERT verb to have produced
  a target, so anything it cannot classify blocks the exemption rather than
  silently missing from the tally.

That second one matters more than its severity label suggests: it is the
exemption granting safety it should not, which is worse than the detector
missing a shape. An exemption must fail closed.

The two bypasses documented last commit stand as documented -- multi-variable
SET and a PREPARE split across lines, neither of which occurs in the tree
(mybd-q8hy2). The KNOWN LIMITS block now notes that the literal PREPARE form
is covered, so the list is not read as broader than it is.

Verified across eight shapes, including both new ones and every previously
passing case: literal-PREPARE UPDATE flags, literal-PREPARE stand-in INSERT
passes, INTO-omitted mixed branch flags, and the 0059 / ON DUPLICATE KEY /
prepared-ALTER-with-ON-UPDATE cases all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* docs(ci): apply the KNOWN LIMITS note the previous commit claimed to make

9850b8efc's message says "The KNOWN LIMITS block now notes that the literal
PREPARE form is covered". It does not — the edit was scripted, its assertion
failed on a wording mismatch, and because that step was not chained to the
commit the commit went ahead anyway with a message describing a change that
was not in the diff.

Applying it for real: the block now states that
'PREPARE ... FROM ''<literal>''' IS covered, so the three listed bypasses are
not read as broader than they are, and the round count says six rather than
five.

Recording the cause rather than quietly fixing it, because the failure mode is
worth knowing: a scripted edit that fails its own precondition, in a command
whose later steps are not gated on it, produces a commit that lies. Chain the
commit to the edit.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): REPLACE is a write, and a PREPARE literal may be double-quoted

Two more from review, both ordinary MySQL rather than exotica:

- REPLACE INTO was not in the write-verb set at all, so a prepared
  'REPLACE INTO issues ...' passed the check entirely. It is now detected, and
  like UPDATE and DELETE it can never be exempted -- this function classifies
  INSERT INTO targets only, so any other write verb must block the exemption
  rather than go untallied. A REPLACE targeting a stand-in is therefore
  flagged too; that is the fail-closed direction and an author can restructure
  it as an INSERT.
- The literal-PREPARE branch matched only single-quoted strings, so
  PREPARE stmt FROM "UPDATE issues ..." slipped past.

Verified across eight shapes: double-quoted literal flags, REPLACE into a real
table flags, REPLACE into a stand-in flags (unclassified, fail-closed), and
the 0059 / literal-stand-in-INSERT / ON DUPLICATE KEY / prepared-ALTER-with-
ON-UPDATE cases all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie

* fix(ci): match the non-INSERT verb boundary the same way the detector does

The exemption's UPDATE/DELETE/REPLACE rejection required whitespace after the
verb, while prepared_has_dml used a token boundary. Dolt accepts a comment as
a token separator, so

    IF(c, 'INSERT INTO __stage ...', 'UPDATE/*x*/ issues SET priority = 3')

was seen as DML by the detector, then sailed through the exemption: the
whitespace-anchored test never noticed the UPDATE, the INSERT verb and target
counts agreed, and the real-table write passed. Reproduced, then fixed by
using the same boundary in both predicates.

Two predicates that must agree on what a verb is should not spell it two
different ways; that divergence is the whole bug.

Verified: comment-separated UPDATE now flags, double-quoted literal REPLACE
flags, and the 0059 / ON DUPLICATE KEY / prepared-ALTER-with-ON-UPDATE cases
plus the clean tree all still pass.

Agent-Signature: claude-opus-5-high on behalf of matt wilkie
2026-08-01 23:28:45 -07:00
Julian Knutsen
b73c987302
ci: decouple artifacts from duplicate checks (bd-4b1dt) (#5264)
Build and upload reusable artifacts immediately while the existing policy and lint wrapper jobs retain independent required ownership. This releases artifact-dependent tests 4.6-6.2 minutes earlier in recent PR runs without dropping any check.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-01 22:49:45 -07:00
matt wilkie
c3e600c940
fix(pr-preflight): warn when a PR-gate workflow is red for every PR while the base shows green (#5251)
* fix(pr-preflight): sample PR-gate health, not just base-branch runs (bd-msll)

The base-branch health check reads gh run list --branch <base>, which only
sees workflows that run on the base branch itself. A job that exists only in
the PR workflow (pr.yml) never runs there, so preflight could report "base is
green" while every fresh PR was red in that job, and pr-babysit mirrors
preflight's verdict. Add a supplemental sample of recent completed
pull_request-event runs across PRs: warn (or block under
PR_PREFLIGHT_BLOCK_RED_BASE=1) only when at least 6 decisive runs span 3+
distinct branches and every one of them failed; stay silent otherwise
(some success, too few samples, or an empty/unreadable sample).

Agent-Signature: claude-sonnet-5-medium on behalf of matt wilkie

* fix(pr-preflight): per-workflow PR-gate detection, warn-only, skip on red base

Review found the lumped-sample design could never fire for a single broken
pr.yml-only job: mixing all pull_request-event runs together diluted one red
workflow against every other green PR workflow. Rework to group_by(workflowName)
and flag any single workflow whose decisive runs (>=5, >=3 distinct
headBranch values) are ALL failure-class, naming that workflow in the message.

Also: drop the PR_PREFLIGHT_BLOCK_RED_BASE coupling entirely (a [block] line
here matched neither pr-babysit's red-base nor transient-block patterns, which
would have parked every merge lane and revoked the red-base merge exception —
wiring this into the patrol is a separate change); skip the sample when the
base branch is already red (that already explains PR redness and avoids the
contradictory "base branch shows green" wording); add startup_failure to the
failure-class regex to mirror pr-babysit's probe; and guard against gh exiting
0 with non-JSON output so a malformed sample stays silent instead of aborting
under set -euo pipefail.

Tests reworked for the per-workflow design: fake gh keys the PR-gate dispatch
on the presence of --event pull_request; new cases cover a mixed-workflow
sample (only the all-red workflow fires and is named), red-base-skips-sample,
an unreadable/non-JSON sample, positive coverage of the base-branch dispatch
via BASE_RUN_LIST, and the newest-run selection. Removed the
PR_PREFLIGHT_BLOCK_RED_BASE=1 block-mode test for this detector.

Agent-Signature: claude-sonnet-5-medium on behalf of matt wilkie

* test(pr-preflight): pin the PR-gate detector's warn-only contract; harden jq guard; document

Review round: a regression test asserts the detector never emits [block]
even under PR_PREFLIGHT_BLOCK_RED_BASE=1 (downstream automation classifies
unrecognized block lines as genuine merge blockers), the group_by jq gets an
abort guard for valid-JSON-but-not-objects input, and scripts/README.md
describes the new sample.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

* fix(pr-preflight): only diagnose the PR gate against a known-green base

Codex review round: with zero decisive base runs preflight has just warned
that base health is undetermined, and the broken-gate message's 'while the
base branch shows green' clause would contradict it. Gate the sample on
base_green_count > 0; fixture cases now supply a green base sample and a new
case pins the undetermined-base silence.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie

* fix(pr-preflight): group the PR-gate sample by workflowDatabaseId, not display name

Codex review round 2: workflow display names are not unique and org/ruleset
runs can lack one entirely, so name-grouping could merge distinct gates.
Group by the stable id, keep the name for display. The repo-wide sample's
base-branch conflation is documented as an accepted limitation of a
warn-only supplemental detector.

Agent-Signature: claude-fable-5-high on behalf of matt wilkie
2026-08-01 22:46:44 -07:00
Julian Knutsen
db966433eb
test: inspect benchmark command environments without exec (bd-l7i27) (#5261)
Expose the exact runBD and runShell command construction so the environment-denylist contract is tested at the process handoff. This removes two self-reexec children and the contention-sensitive five-second test timeout while preserving production execution behavior.

Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of CI Bot

Co-authored-by: CI Bot <ci@beads.test>
2026-08-01 19:50:10 -07:00
Julian Knutsen
06b34deba0
ci: retry Dolt image pulls (bd-m0o1z) (#5228) 2026-08-01 08:18:41 -07:00
CI Bot
f373508dfd merge(bd-ilb): OpenAPI v0 spec, types-only codegen and drift gates 2026-07-31 18:34:34 +00:00
CI Bot
e988fd002d feat(httpapi): OpenAPI v0 spec, types-only codegen, and the drift gates (bd-ilb)
Lands the bd serve wire contract and the machinery that keeps it honest. No
runtime behavior: nothing routes, nothing serves, and no CLI surface changes.

internal/httpapi/spec/openapi.v0.yaml is hand-written and is the source of
truth; internal/httpapi/apigen is generated from it with oapi-codegen
types-only (no server stubs, no router), via a go.mod tool directive so
contributors need no extra install.

The decisions this slice fixes, because they are one-way doors:

- No second wire struct. Issue, IssueWithCounts, IssueDetails and
  IssueWithDependencyMetadata are pinned to the canonical Go types with
  x-go-type, so the generator emits type aliases and the CLI's --json, the
  JSONL interchange and the HTTP bodies all marshal one struct. A hand-written
  Go-to-Go mapping layer would recreate the drift disease this epic exists to
  cure. Guarded by a two-way JSON-tag bijection test, not a round-trip fixture:
  a new omitempty field absent from a fixture round-trips cleanly in both
  directions and proves nothing.
- limit=0 means unlimited, matching the CLI exactly, with defaults 50 (list)
  and 100 (ready) documented as the shared constants. The only exception --
  refusal under --allow-non-loopback -- is in the limit description.
- ready returns IssueWithCounts, not bare Issue.
- Unknown query parameters are rejected with 400 invalid_argument naming the
  parameter in a machine-readable extension member: with types-only codegen the
  decoders are hand-written, and Go's silent-ignore default would WIDEN a
  filtered result set under version skew.
- /v0/beads/issues forces (created_at DESC, id ASC) on every request, and the
  spec documents the deliberate divergence from bd list's priority ordering.
- One RFC 9457 problem+json envelope everywhere, with a frozen code vocabulary
  mapped from the sentinels via errors.Is. 5xx bodies carry a static per-code
  detail; the real error goes only to the log, because driver and dial errors
  routinely embed the DSN.
- Only the status codes the six operations need are documented. Every
  documented status+code pair is permanent wire surface.

Two traps found while building this, both now guarded:

- oapi-codegen v2.6.0 silently drops x-go-type from a component that also
  carries allOf. An allOf-composed IssueWithCounts generated
  "type IssueWithCounts = types.Issue" -- the ready endpoint would have shipped
  with the count fields gone. The pinned schemas therefore repeat their
  property lists, and internal/httpapi/pinning.go turns a regression into a
  build failure.
- context.DeadlineExceeded satisfies net.Error, so a tripped request deadline
  would have mapped to 503 db_unavailable instead of a generic 500.

make api-check is the three-part gate (regenerate, diff-or-fail, spec tests)
and runs from scripts/ci/pr-policy.sh, i.e. in a required PR job -- the Main
workflow is perpetually concurrency-cancelled, so a gate living only there
would be unenforced.

The go.mod diff is the tool directive and its closure. The protocol corpus is
untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 08:37:34 +00:00
CI Bot
e2d82a54d7 refactor(list): route both bd list paths through internal/workapi and delete list_filter.go (bd-fv4)
Second half of the extraction. cmd/bd/list_filter.go is gone; the direct
path (list.go), the proxied path (list_proxied_server.go), and the four
other consumers the file had grown (search, search_proxied_server, count,
count_proxied_server, graph_apply) call workapi.BuildListFilter /
LoadStoreListConfig / LoadUOWListConfig / ApplyStatusFilter /
ValidStatusList instead.

listInput now embeds workapi.ListParams rather than copying into it. A
50-field flags -> params mapping would be a silent-swap hazard the compiler
cannot see; embedding makes cmd/bd fill the shared struct directly, so
there is one definition of a list query and nothing to keep in sync. The
CLI-only fields (output format, pager, tree/deps rendering, effectiveLimit,
--repo) stay in listInput, which is why ListParams is a subset and not the
whole thing.

applyStatusFilter and validStatusList are exported rather than kept
unexported as the design sketch had them: search.go, search_proxied_server.go
and graph_apply.go already called them, and leaving them behind in cmd/bd
would have forked the status-parsing rules.

The --limit flag now registers workapi.DefaultListLimit. internal/config's
list.limit viper default keeps its own literal 50 - internal/config cannot
import workapi, because workapi imports internal/config for
GetCustomTypesFromYAML.

Boundary enforcement lands with the package, per the design: a
workapi-frontend-boundary depguard rule denying cobra and net/http under
internal/workapi, and a pr-policy.sh grep for the symbol-level half depguard
cannot express (config.GetDirectoryLabels, os.Getenv). Both were verified to
fail on a deliberate violation and then reverted.

Gates: protocol corpus unchanged (whole cmd/bd/protocol package green with
BEADS_PROTOCOL_REQUIRE_DOLT=1, no testdata diff); TestBuildListFilterGolden
35/35 against the filters recorded from the old builder; TestEmbeddedList
52 subtests and TestProxiedServerList 65 subtests green; count, search and
graph-apply suites green; golangci-lint clean on cmd/bd and internal/workapi
apart from the two pre-existing main.go gosec baselines.

Agent-Signature: claude-code-opus-5-unknown-reasoning on behalf of CI Bot
2026-07-31 06:01:48 +00:00
Steve Yegge
63a471a1e4
fix(schema): carry main-plane wisps ALTERs onto the ignored track for fresh clones (bd-hs7fa) (#5170)
Fresh clones materialize the clone-local (dolt_ignored) tables from the
ignored migration series with the main schema cursor already at-latest, so
a main-plane migration's DDL against one of those tables silently never
reaches them. Observed in prod on the wy-98eh5 VM re-clone: 0060 added
wisps.storage_class on the main plane only, and every shared wisp scan from
a post-0060 binary failed with Error 1054 (same mechanism as wy-pt82l's
wisps.row_lock, healed by ignored/0013).

- ignored/0020: guarded ALTER ADD wisps.storage_class, mirroring 0060.
- ignored/0021: 0049's wisps TEXT->LONGTEXT widening on the ignored track
  (fresh clones otherwise cap large-content columns at 64KB).
- New embedded convergence test diffs every clone-local table's column
  shape between the fresh-init and fresh-clone doors; it found exactly the
  two gaps above and is the permanent engine-level backstop.
- check-migration-hygiene.sh check D: a new main-plane migration with DDL
  on a clone-local table must ship an ignored-series twin in the same
  change (events is in this class since 0062).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 14:10:40 -07:00
matt wilkie
fdee94f815
fix(preflight): classify status checks latest-per-name, not raw rollup entries (#5115)
* fix(preflight): classify status checks latest-per-name, not raw rollup entries

statusCheckRollup keeps every check run on the head SHA, including runs
from cancelled or superseded check suites. A head whose earlier suite was
cancelled carries stale CANCELLED/FAILURE entries forever next to the
fresh green re-runs of the same checks, so preflight's raw count reports
'N status check(s) failed or require action' while GitHub itself says
CLEAN — permanently blocking autonomous merges of green PRs (observed on
#5073/#5074/#5076: 20 stale cancelled runs plus one stale gate failure
alongside 80 green checks; each needed a manual merge).

Collapse the rollup to the most recent entry per check name/context
(completedAt, else startedAt/createdAt) before classifying, mirroring
GitHub's own latest-per-name mergeability semantics. A genuinely failing
or pending latest run still blocks; regression-tested both ways in the
fake-gh harness, which now takes an injectable rollup via PR_ROLLUP.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P3JtQ3ukoVmfg6LCKcKBBh

* fix(preflight): keep independent checks separate; handle gh zero-time exports

Cross-vendor review findings on the latest-per-name dedupe:

- Group by (__typename, workflowName, name/context), not name alone. A
  commit status and a check run sharing a name are independent required
  gates, and identically named jobs in different workflows must not
  collapse into one entry where a green run could mask the other
  workflow's red one.
- gh exports absent CheckRun times as the Go zero time (0001-01-01...),
  which defeated the // fallback chain. Normalize zero/null times and
  pick the max of completed/started/created, so a fresh in-progress
  rerun supersedes an old completed failure and reports as pending.

Regression tests for all three cases.

Agent-Signature: claude-fable-5-high on behalf of maphew
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P3JtQ3ukoVmfg6LCKcKBBh

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:04:38 -07:00