beads/internal/storage/schema/cli_migrations.go
Jim Wordelman ce70b261c5
storage: add participation_generation (migration 0070) and the versioned-history write fence (#6943)
* schema+storage: restore 0068, move participation_generation to 0070 (be-blvmg)

MAYOR RULING 2026-09-29 on tracker be-waare: migration 0068 shipped to
origin/main as attribution_status-only (be-v33pa). be-h89oq's lineage had
appended participation_generation steps 4-5 to that same file while it was
still unmerged, which check-migration-hygiene.sh Check C correctly rejects
once 0068 is frozen on main. This restores 0068 to origin/main's exact
content and re-homes the participation_generation column (issues + wisps,
design section 16.3 steps 4-5, be-dt74u amendment) in a new migration 0070,
with its clone-local ignored/0028 twin for wisps. Ports back the
participation_generation write-fence consumer code (RecordVersionInTx in
internal/storage/issueops/version_history.go and its callers) and the
dualwrite contract tests that exercise it.

RED: the CLI-bundle direct-DDL override for 0070's guarded PREPARE/ALTER
pair is not yet registered in cli_migrations.go, so
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities fail --
correctly: a fresh CLI-built bundle would silently omit the column
(dolthub/dolt#11345) while a runtime-migrated store has it.

git diff origin/main -- internal/storage/schema/migrations/0068_*.sql is
empty, confirming 0068 is untouched.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* schema: register CLI-bundle direct-DDL override for migration 0070 (be-blvmg)

GREEN: adds cliMigration0070AddParticipationGeneration and its dispatch
entries (cliCompatibleMigrationSQL, cliSubstituteAssumesWispTables), the
same pre-2.3 Dolt CLI PREPARE/ALTER escape hatch 0067's wisps.current_revision
and 0068's attribution_status already use (dolthub/dolt#11345). Makes
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities pass: a
fresh CLI-built bundle now carries participation_generation on both
issues and wisps, matching what a runtime-migrated store gets.

scripts/check-migration-hygiene.sh checks A-D pass; 0068 remains
byte-identical to origin/main.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* build: bazel-sync BUILD.bazel for migration 0070 files (be-blvmg)

make bazel-sync (gazelle) picked up two files the RED commit added that
the hand-ported BUILD.bazel hadn't listed yet: 0070's embedded .up.sql in
the go_library embedsrcs and migration_0070_add_participation_generation_test.go
in the go_test srcs.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* test(feat): red — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3) (refs be-fyaja)

* feat: green — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3)

(refs be-fyaja)

* test(conformance): pin that a wisp reaching the issues plane begins its history (be-31mnz)

Review feedback on #6943, blocker 1: with the flag on, a promoted wisp is
never versioned. None of the five RunParticipationGeneration* cases reaches
a promotion, so nothing caught it.

Two contract cases, wired on all three legs:

- RunParticipationGenerationWispPromotionBeginsHistory: create a wisp with
  the flag on, promote it, and require a stamped participation_generation,
  one version row, and +1 on the next update.
- RunParticipationGenerationWispPersistenceMoveBeginsHistory: the same
  check for a persistence-mode update (bd update --persistent). This mints
  at a different site than promotion on every leg: the guarded update's one
  end-of-update mint on dolt and embeddeddolt, and
  MoveIssuePersistenceInTx's own on uow.

"Wisp" is in the names because this contract already says "promoted" in
R2.3's sense (ProceedsForPromotedRecord), which is a different operation.
The fixture gains CreateWisp, PromoteWisp and MovePersistent closures, and
each leg's FixtureKitIsWired checks them.

Red at this commit on the embedded leg: after the arrival there are 0 rows
and participation_generation is NULL, and after the update there are still
0 rows. A flag-on durable create goes 1 -> 2 in the same store.

* fix(issueops): mint a wisp's first issues-plane version create-shaped (be-31mnz)

Review feedback on #6943, blocker 1. A promotion, or a persistence move out
of the wisps table, inserts a new issues row. A wisp never declares
participation (FR-8), so that row's participation_generation is NULL. Both
sites minted through the update-shaped RecordVersionInTx, and the section
16.2b fence read the NULL as legacy and skipped the mint. Nothing stamps the
column later, so every later update skipped too, and a record created with
versioned history on was never versioned.

These sites now use RecordVersionForCreateInTx, which stamps the column and
mints version 1, as a create on the issues plane does:

- PromoteFromEphemeralInTx (promote.go).
- MoveIssuePersistenceInTx's move out of the wisps table.
- ExecuteUpdate's one end-of-update mint (execution.go), when its
  persistence patch did that move. This is the site bd update --persistent
  reaches on the dolt and embeddeddolt legs, and the review did not list
  it. PersistenceMoveResult.EnteredIssuesPlane tells ExecuteUpdate which
  entry point to use.

One site the review named stays update-shaped on purpose:
MoveIssuePersistenceInTx's same-plane repair, which clears a stray wisp
flag on a row already in issues. That row keeps its own
participation_generation. A row created with the flag on still mints. A
legacy row stays legacy, because a repair is an ordinary write and R2.3
says an ordinary write does not promote it. A create-shaped mint there
would stamp the legacy row and start versioning it.

* docs(schema): 0070 must precede turning versioned history on (be-31mnz)

Review note on #6943 (schema skew). The write fence reads
participation_generation on every flag-on mint of an issues-plane row. On a
store that has not applied 0070 the read fails, and the mutation fails with
it instead of skipping. The migration header now says so, and says that the
flag-off path (the default) returns before the read, so a store below 0070
writes exactly as before.

Comment only. 0070 is not on main, so check-migration-hygiene's frozen-file
check does not apply to it, and no test pins its bytes.

* test(dolt): a persistence repair keeps the row's own participation (be-31mnz)

Pins why MoveIssuePersistenceInTx's same-plane repair stays update-shaped
while promotion and a move out of the wisps table became create-shaped (the
previous commit). The repair clears a stray wisp flag on a row that already
lives in issues, so the row keeps its own participation_generation.

TestPersistenceRepairKeepsTheRowsOwnParticipation runs the repair with
versioned history on for a legacy row (created with the flag off) and a
participating row (created with it on), through both repair entry points:
MoveIssuePersistenceInTx and a guarded update with a persistence patch. They
mint at different sites. The legacy row must stay NULL and mint nothing,
since R2.3 says an ordinary write does not promote it. The participating row
must mint one version. participation_generation must not change in either
case.

It fails if the repair mint is create-shaped, which would stamp and start
versioning the legacy row.

* test(dolt): a delete's rewrite of a legacy neighbor mints nothing (be-31mnz)

Folded in from bee's R1 on #6961, note 1. Once
TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit started creating
its neighbor with history on, nothing covered a delete whose neighbor was
created with history off. Under the section 16.2b fence, the expected result
for that case is different: the rewrite still runs, but it mints nothing.

TestDualWriteDeleteSkipsALegacyNeighborsRewrite creates the fixtures with
history off, turns history on, and force-deletes the cited target. It then
checks four things: the rewrite ran (ReferencesUpdated=1), the neighbor has
no issue_versions rows, its participation_generation is still NULL, and
dolt_status is clean for issue_versions, store_epoch and issues.

* test(embeddeddolt): batch-created rows are stamped with the participation generation

A batch create writes its rows first and mints each created row's first
version last, after the creation-time edges land. The mint has to stay
create-shaped for rows the batch inserted (stamping participation_generation
from the store epoch) and update-shaped for rows it only upserted over
(skipped while the row is a legacy row), so the per-row isNew has to survive
the deferral to the batch's final mint pass.

Pin that on the embedded engine:

- brand-new explicit-id rows are stamped with the store epoch and get one
  version row each, and match the same input run one row at a time with the
  batch-create fast paths off (5 rows, and 250 rows across three INSERT
  statements);
- a legacy row the batch upserts over keeps a NULL participation_generation
  and gets no version row, while the new rows beside it are stamped;
- with versioned history off, no row is stamped and no version row is written.

* ci(bazel): enroll the participation-generation tests in the embedded shards

The embedded-Dolt shard manifest has to name every top-level Test* in
internal/storage/embeddeddolt, or the exact-once coverage check fails.
The two participation-generation fence tests were written before that
check landed on main, and the batch-create stamping test is new, so none
of the three is listed yet. Add them to the 15-shard block as the
generator's incremental output: three inserted lines, no existing
assignment moves.

Also list the new batch-create test file in the embeddeddolt_test srcs,
in sorted position, so the Bazel target compiles it alongside the Go
build. Its imports are already among that target's deps.

Rebase note (maintainer): on current main the embedded storage manifest
uses a 40-shard block; the generator's incremental output placed these
three tests in shards 18, 19 and 39 and re-sorted one existing shard-12
line.

* fix(issueops): keep a created row's updated_at when its first version is minted

Minting a version is bookkeeping. It must not change a user-visible column
of the row it versions.

The create-shaped mint stamps participation_generation with an UPDATE of the
issues row. issues.updated_at is ON UPDATE CURRENT_TIMESTAMP, so an UPDATE
that changes any other column and does not name updated_at rewrites it with
the clock. Before the stamp, a row's first mint left current_revision at its
column default of 1, so the UPDATE changed nothing and the clause never
fired. The stamp moves participation_generation from NULL to the store
epoch, so the clause fires on every row whose first version is minted and
replaces the updated_at an import supplies.

Set updated_at to itself in the stamp, as the blocked-state recompute in
issueops/blocked_state.go already does, so the timestamp the row was created
with survives. Add a regression that batch-creates rows with a supplied
updated_at and requires both the stamp and the unchanged timestamp.

* test(createbatchequiv): re-record the digests the participation-generation fence moves

The fence does not re-mint rows that predate the stamp: a row whose
participation_generation is still NULL gets no new version when a batch
touches it. Main mints one, so six recorded entries move.

The issue_versions entry of five scenarios loses the versions main mints
for rows that predate the stamp: three rows in small, all eleven in
depadd, all fifteen in waitsfor, four in import458 and two in
import458-reject-stale. In depadd, eq-db1 and eq-dl are the only rows
main mints more than once, and each of those mints moves their updated_at
off the fixture date. The fence skips them, so the two rows keep the
fixture date and join the issues_updated_at entry.

No other table, list or counter digest changes, and the fast and per-row
bodies already agree in every scenario. The digests were recorded with
the embedded backend. The doc comment on GoldenDirEnv now says which
entries were re-recorded.

* test(storage): waive the participation-generation contracts on the http leg

main's #7247 (6d21d337ad, "a served http conformance leg") registered a
fourth leg, http, after this PR's original base. The PR's seven
RunParticipationGeneration* entrypoints are neither wired nor waived there,
so TestEveryLegWiresEveryRoleContract/http fails on the rebased branch.

ParticipationGenerationFixture toggles the store's own versioned-history
write flag and reads the raw participation_generation column back: store
write-fence bookkeeping (design §16.2b), not a publicops role the HTTP
client serves. Waive the seven cases on http with a named reason, the same
waiver the donnabox fork carries (09e7ce05af), and keep the gap visible
until a supported HTTP surface is designed and tested.

go test ./internal/storage/ -run TestEveryLegWiresEveryRoleContract now
passes on dolt, embeddeddolt, http and uow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): decide the fence's plane by the table that holds the row

recordVersionInTx chose the plane from the row's flags (IsWisp: Ephemeral
|| NoHistory) and then read participation_generation from the issues table
only. A wisps-TABLE row can carry neither flag, so it passed the IsWisp
return and the issues read failed with sql.ErrNoRows, aborting the whole
mutation whenever versioned history is on. Two producers:

  - bd mol squash clears an ephemeral root's flag in place
    (UpdateIssue {"wisp": false}, after the root's close, in the squash's
    one transaction); update.go and the uow repository's Update both mint
    after it unconditionally.
  - import's "wisp_plane" marker (WispPlaneOverride = &true, flags false)
    routes the row to wisps, and the batch create mints it create-shaped.

On origin/main the same paths do not fail but mint a stray issue_versions
row for a wisps-table row, an FR-8 leak (wisps are never versioned).

Fix: no issues row means nothing to version. ErrNoRows on the
participation_generation read now returns nil (no version, no error) for
update- and create-shaped calls alike. That read is the right signal and
IsActiveWispInTx is not: GetIssueInTx has already resolved the row, so
ErrNoRows on the issues read can only mean "not in issues", which is
exactly the question the fence has to answer before it stamps or advances
an issues row. It costs no extra query. IsActiveWispInTx asks the inverse
question ("is it in wisps?") with one more query per mint, and it folds
every error, including a transient one, into "not a wisp".

Tests come first, as three new conformance contracts with three new
ParticipationGenerationFixture hooks (ClearWispFlag, SquashWispRoot,
CreateUnflaggedWisp). They are wired on the dolt, embeddeddolt and uow legs
as subtests of each leg's TestParticipationGenerationFence (no new
top-level test, so the shard manifests do not change) and waived on http
with participationGenerationRawStoreWaiverReason:

  RunParticipationGenerationWispFlagClearMintsNothing      (create wisp,
      UpdateIssue {"wisp": false}: no error, 0 versions)
  RunParticipationGenerationWispRootSquashMintsNothing     (close +
      UpdateIssue {"wisp": false} in one transaction: no error, 0 versions)
  RunParticipationGenerationUnflaggedWispCreateMintsNothing (CreateIssues
      with WispPlaneOverride=&true, flags false: no error, 0 versions)

Red before the fix, green after, run both ways:
  embeddeddolt (BEADS_TEST_EMBEDDED_DOLT=1): all three FAIL with
    "versioned history: read participation_generation for pg-...: sql: no
    rows in result set"; all three PASS after.
  uow: all three FAIL with the same error; all three PASS after.
  dolt (BEADS_TEST_DOLT_SERVER=local): WispRootSquash and UnflaggedWispCreate
    FAIL with the same error; both PASS after. WispFlagClear passes both
    ways on this leg. The store-level wisp update (updateWisp) runs in a
    transaction versioned history never scopes, so it never reaches the
    mint. For the same reason the leg's SquashWispRoot hook turns the events
    journal on for its one transaction: that pins the wisp writes to the
    scoped regular transaction, as in a journal-enabled workspace, so that
    case does reach the seam.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): a create stamps participation even when the row is flagged no-history

A create-shaped mint (RecordVersionForCreateInTx) of a brand-new
ISSUES-TABLE row that carries NoHistory or Ephemeral returned at the IsWisp
check before the isCreate stamp, so participation_generation stayed NULL
although the row was created with versioned history on.

Import produces exactly that row: bd-r9uce keeps a promoted no-history
wisp's durable row on the issues plane with its stray no_history=true
(import_shared.go applyImportWispPlane: WispPlaneOverride = &false). When
bd update --persistent later repairs the flag, it is a same-plane normalize
whose mint is update-shaped by design (persistence.go: "A row created with
the flag on proceeds; a legacy row stays legacy"). The NULL column made the
fence read the row as legacy, so the repair and every later update were
fenced and the row was never versioned. origin/main versions it after the
repair (0, then 1, then 2 rows).

Fix: on a create-shaped call, a flagged row that the issues table holds
gets participation_generation stamped from store_epoch.epoch, the same
source the minting stamp uses, read in the same transaction. No version is
minted and updated_at is kept (SET ..., updated_at = updated_at), then the
call returns. Update-shaped calls on flagged rows still return before any
read. The table check from the previous commit now runs ahead of the IsWisp
branch for create-shaped calls, so a flagged wisps-table row (the common
wisp create) still mints and stamps nothing. That costs a create-shaped
call on a flagged wisp one primary-key miss on issues, and only while
versioned history is on. The store_epoch read/seed moves into
readStoreEpochInTx so both stamps share it.

Tests first:
  - New conformance contract
    RunParticipationGenerationNoHistoryIssueCreateDeclaresParticipation and
    hook CreateNoHistoryIssue. Wired on dolt, embeddeddolt and uow as a
    TestParticipationGenerationFence subtest and waived on http. With
    history on: create {NoHistory: true, WispPlaneOverride: &false} gives
    participation_generation non-NULL and 0 versions. The persistent repair
    (MovePersistent) then gives 1 version, and an update-shaped edit
    (MutateExisting, a title edit) gives 2.
  - New subtest
    TestCreateBatchParticipationGeneration/NoHistoryRowIsStampedUnversionedAndKeepsTheCallersUpdatedAt
    (embeddeddolt): a batch of such rows with a supplied updated_at comes
    out stamped, unversioned, and with updated_at unchanged.

Red before the fix, green after, run both ways:
  dolt (BEADS_TEST_DOLT_SERVER=local), embeddeddolt
  (BEADS_TEST_EMBEDDED_DOLT=1), uow: the contract FAILS on all three legs
    ("participation_generation for pg-no-history-create is NULL ...", then
    "= 0 after the persistent repair, want 1", then "went from 0 to 0 ...
    want exactly +1"). It PASSES on all three after.
  embeddeddolt subtest: FAILS before the fix ("store_epoch has no row after
    the batch, so nothing stamped the 5 rows"). With the fix minus the
    updated_at = updated_at clause it FAILS on every row's updated_at. It
    PASSES with the full fix.
No new top-level Test*, so the shard manifests do not change. The
createbatchequiv goldens are unchanged: the dolt and embeddeddolt
equivalence tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: fix comments the rebase left stale

Comment-only.

- createbatchequiv.GoldenDirEnv said "One deliberate behavior change since
  ... nothing else moved" and then listed a second re-record, the
  participation-generation fence's. It now says there are two deliberate
  changes, puts each in its own paragraph, and scopes "nothing else moved"
  to apply.json, the only golden the first change touched.
- issueops CreateIssueResult.isNew and deferredVersion still described one
  write path. After main's #7205, a row written by deferredCreates'
  multi-row INSERT reaches finishCreateIssueInTx(..., true) directly
  (create_batch_cache.go flush) and never passes through InsertIssueIfNew
  or CreateIssueInTxWithResult. isNew is InsertIssueIfNew's result on the
  per-row path and always true for a multi-row-INSERT row. deferredVersion
  is filled from both paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): pin the create/update shape split in the version guard

TestEveryVersionedEntryPointMints counts RecordVersionInTx and
RecordVersionForCreateInTx as one seam. A create-shaped site that fell back
to the update-shaped entry therefore stayed green, though the fence then
reads the new row's NULL participation_generation as legacy and the record
is never versioned. The reverse case also stayed green: an update path that
reached the create-shaped entry would stamp an existing legacy row and
promote it, which R2.3 forbids.

Add the split to the structural guard. It is two small tests over the same
journalscan fixpoint and mintVersion edge discipline, with no AST changes:

- TestCreateShapedEntryPointsMintCreateShaped: the create-shaped mint sites
  (finishCreateIssueInTx, CreateIssuesInTxWithContext,
  PromoteFromEphemeralInTx, moveIssuePersistenceInTx, executeUpdate) must
  each call RecordVersionForCreateInTx themselves. A delegate is not enough
  here, because executeUpdate runs its move with mintVersion=false and a
  batch switches finishCreateIssueInTx's mint off at run time. Every
  create-shaped entry point, the delegates and #7205's
  deferredCreates.flush included, must reach it.
- TestUpdateShapedEntryPointsNeverMintCreateShaped: every other
  versionedEntryPoint must not reach RecordVersionForCreateInTx. The
  exception is ApplyBatchInTx, which runs create and update items together
  and is listed as mixed with a reason.

Mutation-checked, each probe reverted afterwards. executeUpdate's
EnteredIssuesPlane mint, PromoteFromEphemeralInTx's mint, or the batch's
deferred create-shaped mint switched to RecordVersionInTx: each fails the
create half. update.go's end-of-update mint switched to
RecordVersionForCreateInTx: fails the update half for UpdateIssueInTx and
every entry point that writes through it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): hold the dual-shape mint sites to both halves of the split

f006c871c5 listed executeUpdate, moveIssuePersistenceInTx,
finishCreateIssueInTx and CreateIssuesInTxWithContext (and the
ExecuteUpdate/MoveIssuePersistenceInTx/create wrappers that delegate to
them) as create-shaped, so TestUpdateShapedEntryPointsNeverMintCreateShaped
skipped them. Each one also mints update-shaped, on its existing-row or
same-plane branch, which is the common path: a guarded update of an
existing row, the bd update --persistent repair of an issues-plane row's
flags, a create or batch create that resolved to a row already there.
Switching that branch to RecordVersionForCreateInTx stamped a legacy row
out of legacy (R2.3), and every guard stayed green.

Split the lists by what each function actually mints:

- versionCreateShapedMintSites keeps only the pure create-shaped site,
  PromoteFromEphemeralInTx. It must call RecordVersionForCreateInTx and
  must not call RecordVersionInTx.
- versionDualShapeMintSites names the four, each with its two branches.
  TestDualShapeMintSitesCallBothEntries requires each to call BOTH entries
  directly. A delegate does not count, for the same mintVersion=false and
  DeferVersionMint reasons as before.
- versionCreateShapedEntryPoints keeps PromoteFromEphemeralInTx and
  deferredCreates.flush, which passes isNew=true for every row.
- versionDualShapeEntryPoints replaces versionMixedShapeEntryPoints. It
  holds ApplyBatchInTx and the create, persistence-move and guarded-update
  entries that delegate to a dual-shape site, each with a reason. Each must
  reach both entries transitively.

The update half skips the create-shaped and dual-shape entry points, as
before. The update-shaped branch inside a dual-shape site is now the new
test's to pin.

Mutation-checked in a throwaway worktree. Switching the update-shaped mint
to RecordVersionForCreateInTx at execution.go:301, persistence.go:97,
create.go:260 or create.go:499 now fails
TestDualShapeMintSitesCallBothEntries for that site. Before this commit,
the whole issueops package stayed green under each of these switches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(httpclient): run the participation-generation fence over the served wire

edd3b72a1a waived all eleven ParticipationGeneration contracts on the http
leg, saying the client had "no write flag to toggle and no raw column to
read". That reason was wrong. served_dualwrite_history_test.go already
flips the reference store's VersionedHistoryConfigurer and reads its raw
columns, with the client driving the mutation. The waiver also hid the
one harness that runs the fence behind the update a connected workspace
actually sends. Every store leg's MutateExisting is store.UpdateIssue,
but the served Lifecycle.Update reaches the guarded update,
ExecuteUpdate.

Add served_participation_generation_test.go on the same pattern:

- Bound to the client: Mutate is Lifecycle.Create, MutateExisting is
  Lifecycle.Update, MovePersistent is the persistence-mode
  Lifecycle.Update that bd update --persistent sends, and PromoteWisp is
  issueops.Promote over the client's Reader/Lifecycle/Commenter, the
  entry bd promote dials over http.
- Bound to the reference store, as before: SetFlag, the CreateWisp seed,
  and the raw reads.

TestServedParticipationGenerationFence runs seven cases:
CreateWithFlagOnStampsAValue, CreateWithFlagOffLeavesLegacyNull,
SkipsUpdateOnLegacyRecord, ProceedsForPromotedRecord,
FenceIgnoresSchemaSkewHatch, WispPromotionBeginsHistory and
WispPersistenceMoveBeginsHistory. All seven pass, and no case turned up a
server-side fence bug.

Four cases stay waived, now with a reason that names the actual wire gap
(participationGenerationWireGapWaiverReason):

- WispFlagClearMintsNothing: the in-place {"wisp": false} is an untyped
  UpdateIssue, and IssuePatch.Persistence moves the row instead.
- WispRootSquashMintsNothing: it needs RunInTransaction. That and
  UpdateIssue are both on the client's unsupported allowlist (D8).
- UnflaggedWispCreateMintsNothing and
  NoHistoryIssueCreateDeclaresParticipation: both need WispPlaneOverride,
  which every create shape on the wire drops
  (roleIgnoredCreateIssueMembers).

The new file is listed in httpclient_test's srcs. The served lane deals
unpinned tests round-robin, so no shard manifest names it.

Mutation-checked in a throwaway worktree. With execution.go:301 (the
guarded update's update-shaped mint) switched to
RecordVersionForCreateInTx, SkipsUpdateOnLegacyRecord and
FenceIgnoresSchemaSkewHatch fail here ("version row count for
hpg-legacy-skip went from 0 to 1"). The embeddeddolt and uow
Participation|DualWrite|Version suites stay green under the same probe.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): an update never stamps a legacy no-history row

7c00d65fa6 made the seam write participation_generation on a flagged
(NoHistory or Ephemeral) issues-table row when the call is create-shaped.
Only the early `if IsWisp(issue) && !isCreate { return nil }` keeps an
update-shaped call out of that branch, and nothing pinned it. The new
contract and its subtest covered only the positive, create-shaped side.
With the early return dropped, an ordinary update stamped a legacy
no-history row out of legacy (R2.3), and every suite stayed green.

Add RunParticipationGenerationLegacyNoHistoryIssueStaysLegacy beside
NoHistoryIssueCreateDeclaresParticipation as its negative twin:

1. Create the no-history issues-table row with the flag OFF.
2. Turn the flag on and run MutateExisting. participation_generation must
   stay NULL, with 0 versions and current_revision unchanged.
3. Run MovePersistent, the bd update --persistent repair that clears the
   flag in place. The repair is update-shaped too, so the same three
   things must still hold.

Wired on all four legs: dolt, embeddeddolt, uow and http. On http the
case exercises the client's Lifecycle.Update and persistence update.
Their precondition, the flag-off no-history create, is seeded through
the reference store, because no client create can land that row in the
issues table. That gap is also why the case whose SUBJECT is that create
stays waived on http.

Mutation-checked in a throwaway worktree, with the early return made
`if false && IsWisp(issue) && !isCreate`. The new case fails on all four
legs and is the only case that fails:
"participation_generation for pg-no-history-legacy = 1 after an
update-shaped mutation with the flag on, want NULL" (dolt server leg,
embeddeddolt, uow and http). At tip it passes on all four.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(issueops): say which table and which mint the fence comments mean

Comment and doc corrections. No code changes.

- RecordVersionForCreateInTx's doc comment covered a flagged issues-table
  row but not a wisps-table row. A row the wisps table holds mints and
  stamps nothing, whatever its flags say: there is no issues row to stamp
  (FR-8). That holds for an ordinary wisp create and for import's
  unflagged "wisp_plane" row.
- The no-history stamp branch said the bd update --persistent repair mints
  update-shaped "(MoveIssuePersistenceInTx)". That is true only on the
  unit-of-work path. On dolt and embeddeddolt the repair goes through the
  guarded update: executeUpdate runs moveIssuePersistenceInTx with
  mintVersion=false and mints once at its own end. The fixture's
  MovePersistent doc already says this.
- VERSIONED-BEADS-WRITE-PATHS.md row 13 still credited the whole wisp
  exclusion to `RecordVersionInTx returns on IsWisp` and called every case
  a NO-OP. After ec52d29030, the table holding the row decides its plane:
  only an update-shaped call on a flagged wisp returns on IsWisp. Every
  other wisps-table row, including an unflagged one, is excluded by the
  issues-table participation_generation read, which finds no row. After
  7c00d65fa6, a create-shaped call on a flagged issues-table row is
  stamped but not versioned, so the row says that too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): the wisps-table cases check the row is still a wisp

runWispTableRowMintsNothing asserted 0 versions but never checked its own
precondition, that the wisps table still holds the row. Suppose a step
moved the row onto the issues plane through an update-shaped mint. That
mint would hit the new row's NULL participation_generation and be fenced,
so the count stayed 0 and the case passed without saying anything about
a wisps-table row.

- Add an InWispsTable hook to ParticipationGenerationFixture and bind it
  on dolt, embeddeddolt and uow as a raw wisps-table count. The
  wisps-table body now Fatals if the row is not in the wisps table after
  its steps. Each leg's KitIsWired test checks the hook. On http it
  stays nil with the other wisps-table hooks, whose cases are waived
  there.
- The dolt ClearWispFlag hook now bites. A bare store.UpdateIssue on a
  wisp goes through updateWisp, whose transaction versioned history never
  scopes. So WispFlagClearMintsNothing passed on dolt with or without the
  ec52d29030 fix, as that commit's message admits. The hook now runs the
  lone flag clear in a RunInTransaction with the events journal on: the
  journal-enabled shape the SquashWispRoot hook already uses.
- Both hooks go through a new withEventsJournal helper. It restores the
  store's previous journal setting rather than unconditionally setting it
  to false afterwards, which assumed the store started with the journal
  off.

Mutation-checked in throwaway worktrees:
- With version_history.go at ec52d29030^, dolt's WispFlagClearMintsNothing
  now fails ("read participation_generation for pg-flag-clear: sql: no
  rows in result set"). It used to pass there.
- Embedded ClearWispFlag rerouted as a flag-off persistence move followed
  by a flag-on title update: the case fails on the new precondition ("is
  not in the wisps table after flag-clear"). With the precondition
  disabled, the same probe passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(storage): review fixups for the participation_generation fence

The review of d17b716115 approved the PR with no required changes and
raised four minors and three nits. Each one below says what changed and
why, or why it did not.

Minor 1, the upsert create's new-or-existing choice was untested. The
unit-of-work create that is not CreateOnly (bd batch, bd todo, tracker
sync and mol port) inserts through an upsert, and db.Insert's existed
probe picks the mint's shape; forcing it either way shipped green. The
new uow TestParticipationGenerationUpsertCreate pins all three outcomes:
a new row is stamped and begins its history, a legacy row the upsert
resolves to stays legacy with no version, and a participating row mints
exactly one more. No served-route case is added because every served
create is CreateOnly (uow createParams), so the http wire cannot reach
this branch. The cmd/bd proxied-server batch and todo creates call this
same use-case method.

Minor 2, an update-shaped mint still lets issues.updated_at move
through ON UPDATE CURRENT_TIMESTAMP. Not changed here. The maintainer
listed it on the PR as follow-up (2), and the approving review of
d17b716115 accepted carrying it as a follow-up. It does not change
the fence's contract as pinned.

Minor 3, the depadd and waitsfor goldens had been re-recorded to zero
issue_versions rows. Their seeds run with history off, so every seeded
row was legacy and the harness stopped checking the dependency-add,
close and reopen mints. Both scenarios now stamp their seeded
issues-table rows after the seed (stampParticipation; wisps stay
unstamped). Their goldens are back to main's pre-fence files byte for
byte: depadd mints 11 version rows and waitsfor 15. The fenced legacy
outcome stays pinned by small, import458 and import458-reject-stale. A
run scenario that mints no version row now fails rather than recording
an empty digest.

Minor 4, PromoteFromEphemeralInTx always minted create-shaped. When the
issues plane already held the id (the dual presence GH#4455 records),
its upsert resolved to that row and the create-shaped mint stamped a
legacy record (R2.3). It now keeps InsertIssueIfNew's isNew and mints
update-shaped for an existing row, as finishCreateIssueInTx does. The
version guard lists it as a dual-shape mint site and entry point, and
PromoteOverDualPresenceKeepsLegacy pins the behavior at run time.

Nit 1: with versioned history off, db.Insert skips the existed probe,
since both mint entries return before reading anything. A create costs
the round trips it did before the fence.

Nit 2: the MINT COST comment now counts the fence read (about 6 to 7
round trips) and no longer says the snapshot reads current_revision;
the fence read can select it for #6379's ordinal seed. Moving that read
now would only add a column nothing consumes yet.

Nit 3: the http leg's participation-generation wire-gap waiver opens as
PENDING and names the wire mutation that would retire each case.

Validation, in a scrubbed env with -tags=gms_pure_go on embedded and
server Dolt (all pass, none skipped):
- uow: TestParticipationGenerationFence, its FixtureKitIsWired check
  and TestParticipationGenerationUpsertCreate with its 4 subtests.
- embeddeddolt: the create-batch equivalence parts (small/depadd,
  apply/waitsfor), the parts-coverage guard, the large scenarios, and
  the fence and kit.
- dolt: TestCreateBatchFastPathsMatchPerRow_Dolt, the fence,
  FenceIgnoresSchemaSkewHatch and the kit.
- httpclient: TestServedParticipationGenerationFence.
- TestEveryLegWiresEveryRoleContract, the issueops structural guards,
  and the issueops, storage and domain/db packages.

Eleven mutants, graded in 13 runs, each fail the test meant to catch
them:
- upsert always create-shaped or always update-shaped;
- the probe guard inverted;
- promote always create-shaped or always update-shaped;
- the seed left unstamped;
- the dependency-add, close or reopen mint dropped;
- two on-disk single-entry promotes.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): a wisp upsert must not stamp a legacy issues twin

domain/db's Insert chose the mint shape of its upsert (the create that
is not CreateOnly) from an existence probe of the target table. For a
wisps-table insert that probe asks the wrong plane: the seam snapshots
the issues table first (GetIssueInTx), so when the issues plane already
held the id as a legacy row, the wisp create took the create-shaped
mint and stamped and versioned that untouched twin. An ordinary wisp
write promoted a legacy record, which R2.3 forbids. bd mol port and the
proxied mol and wisp writers call CreateWisp this way. It is latent
until versioned history is enabled (#6661).

A wisps-table insert never creates an issues row, so it now mints
update-shaped, without the probe. The fence leaves a legacy twin alone,
a wisp-only id still mints nothing through either entry (FR-8), and a
participating twin mints the one version main already minted for it.
Insert's comment now records that last case: the dual presence
predates the fence, and this upsert does not refuse it the way
issueops' create path does.

TestParticipationGenerationUpsertCreate gains
WispOverLegacyIssuesRowKeepsItLegacy, which fails against the previous
dispatch (twin stamped, one version row). Also from review: the version
shape guard names the dual-shape site its parse cannot see, and the
row-lock exemption comment counts all three of the seam's UPDATE issues
statements.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
2026-10-18 13:10:57 +00:00

516 lines
27 KiB
Go

package schema
// cliCompatibleMigrationSQL returns migration SQL suitable for `dolt sql -q`
// against a fresh test database. The Dolt CLI's batch execution path
// (`dolt sql -q`/`-f`, which is what AllMigrationsSQL below feeds) silently
// no-ops a PREPARE/EXECUTE statement whose prepared text is DML — UPDATE,
// INSERT, or DELETE built into a `SET @sql = '...'` string and run via
// PREPARE ... FROM @sql; EXECUTE stmt — while EXECUTE reports success and a
// prepared SELECT or a direct (non-prepared) statement on the same path
// executes correctly. This is dolthub/dolt#11345, verified on dolt 2.2.0 and
// 2.2.2. Prepared ALTER TABLE is the same underlying limitation and one
// instance of it, not the whole scope: the fresh-schema bundle uses direct
// DDL below wherever a source migration guards an ALTER with PREPARE for
// idempotent re-runs. The bundle's contract is to reproduce the runtime
// committed schema for a fresh database; runtime migrations still use the
// source files and remain the source of truth for upgrades of existing
// databases, where PREPARE/EXECUTE runs over a real driver connection and
// this limitation does not apply.
//
// Dolt 2.3.0 fixed #11345, and that fix MASKS a missing override here rather
// than removing the need for one: on 2.3.x the CLI executes the prepared
// ALTER, so a bundle that forgot the direct statement still lands the right
// schema and TestCLIBundleMatchesRuntimeCommittedSchema goes green against a
// 2.2.0 sql-server. So every override below must be justified against the
// pre-2.3 behavior, and the string assertions in
// TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities — which need
// no Dolt binary at all — are the guard that survives a CLI version bump.
//
// The measurement that check stands in for: run AllMigrationsSQL() through
// each CLI and diff information_schema. On 2026-08-21, with 0060's and 0065's
// overrides both in place, dolt 2.1.8, 2.2.0 (the pinned CLI) and 2.3.1 agree
// on all 504 snapshot lines including the wisp_ tables the parity oracle
// excludes — so no prepared statement the bundle still carries changes its
// committed schema. Before that, 2.3.1 landed three extra columns (0060's
// storage_class, gastownhall/beads#5903) and one extra widening (0065's
// wisp_comments.text). cli_prepared_ddl.go is what keeps a new migration from
// re-opening that gap.
//
// Every override below is written for the fresh bundle, where the whole main
// series has run in order. A caller replaying the substitution over a drifted
// database — one that never synced the clone-local wisp tables, say — must
// consult cliSubstituteAssumesWispTables first and fall back to the frozen
// source text, whose own PREPARE guards cover the absent-table case.
//
// For a migration whose PREPARE'd DML matters on this path (not just DDL),
// the fix is not a direct-SQL override here — it is to not depend on
// PREPARE'd writes to real tables in the source migration at all. Migration
// 0059 (gastownhall/beads#4877) is the pattern: real-table mutations are
// direct SQL; the only PREPARE'd statements are best-effort, guarded copies
// into throwaway stand-in tables that a direct statement then reads, so a
// silent no-op there degrades gracefully instead of corrupting state.
// scripts/check-migration-hygiene.sh flags new migrations that use PREPARE'd
// DML instead of that pattern.
func cliCompatibleMigrationSQL(name, sqlText string) string {
switch name {
case "0008_create_child_counters.up.sql":
// Fresh bundle bakes the final FK shape that runtime reaches after
// 0039 drops the original FK and ignored migration 0002 re-adds it.
return cliMigration0008CreateChildCounters
case "0023_add_no_history_column.up.sql":
return cliMigration0023AddNoHistoryColumn
case "0027_add_started_at.up.sql":
return cliMigration0027AddStartedAt
case "0032_drop_schema_migrations_applied_at.up.sql":
return cliMigration0032DropSchemaMigrationsAppliedAt
case "0033_add_wisp_type_column.up.sql":
// No-op on fresh schema: wisp_type is already in squashed base 0001.
return "SELECT 1;"
case "0034_add_spec_id_column.up.sql":
// No-op on fresh schema: spec_id and idx_issues_spec_id are in 0001.
return "SELECT 1;"
case "0039_drop_child_counters_fk.up.sql":
// No-op here because 0008 already emits the final ignored-0002 FK.
return "SELECT 1;"
case "0041_split_dependencies_target.up.sql":
return cliMigration0041SplitDependenciesTarget
case "0043_drop_dependencies_generated_column.up.sql":
return cliMigration0043DropDependenciesGeneratedColumn
case "0046_add_is_blocked.up.sql":
// Fresh databases contain no issue graph, so only the schema delta is
// needed; the source migration's recursive backfill is dead work here.
return cliMigration0046AddIsBlocked
case "0049_longtext_large_content_columns.up.sql":
return cliMigration0049LongtextLargeContentColumns
case "0051_drop_aux_id_defaults.up.sql":
// Direct DDL: the source migration's PREPARE/EXECUTE guards exist for
// re-run safety on upgraded databases; a fresh bundle always has the
// 0004/0005/0009/0010 defaults to drop.
return cliMigration0051DropAuxIDDefaults
case "0053_repair_rig_wisps.up.sql":
// The source migration uses PREPARE guards so older upgraded
// workspaces without local wisp tables can no-op safely. Fresh CLI
// bundles already have the base wisp tables, and the Dolt CLI test
// path needs direct DML for deterministic fixture repair.
return cliMigration0053RepairRigWisps
case "0054_add_lease_columns.up.sql":
// Fresh bundle bakes the lease columns directly: the Dolt CLI does not
// apply the prepared ALTER TABLE statements the runtime migration uses
// for idempotent re-runs on upgraded databases.
return cliMigration0054AddLeaseColumns
case "0055_move_leases_to_table.up.sql":
// Direct DDL for the same reason as 0054. A fresh bundle has no live
// leases to copy (0054 just added empty columns), so this is pure
// schema delta: create the ephemeral leases table, drop the issues/
// wisps lease columns 0054 added. row_lock stays (see the migration).
return cliMigration0055MoveLeasesToTable
case "0060_add_storage_class.up.sql":
// Direct DDL for the same reason as 0054: the source migration's
// PREPARE guards make it idempotent on upgraded databases, and a
// fresh bundle always needs the column on both planes.
return cliMigration0060AddStorageClass
case "0065_widen_wisp_comments_text.up.sql":
// Direct DDL for the same reason as 0060. wisp_comments is created
// by main-plane 0021, so a fresh bundle always has the table and
// always needs the widening. Replays over a database that never
// synced the wisp tables must use the frozen source text instead --
// see cliSubstituteAssumesWispTables.
return cliMigration0065WidenWispCommentsText
case "0066_add_events_journal_actor.up.sql":
// Direct DDL for the same reason as 0060: the source migration's
// PREPARE guard (INFORMATION_SCHEMA probe) makes it idempotent on
// upgraded databases, and the 2.2.x CLI no-ops the prepared ALTER
// (measured 2026-08-20: AllMigrationsSQL() through dolt 2.2.0 lands
// bd_events_journal WITHOUT actor; 2.3.1 — and 2.2.3, which already
// carries the #11345 fix — land it; CI pins 2.2.0). The table
// itself is always present here: 0064's prepared RENAME executes on
// 2.2.0 (same measurement), so a fresh bundle always needs the column.
return cliMigration0066AddEventsJournalActor
case "0067_add_versioned_beads_schema.up.sql":
// Direct DDL for the same reason as 0060: the source migration's
// PREPARE guards (INFORMATION_SCHEMA probes) are what make the raw
// .up.sql idempotent when replayed onto an already-migrated store,
// and the 2.2.x CLI no-ops a prepared ADD COLUMN. Both planes always
// need the column here -- a fresh bundle runs the whole main series
// in order, so `issues` and `wisps` both exist and neither carries
// current_revision yet. The CREATE TABLEs are carried over verbatim:
// IF NOT EXISTS is ordinary unwrapped DDL that the CLI executes.
// Replays over a database that never synced the wisp tables must use
// the frozen source text instead -- see cliSubstituteAssumesWispTables.
return cliMigration0067AddVersionedBeadsSchema
case "0068_add_attribution_status.up.sql":
// Direct DDL for the same reason as 0067: the source migration's
// PREPARE guards (INFORMATION_SCHEMA probes) are what make the raw
// .up.sql idempotent when replayed onto an already-migrated store,
// and the 2.2.x CLI no-ops a prepared ADD COLUMN and MODIFY COLUMN
// alike. issue_versions is always present here -- 0067 runs earlier
// in the same fresh-bundle series -- never carries attribution_status
// yet, and still has durable_state as the JSON type 0067 gave it, so
// both of 0068's steps always fire on a fresh bundle.
return cliMigration0068AddAttributionStatus
case "0069_widen_issue_versions_datetime_precision.up.sql":
// Direct DDL for the same reason as 0068: the source migration's
// PREPARE guards (DATETIME_PRECISION probes) are what make the raw
// .up.sql idempotent when replayed onto an already-widened store,
// and the 2.2.x CLI no-ops a prepared MODIFY COLUMN. Nothing before
// 0069 in the fresh-bundle series widens change_at or removed_at --
// 0067 creates both as plain DATETIME and 0068 retypes only
// durable_state -- so both of 0069's MODIFYs always fire here too.
return cliMigration0069WidenIssueVersionsDatetimePrecision
case "0070_add_participation_generation.up.sql":
// Direct DDL for the same reason as 0067: the source migration's
// PREPARE guards (INFORMATION_SCHEMA probes) are what make the raw
// .up.sql idempotent when replayed onto an already-migrated store,
// and the 2.2.x CLI no-ops a prepared ADD COLUMN. issues and wisps
// both exist by this point in the series (0067 creates the versioned
// columns on both) and neither carries participation_generation yet,
// so both of 0070's ALTERs always fire on a fresh bundle -- this
// substitute ALTERs wisps directly, the same as 0067's, so it
// belongs on cliSubstituteAssumesWispTables too.
return cliMigration0070AddParticipationGeneration
default:
return sqlText
}
}
// cliSubstituteAssumesWispTables reports whether cliCompatibleMigrationSQL's
// substitute for name presumes the clone-local wisp_* tables already exist.
//
// AllMigrationsSQL() always satisfies that presumption — the main series
// creates every wisp table on its way past — so the bundle route is
// unaffected. A replay over a drifted database is not: #4695/#4176 is the
// shape where the main cursor arrives at-latest with the wisp tables never
// synced (they are dolt_ignored, so a clone can simply not have them), and
// 0047's repair recreates only wisps and wisp_dependencies. On that database
// the substitute's direct DDL aborts the batch with `table not found` while
// the frozen source text's own PREPARE guards correctly no-op: an
// INFORMATION_SCHEMA probe for a missing table yields NULL, and
// `IF(NULL = 1, '<ddl>', 'SELECT 1')` takes the no-op branch.
//
// So a replay caller must use the frozen text for these, and only these.
// The other substitutes are safe to replay because they touch main-plane
// tables that are always present.
func cliSubstituteAssumesWispTables(name string) bool {
switch name {
case "0053_repair_rig_wisps.up.sql":
// cliMigration0053RepairRigWisps drops every @has_wisp_* guard and
// reads all five wisp tables unconditionally.
return true
case "0065_widen_wisp_comments_text.up.sql":
// cliMigration0065WidenWispCommentsText is a bare MODIFY on
// wisp_comments.
return true
case "0067_add_versioned_beads_schema.up.sql":
// cliMigration0067AddVersionedBeadsSchema drops the source's
// @wisps_cr_needs_add table-exists guard and ALTERs wisps directly.
return true
case "0070_add_participation_generation.up.sql":
// cliMigration0070AddParticipationGeneration drops the source's
// @wisps_pg_needs_add table-exists guard and ALTERs wisps directly,
// the same shape 0067's override already uses for current_revision.
return true
default:
return false
}
}
const cliMigration0008CreateChildCounters = `CREATE TABLE IF NOT EXISTS child_counters (
parent_id VARCHAR(255) PRIMARY KEY,
last_child INT NOT NULL DEFAULT 0,
CONSTRAINT fk_counter_parent FOREIGN KEY (parent_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE
);`
const cliMigration0023AddNoHistoryColumn = `ALTER TABLE issues ADD COLUMN no_history TINYINT(1) DEFAULT 0;
ALTER TABLE wisps ADD COLUMN no_history TINYINT(1) DEFAULT 0;`
const cliMigration0027AddStartedAt = `ALTER TABLE issues ADD COLUMN started_at DATETIME;
ALTER TABLE wisps ADD COLUMN started_at DATETIME;`
const cliMigration0032DropSchemaMigrationsAppliedAt = `ALTER TABLE schema_migrations DROP COLUMN applied_at;`
const cliMigration0054AddLeaseColumns = `ALTER TABLE issues ADD COLUMN lease_expires_at DATETIME;
ALTER TABLE issues ADD COLUMN heartbeat_at DATETIME;
ALTER TABLE issues ADD COLUMN row_lock BIGINT NOT NULL DEFAULT 0;
CREATE INDEX idx_issues_lease ON issues (status, lease_expires_at);
ALTER TABLE wisps ADD COLUMN lease_expires_at DATETIME;
ALTER TABLE wisps ADD COLUMN heartbeat_at DATETIME;
ALTER TABLE wisps ADD COLUMN row_lock BIGINT NOT NULL DEFAULT 0;`
const cliMigration0055MoveLeasesToTable = `CREATE TABLE leases (
issue_id VARCHAR(255) PRIMARY KEY,
holder VARCHAR(255) NOT NULL,
granted_at DATETIME NOT NULL,
lease_expires_at DATETIME NOT NULL,
heartbeat_at DATETIME NOT NULL,
INDEX idx_leases_expires (lease_expires_at)
);
ALTER TABLE issues DROP INDEX idx_issues_lease;
ALTER TABLE issues DROP COLUMN lease_expires_at;
ALTER TABLE issues DROP COLUMN heartbeat_at;
ALTER TABLE wisps DROP COLUMN lease_expires_at;
ALTER TABLE wisps DROP COLUMN heartbeat_at;`
const cliMigration0060AddStorageClass = `ALTER TABLE issues ADD COLUMN storage_class VARCHAR(16);
ALTER TABLE wisps ADD COLUMN storage_class VARCHAR(16);`
const cliMigration0065WidenWispCommentsText = `ALTER TABLE wisp_comments MODIFY COLUMN text LONGTEXT NOT NULL;`
const cliMigration0066AddEventsJournalActor = `ALTER TABLE bd_events_journal ADD COLUMN actor VARCHAR(255) NOT NULL DEFAULT '';`
// cliMigration0067AddVersionedBeadsSchema is 0067 with its two guarded
// PREPARE blocks replaced by the direct ALTERs they would run on a fresh
// database. The CREATE TABLEs are the source file's own text: CREATE TABLE
// IF NOT EXISTS executes on the CLI batch path unchanged.
const cliMigration0067AddVersionedBeadsSchema = `CREATE TABLE IF NOT EXISTS issue_versions (
issue_id VARCHAR(255) NOT NULL,
revision BIGINT NOT NULL,
epoch INT NOT NULL,
durable_state JSON,
change_actor VARCHAR(255),
change_agent VARCHAR(255),
change_message TEXT,
change_at DATETIME NOT NULL,
removed_at DATETIME,
removed_reason VARCHAR(255),
PRIMARY KEY (issue_id, revision)
);
CREATE TABLE IF NOT EXISTS store_epoch (
id TINYINT(1) NOT NULL DEFAULT 1,
epoch INT NOT NULL DEFAULT 1,
bumped_at DATETIME,
bumped_reason VARCHAR(255),
PRIMARY KEY (id),
CONSTRAINT ck_store_epoch_singleton CHECK (id = 1)
);
ALTER TABLE issues ADD COLUMN current_revision BIGINT NOT NULL DEFAULT 1;
ALTER TABLE wisps ADD COLUMN current_revision BIGINT NOT NULL DEFAULT 1;`
// cliMigration0068AddAttributionStatus is 0068 with its two guarded PREPARE
// blocks replaced by the direct ALTERs they would run on a fresh database:
// step 6's ADD COLUMN attribution_status, and step 7's MODIFY COLUMN
// durable_state LONGBLOB (the byte-preserving type the review on
// gastownhall/beads#6358 item 4 asked for -- see the migration's step 7
// header). issue_versions is created earlier in the same series by 0067,
// which still creates durable_state as JSON and whose override is left
// untouched: 0068 is what retypes it. So the column always needs adding and
// the retype always fires here; no wisps twin exists for this table.
const cliMigration0068AddAttributionStatus = `ALTER TABLE issue_versions ADD COLUMN attribution_status VARCHAR(20) NOT NULL;
ALTER TABLE issue_versions MODIFY COLUMN durable_state LONGBLOB;`
// cliMigration0069WidenIssueVersionsDatetimePrecision is 0069 with its two
// guarded PREPARE blocks replaced by the direct MODIFYs they would run on a
// fresh database: change_at and removed_at from 0067's plain DATETIME to
// DATETIME(6) (the microsecond widen be-hs42e.8 / gastownhall/beads#6132
// asks for -- see the migration's header). Neither 0067's override nor
// 0068's touches either column, so both retypes always fire here; no wisps
// twin exists for this table.
const cliMigration0069WidenIssueVersionsDatetimePrecision = `ALTER TABLE issue_versions MODIFY COLUMN change_at DATETIME(6) NOT NULL;
ALTER TABLE issue_versions MODIFY COLUMN removed_at DATETIME(6);`
// cliMigration0070AddParticipationGeneration is 0070 with its two guarded
// PREPARE blocks replaced by the direct ALTERs they would run on a fresh
// database: participation_generation BIGINT NULL on issues, mirrored
// inertly on wisps (design §16.3 steps 4-5, be-dt74u amendment, be-h89oq).
// issues and wisps both exist by this point in the series (0067 creates the
// versioned columns on both) and neither carries participation_generation
// yet, so both ALTERs always fire on a fresh bundle -- this substitute
// ALTERs wisps directly, the same as 0067's, so it belongs on
// cliSubstituteAssumesWispTables too.
const cliMigration0070AddParticipationGeneration = `ALTER TABLE issues ADD COLUMN participation_generation BIGINT NULL;
ALTER TABLE wisps ADD COLUMN participation_generation BIGINT NULL;`
const cliMigration0041SplitDependenciesTarget = `DELETE FROM dolt_nonlocal_tables;
CALL DOLT_COMMIT('-Am', 'disable nonlocal tables for fk migrations');
SET FOREIGN_KEY_CHECKS = 0;
ALTER TABLE dependencies ADD COLUMN depends_on_issue_id VARCHAR(255) NULL;
ALTER TABLE dependencies ADD COLUMN depends_on_wisp_id VARCHAR(255) NULL;
ALTER TABLE dependencies ADD COLUMN depends_on_external VARCHAR(255) NULL;
UPDATE dependencies SET depends_on_external = depends_on_id WHERE depends_on_id LIKE 'external:%';
UPDATE dependencies d JOIN wisps w ON w.id = d.depends_on_id SET d.depends_on_wisp_id = d.depends_on_id WHERE d.depends_on_external IS NULL;
UPDATE dependencies d JOIN issues i ON i.id = d.depends_on_id SET d.depends_on_issue_id = d.depends_on_id WHERE d.depends_on_external IS NULL AND d.depends_on_wisp_id IS NULL;
UPDATE dependencies SET depends_on_external = depends_on_id WHERE depends_on_external IS NULL AND depends_on_wisp_id IS NULL AND depends_on_issue_id IS NULL;
ALTER TABLE dependencies DROP INDEX idx_dependencies_depends_on;
ALTER TABLE dependencies DROP INDEX idx_dependencies_depends_on_type;
ALTER TABLE dependencies DROP PRIMARY KEY;
ALTER TABLE dependencies DROP COLUMN depends_on_id;
ALTER TABLE dependencies ADD CONSTRAINT fk_dep_issue_target FOREIGN KEY (depends_on_issue_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE dependencies ADD COLUMN depends_on_id VARCHAR(255) AS (COALESCE(depends_on_issue_id, depends_on_wisp_id, depends_on_external)) STORED;
ALTER TABLE dependencies ADD PRIMARY KEY (issue_id, depends_on_id);
ALTER TABLE dependencies ADD INDEX idx_dep_wisp_target (depends_on_wisp_id);
ALTER TABLE dependencies ADD INDEX idx_dep_issue_target (depends_on_issue_id);
ALTER TABLE dependencies ADD INDEX idx_dep_external_target (depends_on_external);
ALTER TABLE dependencies ADD INDEX idx_dep_type_target (type, depends_on_id);
ALTER TABLE dependencies ADD CONSTRAINT ck_dep_one_target CHECK ((depends_on_issue_id IS NOT NULL) + (depends_on_wisp_id IS NOT NULL) + (depends_on_external IS NOT NULL) = 1);
SET FOREIGN_KEY_CHECKS = 1;`
const cliMigration0043DropDependenciesGeneratedColumn = `SET FOREIGN_KEY_CHECKS = 0;
ALTER TABLE dependencies DROP INDEX idx_dep_type_target;
ALTER TABLE dependencies DROP FOREIGN KEY fk_dep_issue_target;
ALTER TABLE dependencies DROP FOREIGN KEY fk_dep_issue;
ALTER TABLE dependencies DROP PRIMARY KEY;
ALTER TABLE dependencies DROP COLUMN depends_on_id;
ALTER TABLE dependencies ADD COLUMN id CHAR(36) NOT NULL DEFAULT (UUID()) PRIMARY KEY FIRST;
ALTER TABLE dependencies ADD UNIQUE KEY uk_dep_issue_target (issue_id, depends_on_issue_id);
ALTER TABLE dependencies ADD UNIQUE KEY uk_dep_wisp_target (issue_id, depends_on_wisp_id);
ALTER TABLE dependencies ADD UNIQUE KEY uk_dep_external_target (issue_id, depends_on_external);
ALTER TABLE dependencies ADD INDEX idx_dep_type_issue (type, depends_on_issue_id);
ALTER TABLE dependencies ADD INDEX idx_dep_type_wisp (type, depends_on_wisp_id);
ALTER TABLE dependencies ADD INDEX idx_dep_type_external (type, depends_on_external);
ALTER TABLE dependencies ADD CONSTRAINT fk_dep_issue FOREIGN KEY (issue_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TABLE dependencies ADD CONSTRAINT fk_dep_issue_target FOREIGN KEY (depends_on_issue_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE;
SET FOREIGN_KEY_CHECKS = 1;`
const cliMigration0046AddIsBlocked = `ALTER TABLE issues ADD COLUMN is_blocked TINYINT(1) NOT NULL DEFAULT 0;
CREATE INDEX idx_issues_is_blocked ON issues(is_blocked, status);`
const cliMigration0049LongtextLargeContentColumns = `ALTER TABLE issues MODIFY COLUMN description LONGTEXT NOT NULL, MODIFY COLUMN design LONGTEXT NOT NULL, MODIFY COLUMN acceptance_criteria LONGTEXT NOT NULL, MODIFY COLUMN notes LONGTEXT NOT NULL;
ALTER TABLE issues MODIFY COLUMN close_reason LONGTEXT DEFAULT '';
ALTER TABLE wisps MODIFY COLUMN description LONGTEXT NOT NULL DEFAULT '', MODIFY COLUMN design LONGTEXT NOT NULL DEFAULT '', MODIFY COLUMN acceptance_criteria LONGTEXT NOT NULL DEFAULT '', MODIFY COLUMN notes LONGTEXT NOT NULL DEFAULT '';
ALTER TABLE wisps MODIFY COLUMN close_reason LONGTEXT DEFAULT '';
ALTER TABLE comments MODIFY COLUMN text LONGTEXT NOT NULL;`
const cliMigration0051DropAuxIDDefaults = `ALTER TABLE events ALTER COLUMN id DROP DEFAULT;
ALTER TABLE comments ALTER COLUMN id DROP DEFAULT;
ALTER TABLE issue_snapshots ALTER COLUMN id DROP DEFAULT;
ALTER TABLE compaction_snapshots ALTER COLUMN id DROP DEFAULT;`
const cliMigration0053RepairRigWisps = `SET FOREIGN_KEY_CHECKS = 0;
INSERT IGNORE INTO issues (
id, content_hash, title, description, design, acceptance_criteria, notes,
status, priority, issue_type, assignee, estimated_minutes, created_at,
created_by, owner, updated_at, closed_at, closed_by_session, external_ref,
spec_id, compaction_level, compacted_at, compacted_at_commit, original_size,
sender, ephemeral, wisp_type, pinned, is_template, mol_type, work_type,
source_system, metadata, source_repo, close_reason, event_kind, actor,
target, payload, await_type, await_id, timeout_ns, waiters, hook_bead,
role_bead, agent_state, last_activity, role_type, rig, due_at, defer_until,
no_history, started_at
)
SELECT
id, content_hash, title, description, design, acceptance_criteria, notes,
status, priority, issue_type, assignee, estimated_minutes, created_at,
created_by, owner, updated_at, closed_at, closed_by_session, external_ref,
spec_id, compaction_level, compacted_at, compacted_at_commit, original_size,
sender, ephemeral, wisp_type, pinned, is_template, mol_type, work_type,
source_system, metadata, source_repo, close_reason, event_kind, actor,
target, payload, await_type, await_id, timeout_ns, waiters, hook_bead,
role_bead, agent_state, last_activity, role_type, rig, due_at, defer_until,
no_history, started_at
FROM wisps
WHERE issue_type = 'rig';
UPDATE issues
SET ephemeral = 0
WHERE issue_type = 'rig'
AND id IN (SELECT id FROM wisps WHERE issue_type = 'rig');
INSERT IGNORE INTO labels (issue_id, label)
SELECT wl.issue_id, wl.label
FROM wisp_labels wl
JOIN wisps w ON w.id = wl.issue_id
WHERE w.issue_type = 'rig';
INSERT IGNORE INTO dependencies (
id, issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external,
type, created_at, created_by, metadata, thread_id
)
SELECT
CONCAT(
SUBSTR(MD5(CONCAT(wd.issue_id, CHAR(31), COALESCE(wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external))), 1, 8),
'-',
SUBSTR(MD5(CONCAT(wd.issue_id, CHAR(31), COALESCE(wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external))), 9, 4),
'-',
SUBSTR(MD5(CONCAT(wd.issue_id, CHAR(31), COALESCE(wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external))), 13, 4),
'-',
SUBSTR(MD5(CONCAT(wd.issue_id, CHAR(31), COALESCE(wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external))), 17, 4),
'-',
SUBSTR(MD5(CONCAT(wd.issue_id, CHAR(31), COALESCE(wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external))), 21, 12)
),
wd.issue_id, wd.depends_on_issue_id, wd.depends_on_wisp_id, wd.depends_on_external,
wd.type, wd.created_at, wd.created_by, wd.metadata, wd.thread_id
FROM wisp_dependencies wd
JOIN wisps w ON w.id = wd.issue_id
WHERE w.issue_type = 'rig';
INSERT IGNORE INTO events (id, issue_id, event_type, actor, old_value, new_value, comment, created_at)
SELECT we.id, we.issue_id, we.event_type, we.actor, we.old_value, we.new_value, we.comment, we.created_at
FROM wisp_events we
JOIN wisps w ON w.id = we.issue_id
WHERE w.issue_type = 'rig';
INSERT IGNORE INTO comments (id, issue_id, author, text, created_at)
SELECT wc.id, wc.issue_id, wc.author, wc.text, wc.created_at
FROM wisp_comments wc
JOIN wisps w ON w.id = wc.issue_id
WHERE w.issue_type = 'rig';
CREATE TABLE IF NOT EXISTS wisp_child_counters (
parent_id VARCHAR(255) PRIMARY KEY,
last_child INT NOT NULL DEFAULT 0
);
INSERT IGNORE INTO child_counters (parent_id, last_child)
SELECT wcc.parent_id, wcc.last_child
FROM wisp_child_counters wcc
JOIN wisps w ON w.id = wcc.parent_id
WHERE w.issue_type = 'rig';
UPDATE child_counters cc
JOIN wisp_child_counters wcc ON wcc.parent_id = cc.parent_id
JOIN wisps w ON w.id = wcc.parent_id
SET cc.last_child = GREATEST(cc.last_child, wcc.last_child)
WHERE w.issue_type = 'rig';
DELETE wd FROM wisp_dependencies wd
JOIN wisps w ON w.id = wd.issue_id
WHERE w.issue_type = 'rig';
REPLACE INTO dependencies (
id, issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external,
type, created_at, created_by, metadata, thread_id
)
SELECT
d.id, d.issue_id, d.depends_on_wisp_id, NULL, d.depends_on_external,
d.type, d.created_at, d.created_by, d.metadata, d.thread_id
FROM dependencies d
JOIN wisps w ON w.id = d.depends_on_wisp_id
WHERE w.issue_type = 'rig';
REPLACE INTO wisp_dependencies (
issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external,
type, created_at, created_by, metadata, thread_id
)
SELECT
wd.issue_id, wd.depends_on_wisp_id, NULL, wd.depends_on_external,
wd.type, wd.created_at, wd.created_by, wd.metadata, wd.thread_id
FROM wisp_dependencies wd
JOIN wisps w ON w.id = wd.depends_on_wisp_id
WHERE w.issue_type = 'rig';
DELETE wl FROM wisp_labels wl
JOIN wisps w ON w.id = wl.issue_id
WHERE w.issue_type = 'rig';
DELETE we FROM wisp_events we
JOIN wisps w ON w.id = we.issue_id
WHERE w.issue_type = 'rig';
DELETE wc FROM wisp_comments wc
JOIN wisps w ON w.id = wc.issue_id
WHERE w.issue_type = 'rig';
DELETE wcc FROM wisp_child_counters wcc
JOIN wisps w ON w.id = wcc.parent_id
WHERE w.issue_type = 'rig';
DELETE FROM wisps WHERE issue_type = 'rig';
SET FOREIGN_KEY_CHECKS = 1;`