beads/internal/storage/schema/schema_test.go
Jim Wordelman ce70b261c5
storage: add participation_generation (migration 0070) and the versioned-history write fence (#6943)
* schema+storage: restore 0068, move participation_generation to 0070 (be-blvmg)

MAYOR RULING 2026-09-29 on tracker be-waare: migration 0068 shipped to
origin/main as attribution_status-only (be-v33pa). be-h89oq's lineage had
appended participation_generation steps 4-5 to that same file while it was
still unmerged, which check-migration-hygiene.sh Check C correctly rejects
once 0068 is frozen on main. This restores 0068 to origin/main's exact
content and re-homes the participation_generation column (issues + wisps,
design section 16.3 steps 4-5, be-dt74u amendment) in a new migration 0070,
with its clone-local ignored/0028 twin for wisps. Ports back the
participation_generation write-fence consumer code (RecordVersionInTx in
internal/storage/issueops/version_history.go and its callers) and the
dualwrite contract tests that exercise it.

RED: the CLI-bundle direct-DDL override for 0070's guarded PREPARE/ALTER
pair is not yet registered in cli_migrations.go, so
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities fail --
correctly: a fresh CLI-built bundle would silently omit the column
(dolthub/dolt#11345) while a runtime-migrated store has it.

git diff origin/main -- internal/storage/schema/migrations/0068_*.sql is
empty, confirming 0068 is untouched.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* schema: register CLI-bundle direct-DDL override for migration 0070 (be-blvmg)

GREEN: adds cliMigration0070AddParticipationGeneration and its dispatch
entries (cliCompatibleMigrationSQL, cliSubstituteAssumesWispTables), the
same pre-2.3 Dolt CLI PREPARE/ALTER escape hatch 0067's wisps.current_revision
and 0068's attribution_status already use (dolthub/dolt#11345). Makes
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities pass: a
fresh CLI-built bundle now carries participation_generation on both
issues and wisps, matching what a runtime-migrated store gets.

scripts/check-migration-hygiene.sh checks A-D pass; 0068 remains
byte-identical to origin/main.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* build: bazel-sync BUILD.bazel for migration 0070 files (be-blvmg)

make bazel-sync (gazelle) picked up two files the RED commit added that
the hand-ported BUILD.bazel hadn't listed yet: 0070's embedded .up.sql in
the go_library embedsrcs and migration_0070_add_participation_generation_test.go
in the go_test srcs.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* test(feat): red — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3) (refs be-fyaja)

* feat: green — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3)

(refs be-fyaja)

* test(conformance): pin that a wisp reaching the issues plane begins its history (be-31mnz)

Review feedback on #6943, blocker 1: with the flag on, a promoted wisp is
never versioned. None of the five RunParticipationGeneration* cases reaches
a promotion, so nothing caught it.

Two contract cases, wired on all three legs:

- RunParticipationGenerationWispPromotionBeginsHistory: create a wisp with
  the flag on, promote it, and require a stamped participation_generation,
  one version row, and +1 on the next update.
- RunParticipationGenerationWispPersistenceMoveBeginsHistory: the same
  check for a persistence-mode update (bd update --persistent). This mints
  at a different site than promotion on every leg: the guarded update's one
  end-of-update mint on dolt and embeddeddolt, and
  MoveIssuePersistenceInTx's own on uow.

"Wisp" is in the names because this contract already says "promoted" in
R2.3's sense (ProceedsForPromotedRecord), which is a different operation.
The fixture gains CreateWisp, PromoteWisp and MovePersistent closures, and
each leg's FixtureKitIsWired checks them.

Red at this commit on the embedded leg: after the arrival there are 0 rows
and participation_generation is NULL, and after the update there are still
0 rows. A flag-on durable create goes 1 -> 2 in the same store.

* fix(issueops): mint a wisp's first issues-plane version create-shaped (be-31mnz)

Review feedback on #6943, blocker 1. A promotion, or a persistence move out
of the wisps table, inserts a new issues row. A wisp never declares
participation (FR-8), so that row's participation_generation is NULL. Both
sites minted through the update-shaped RecordVersionInTx, and the section
16.2b fence read the NULL as legacy and skipped the mint. Nothing stamps the
column later, so every later update skipped too, and a record created with
versioned history on was never versioned.

These sites now use RecordVersionForCreateInTx, which stamps the column and
mints version 1, as a create on the issues plane does:

- PromoteFromEphemeralInTx (promote.go).
- MoveIssuePersistenceInTx's move out of the wisps table.
- ExecuteUpdate's one end-of-update mint (execution.go), when its
  persistence patch did that move. This is the site bd update --persistent
  reaches on the dolt and embeddeddolt legs, and the review did not list
  it. PersistenceMoveResult.EnteredIssuesPlane tells ExecuteUpdate which
  entry point to use.

One site the review named stays update-shaped on purpose:
MoveIssuePersistenceInTx's same-plane repair, which clears a stray wisp
flag on a row already in issues. That row keeps its own
participation_generation. A row created with the flag on still mints. A
legacy row stays legacy, because a repair is an ordinary write and R2.3
says an ordinary write does not promote it. A create-shaped mint there
would stamp the legacy row and start versioning it.

* docs(schema): 0070 must precede turning versioned history on (be-31mnz)

Review note on #6943 (schema skew). The write fence reads
participation_generation on every flag-on mint of an issues-plane row. On a
store that has not applied 0070 the read fails, and the mutation fails with
it instead of skipping. The migration header now says so, and says that the
flag-off path (the default) returns before the read, so a store below 0070
writes exactly as before.

Comment only. 0070 is not on main, so check-migration-hygiene's frozen-file
check does not apply to it, and no test pins its bytes.

* test(dolt): a persistence repair keeps the row's own participation (be-31mnz)

Pins why MoveIssuePersistenceInTx's same-plane repair stays update-shaped
while promotion and a move out of the wisps table became create-shaped (the
previous commit). The repair clears a stray wisp flag on a row that already
lives in issues, so the row keeps its own participation_generation.

TestPersistenceRepairKeepsTheRowsOwnParticipation runs the repair with
versioned history on for a legacy row (created with the flag off) and a
participating row (created with it on), through both repair entry points:
MoveIssuePersistenceInTx and a guarded update with a persistence patch. They
mint at different sites. The legacy row must stay NULL and mint nothing,
since R2.3 says an ordinary write does not promote it. The participating row
must mint one version. participation_generation must not change in either
case.

It fails if the repair mint is create-shaped, which would stamp and start
versioning the legacy row.

* test(dolt): a delete's rewrite of a legacy neighbor mints nothing (be-31mnz)

Folded in from bee's R1 on #6961, note 1. Once
TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit started creating
its neighbor with history on, nothing covered a delete whose neighbor was
created with history off. Under the section 16.2b fence, the expected result
for that case is different: the rewrite still runs, but it mints nothing.

TestDualWriteDeleteSkipsALegacyNeighborsRewrite creates the fixtures with
history off, turns history on, and force-deletes the cited target. It then
checks four things: the rewrite ran (ReferencesUpdated=1), the neighbor has
no issue_versions rows, its participation_generation is still NULL, and
dolt_status is clean for issue_versions, store_epoch and issues.

* test(embeddeddolt): batch-created rows are stamped with the participation generation

A batch create writes its rows first and mints each created row's first
version last, after the creation-time edges land. The mint has to stay
create-shaped for rows the batch inserted (stamping participation_generation
from the store epoch) and update-shaped for rows it only upserted over
(skipped while the row is a legacy row), so the per-row isNew has to survive
the deferral to the batch's final mint pass.

Pin that on the embedded engine:

- brand-new explicit-id rows are stamped with the store epoch and get one
  version row each, and match the same input run one row at a time with the
  batch-create fast paths off (5 rows, and 250 rows across three INSERT
  statements);
- a legacy row the batch upserts over keeps a NULL participation_generation
  and gets no version row, while the new rows beside it are stamped;
- with versioned history off, no row is stamped and no version row is written.

* ci(bazel): enroll the participation-generation tests in the embedded shards

The embedded-Dolt shard manifest has to name every top-level Test* in
internal/storage/embeddeddolt, or the exact-once coverage check fails.
The two participation-generation fence tests were written before that
check landed on main, and the batch-create stamping test is new, so none
of the three is listed yet. Add them to the 15-shard block as the
generator's incremental output: three inserted lines, no existing
assignment moves.

Also list the new batch-create test file in the embeddeddolt_test srcs,
in sorted position, so the Bazel target compiles it alongside the Go
build. Its imports are already among that target's deps.

Rebase note (maintainer): on current main the embedded storage manifest
uses a 40-shard block; the generator's incremental output placed these
three tests in shards 18, 19 and 39 and re-sorted one existing shard-12
line.

* fix(issueops): keep a created row's updated_at when its first version is minted

Minting a version is bookkeeping. It must not change a user-visible column
of the row it versions.

The create-shaped mint stamps participation_generation with an UPDATE of the
issues row. issues.updated_at is ON UPDATE CURRENT_TIMESTAMP, so an UPDATE
that changes any other column and does not name updated_at rewrites it with
the clock. Before the stamp, a row's first mint left current_revision at its
column default of 1, so the UPDATE changed nothing and the clause never
fired. The stamp moves participation_generation from NULL to the store
epoch, so the clause fires on every row whose first version is minted and
replaces the updated_at an import supplies.

Set updated_at to itself in the stamp, as the blocked-state recompute in
issueops/blocked_state.go already does, so the timestamp the row was created
with survives. Add a regression that batch-creates rows with a supplied
updated_at and requires both the stamp and the unchanged timestamp.

* test(createbatchequiv): re-record the digests the participation-generation fence moves

The fence does not re-mint rows that predate the stamp: a row whose
participation_generation is still NULL gets no new version when a batch
touches it. Main mints one, so six recorded entries move.

The issue_versions entry of five scenarios loses the versions main mints
for rows that predate the stamp: three rows in small, all eleven in
depadd, all fifteen in waitsfor, four in import458 and two in
import458-reject-stale. In depadd, eq-db1 and eq-dl are the only rows
main mints more than once, and each of those mints moves their updated_at
off the fixture date. The fence skips them, so the two rows keep the
fixture date and join the issues_updated_at entry.

No other table, list or counter digest changes, and the fast and per-row
bodies already agree in every scenario. The digests were recorded with
the embedded backend. The doc comment on GoldenDirEnv now says which
entries were re-recorded.

* test(storage): waive the participation-generation contracts on the http leg

main's #7247 (6d21d337ad, "a served http conformance leg") registered a
fourth leg, http, after this PR's original base. The PR's seven
RunParticipationGeneration* entrypoints are neither wired nor waived there,
so TestEveryLegWiresEveryRoleContract/http fails on the rebased branch.

ParticipationGenerationFixture toggles the store's own versioned-history
write flag and reads the raw participation_generation column back: store
write-fence bookkeeping (design §16.2b), not a publicops role the HTTP
client serves. Waive the seven cases on http with a named reason, the same
waiver the donnabox fork carries (09e7ce05af), and keep the gap visible
until a supported HTTP surface is designed and tested.

go test ./internal/storage/ -run TestEveryLegWiresEveryRoleContract now
passes on dolt, embeddeddolt, http and uow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): decide the fence's plane by the table that holds the row

recordVersionInTx chose the plane from the row's flags (IsWisp: Ephemeral
|| NoHistory) and then read participation_generation from the issues table
only. A wisps-TABLE row can carry neither flag, so it passed the IsWisp
return and the issues read failed with sql.ErrNoRows, aborting the whole
mutation whenever versioned history is on. Two producers:

  - bd mol squash clears an ephemeral root's flag in place
    (UpdateIssue {"wisp": false}, after the root's close, in the squash's
    one transaction); update.go and the uow repository's Update both mint
    after it unconditionally.
  - import's "wisp_plane" marker (WispPlaneOverride = &true, flags false)
    routes the row to wisps, and the batch create mints it create-shaped.

On origin/main the same paths do not fail but mint a stray issue_versions
row for a wisps-table row, an FR-8 leak (wisps are never versioned).

Fix: no issues row means nothing to version. ErrNoRows on the
participation_generation read now returns nil (no version, no error) for
update- and create-shaped calls alike. That read is the right signal and
IsActiveWispInTx is not: GetIssueInTx has already resolved the row, so
ErrNoRows on the issues read can only mean "not in issues", which is
exactly the question the fence has to answer before it stamps or advances
an issues row. It costs no extra query. IsActiveWispInTx asks the inverse
question ("is it in wisps?") with one more query per mint, and it folds
every error, including a transient one, into "not a wisp".

Tests come first, as three new conformance contracts with three new
ParticipationGenerationFixture hooks (ClearWispFlag, SquashWispRoot,
CreateUnflaggedWisp). They are wired on the dolt, embeddeddolt and uow legs
as subtests of each leg's TestParticipationGenerationFence (no new
top-level test, so the shard manifests do not change) and waived on http
with participationGenerationRawStoreWaiverReason:

  RunParticipationGenerationWispFlagClearMintsNothing      (create wisp,
      UpdateIssue {"wisp": false}: no error, 0 versions)
  RunParticipationGenerationWispRootSquashMintsNothing     (close +
      UpdateIssue {"wisp": false} in one transaction: no error, 0 versions)
  RunParticipationGenerationUnflaggedWispCreateMintsNothing (CreateIssues
      with WispPlaneOverride=&true, flags false: no error, 0 versions)

Red before the fix, green after, run both ways:
  embeddeddolt (BEADS_TEST_EMBEDDED_DOLT=1): all three FAIL with
    "versioned history: read participation_generation for pg-...: sql: no
    rows in result set"; all three PASS after.
  uow: all three FAIL with the same error; all three PASS after.
  dolt (BEADS_TEST_DOLT_SERVER=local): WispRootSquash and UnflaggedWispCreate
    FAIL with the same error; both PASS after. WispFlagClear passes both
    ways on this leg. The store-level wisp update (updateWisp) runs in a
    transaction versioned history never scopes, so it never reaches the
    mint. For the same reason the leg's SquashWispRoot hook turns the events
    journal on for its one transaction: that pins the wisp writes to the
    scoped regular transaction, as in a journal-enabled workspace, so that
    case does reach the seam.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): a create stamps participation even when the row is flagged no-history

A create-shaped mint (RecordVersionForCreateInTx) of a brand-new
ISSUES-TABLE row that carries NoHistory or Ephemeral returned at the IsWisp
check before the isCreate stamp, so participation_generation stayed NULL
although the row was created with versioned history on.

Import produces exactly that row: bd-r9uce keeps a promoted no-history
wisp's durable row on the issues plane with its stray no_history=true
(import_shared.go applyImportWispPlane: WispPlaneOverride = &false). When
bd update --persistent later repairs the flag, it is a same-plane normalize
whose mint is update-shaped by design (persistence.go: "A row created with
the flag on proceeds; a legacy row stays legacy"). The NULL column made the
fence read the row as legacy, so the repair and every later update were
fenced and the row was never versioned. origin/main versions it after the
repair (0, then 1, then 2 rows).

Fix: on a create-shaped call, a flagged row that the issues table holds
gets participation_generation stamped from store_epoch.epoch, the same
source the minting stamp uses, read in the same transaction. No version is
minted and updated_at is kept (SET ..., updated_at = updated_at), then the
call returns. Update-shaped calls on flagged rows still return before any
read. The table check from the previous commit now runs ahead of the IsWisp
branch for create-shaped calls, so a flagged wisps-table row (the common
wisp create) still mints and stamps nothing. That costs a create-shaped
call on a flagged wisp one primary-key miss on issues, and only while
versioned history is on. The store_epoch read/seed moves into
readStoreEpochInTx so both stamps share it.

Tests first:
  - New conformance contract
    RunParticipationGenerationNoHistoryIssueCreateDeclaresParticipation and
    hook CreateNoHistoryIssue. Wired on dolt, embeddeddolt and uow as a
    TestParticipationGenerationFence subtest and waived on http. With
    history on: create {NoHistory: true, WispPlaneOverride: &false} gives
    participation_generation non-NULL and 0 versions. The persistent repair
    (MovePersistent) then gives 1 version, and an update-shaped edit
    (MutateExisting, a title edit) gives 2.
  - New subtest
    TestCreateBatchParticipationGeneration/NoHistoryRowIsStampedUnversionedAndKeepsTheCallersUpdatedAt
    (embeddeddolt): a batch of such rows with a supplied updated_at comes
    out stamped, unversioned, and with updated_at unchanged.

Red before the fix, green after, run both ways:
  dolt (BEADS_TEST_DOLT_SERVER=local), embeddeddolt
  (BEADS_TEST_EMBEDDED_DOLT=1), uow: the contract FAILS on all three legs
    ("participation_generation for pg-no-history-create is NULL ...", then
    "= 0 after the persistent repair, want 1", then "went from 0 to 0 ...
    want exactly +1"). It PASSES on all three after.
  embeddeddolt subtest: FAILS before the fix ("store_epoch has no row after
    the batch, so nothing stamped the 5 rows"). With the fix minus the
    updated_at = updated_at clause it FAILS on every row's updated_at. It
    PASSES with the full fix.
No new top-level Test*, so the shard manifests do not change. The
createbatchequiv goldens are unchanged: the dolt and embeddeddolt
equivalence tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: fix comments the rebase left stale

Comment-only.

- createbatchequiv.GoldenDirEnv said "One deliberate behavior change since
  ... nothing else moved" and then listed a second re-record, the
  participation-generation fence's. It now says there are two deliberate
  changes, puts each in its own paragraph, and scopes "nothing else moved"
  to apply.json, the only golden the first change touched.
- issueops CreateIssueResult.isNew and deferredVersion still described one
  write path. After main's #7205, a row written by deferredCreates'
  multi-row INSERT reaches finishCreateIssueInTx(..., true) directly
  (create_batch_cache.go flush) and never passes through InsertIssueIfNew
  or CreateIssueInTxWithResult. isNew is InsertIssueIfNew's result on the
  per-row path and always true for a multi-row-INSERT row. deferredVersion
  is filled from both paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): pin the create/update shape split in the version guard

TestEveryVersionedEntryPointMints counts RecordVersionInTx and
RecordVersionForCreateInTx as one seam. A create-shaped site that fell back
to the update-shaped entry therefore stayed green, though the fence then
reads the new row's NULL participation_generation as legacy and the record
is never versioned. The reverse case also stayed green: an update path that
reached the create-shaped entry would stamp an existing legacy row and
promote it, which R2.3 forbids.

Add the split to the structural guard. It is two small tests over the same
journalscan fixpoint and mintVersion edge discipline, with no AST changes:

- TestCreateShapedEntryPointsMintCreateShaped: the create-shaped mint sites
  (finishCreateIssueInTx, CreateIssuesInTxWithContext,
  PromoteFromEphemeralInTx, moveIssuePersistenceInTx, executeUpdate) must
  each call RecordVersionForCreateInTx themselves. A delegate is not enough
  here, because executeUpdate runs its move with mintVersion=false and a
  batch switches finishCreateIssueInTx's mint off at run time. Every
  create-shaped entry point, the delegates and #7205's
  deferredCreates.flush included, must reach it.
- TestUpdateShapedEntryPointsNeverMintCreateShaped: every other
  versionedEntryPoint must not reach RecordVersionForCreateInTx. The
  exception is ApplyBatchInTx, which runs create and update items together
  and is listed as mixed with a reason.

Mutation-checked, each probe reverted afterwards. executeUpdate's
EnteredIssuesPlane mint, PromoteFromEphemeralInTx's mint, or the batch's
deferred create-shaped mint switched to RecordVersionInTx: each fails the
create half. update.go's end-of-update mint switched to
RecordVersionForCreateInTx: fails the update half for UpdateIssueInTx and
every entry point that writes through it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): hold the dual-shape mint sites to both halves of the split

f006c871c5 listed executeUpdate, moveIssuePersistenceInTx,
finishCreateIssueInTx and CreateIssuesInTxWithContext (and the
ExecuteUpdate/MoveIssuePersistenceInTx/create wrappers that delegate to
them) as create-shaped, so TestUpdateShapedEntryPointsNeverMintCreateShaped
skipped them. Each one also mints update-shaped, on its existing-row or
same-plane branch, which is the common path: a guarded update of an
existing row, the bd update --persistent repair of an issues-plane row's
flags, a create or batch create that resolved to a row already there.
Switching that branch to RecordVersionForCreateInTx stamped a legacy row
out of legacy (R2.3), and every guard stayed green.

Split the lists by what each function actually mints:

- versionCreateShapedMintSites keeps only the pure create-shaped site,
  PromoteFromEphemeralInTx. It must call RecordVersionForCreateInTx and
  must not call RecordVersionInTx.
- versionDualShapeMintSites names the four, each with its two branches.
  TestDualShapeMintSitesCallBothEntries requires each to call BOTH entries
  directly. A delegate does not count, for the same mintVersion=false and
  DeferVersionMint reasons as before.
- versionCreateShapedEntryPoints keeps PromoteFromEphemeralInTx and
  deferredCreates.flush, which passes isNew=true for every row.
- versionDualShapeEntryPoints replaces versionMixedShapeEntryPoints. It
  holds ApplyBatchInTx and the create, persistence-move and guarded-update
  entries that delegate to a dual-shape site, each with a reason. Each must
  reach both entries transitively.

The update half skips the create-shaped and dual-shape entry points, as
before. The update-shaped branch inside a dual-shape site is now the new
test's to pin.

Mutation-checked in a throwaway worktree. Switching the update-shaped mint
to RecordVersionForCreateInTx at execution.go:301, persistence.go:97,
create.go:260 or create.go:499 now fails
TestDualShapeMintSitesCallBothEntries for that site. Before this commit,
the whole issueops package stayed green under each of these switches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(httpclient): run the participation-generation fence over the served wire

edd3b72a1a waived all eleven ParticipationGeneration contracts on the http
leg, saying the client had "no write flag to toggle and no raw column to
read". That reason was wrong. served_dualwrite_history_test.go already
flips the reference store's VersionedHistoryConfigurer and reads its raw
columns, with the client driving the mutation. The waiver also hid the
one harness that runs the fence behind the update a connected workspace
actually sends. Every store leg's MutateExisting is store.UpdateIssue,
but the served Lifecycle.Update reaches the guarded update,
ExecuteUpdate.

Add served_participation_generation_test.go on the same pattern:

- Bound to the client: Mutate is Lifecycle.Create, MutateExisting is
  Lifecycle.Update, MovePersistent is the persistence-mode
  Lifecycle.Update that bd update --persistent sends, and PromoteWisp is
  issueops.Promote over the client's Reader/Lifecycle/Commenter, the
  entry bd promote dials over http.
- Bound to the reference store, as before: SetFlag, the CreateWisp seed,
  and the raw reads.

TestServedParticipationGenerationFence runs seven cases:
CreateWithFlagOnStampsAValue, CreateWithFlagOffLeavesLegacyNull,
SkipsUpdateOnLegacyRecord, ProceedsForPromotedRecord,
FenceIgnoresSchemaSkewHatch, WispPromotionBeginsHistory and
WispPersistenceMoveBeginsHistory. All seven pass, and no case turned up a
server-side fence bug.

Four cases stay waived, now with a reason that names the actual wire gap
(participationGenerationWireGapWaiverReason):

- WispFlagClearMintsNothing: the in-place {"wisp": false} is an untyped
  UpdateIssue, and IssuePatch.Persistence moves the row instead.
- WispRootSquashMintsNothing: it needs RunInTransaction. That and
  UpdateIssue are both on the client's unsupported allowlist (D8).
- UnflaggedWispCreateMintsNothing and
  NoHistoryIssueCreateDeclaresParticipation: both need WispPlaneOverride,
  which every create shape on the wire drops
  (roleIgnoredCreateIssueMembers).

The new file is listed in httpclient_test's srcs. The served lane deals
unpinned tests round-robin, so no shard manifest names it.

Mutation-checked in a throwaway worktree. With execution.go:301 (the
guarded update's update-shaped mint) switched to
RecordVersionForCreateInTx, SkipsUpdateOnLegacyRecord and
FenceIgnoresSchemaSkewHatch fail here ("version row count for
hpg-legacy-skip went from 0 to 1"). The embeddeddolt and uow
Participation|DualWrite|Version suites stay green under the same probe.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): an update never stamps a legacy no-history row

7c00d65fa6 made the seam write participation_generation on a flagged
(NoHistory or Ephemeral) issues-table row when the call is create-shaped.
Only the early `if IsWisp(issue) && !isCreate { return nil }` keeps an
update-shaped call out of that branch, and nothing pinned it. The new
contract and its subtest covered only the positive, create-shaped side.
With the early return dropped, an ordinary update stamped a legacy
no-history row out of legacy (R2.3), and every suite stayed green.

Add RunParticipationGenerationLegacyNoHistoryIssueStaysLegacy beside
NoHistoryIssueCreateDeclaresParticipation as its negative twin:

1. Create the no-history issues-table row with the flag OFF.
2. Turn the flag on and run MutateExisting. participation_generation must
   stay NULL, with 0 versions and current_revision unchanged.
3. Run MovePersistent, the bd update --persistent repair that clears the
   flag in place. The repair is update-shaped too, so the same three
   things must still hold.

Wired on all four legs: dolt, embeddeddolt, uow and http. On http the
case exercises the client's Lifecycle.Update and persistence update.
Their precondition, the flag-off no-history create, is seeded through
the reference store, because no client create can land that row in the
issues table. That gap is also why the case whose SUBJECT is that create
stays waived on http.

Mutation-checked in a throwaway worktree, with the early return made
`if false && IsWisp(issue) && !isCreate`. The new case fails on all four
legs and is the only case that fails:
"participation_generation for pg-no-history-legacy = 1 after an
update-shaped mutation with the flag on, want NULL" (dolt server leg,
embeddeddolt, uow and http). At tip it passes on all four.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(issueops): say which table and which mint the fence comments mean

Comment and doc corrections. No code changes.

- RecordVersionForCreateInTx's doc comment covered a flagged issues-table
  row but not a wisps-table row. A row the wisps table holds mints and
  stamps nothing, whatever its flags say: there is no issues row to stamp
  (FR-8). That holds for an ordinary wisp create and for import's
  unflagged "wisp_plane" row.
- The no-history stamp branch said the bd update --persistent repair mints
  update-shaped "(MoveIssuePersistenceInTx)". That is true only on the
  unit-of-work path. On dolt and embeddeddolt the repair goes through the
  guarded update: executeUpdate runs moveIssuePersistenceInTx with
  mintVersion=false and mints once at its own end. The fixture's
  MovePersistent doc already says this.
- VERSIONED-BEADS-WRITE-PATHS.md row 13 still credited the whole wisp
  exclusion to `RecordVersionInTx returns on IsWisp` and called every case
  a NO-OP. After ec52d29030, the table holding the row decides its plane:
  only an update-shaped call on a flagged wisp returns on IsWisp. Every
  other wisps-table row, including an unflagged one, is excluded by the
  issues-table participation_generation read, which finds no row. After
  7c00d65fa6, a create-shaped call on a flagged issues-table row is
  stamped but not versioned, so the row says that too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): the wisps-table cases check the row is still a wisp

runWispTableRowMintsNothing asserted 0 versions but never checked its own
precondition, that the wisps table still holds the row. Suppose a step
moved the row onto the issues plane through an update-shaped mint. That
mint would hit the new row's NULL participation_generation and be fenced,
so the count stayed 0 and the case passed without saying anything about
a wisps-table row.

- Add an InWispsTable hook to ParticipationGenerationFixture and bind it
  on dolt, embeddeddolt and uow as a raw wisps-table count. The
  wisps-table body now Fatals if the row is not in the wisps table after
  its steps. Each leg's KitIsWired test checks the hook. On http it
  stays nil with the other wisps-table hooks, whose cases are waived
  there.
- The dolt ClearWispFlag hook now bites. A bare store.UpdateIssue on a
  wisp goes through updateWisp, whose transaction versioned history never
  scopes. So WispFlagClearMintsNothing passed on dolt with or without the
  ec52d29030 fix, as that commit's message admits. The hook now runs the
  lone flag clear in a RunInTransaction with the events journal on: the
  journal-enabled shape the SquashWispRoot hook already uses.
- Both hooks go through a new withEventsJournal helper. It restores the
  store's previous journal setting rather than unconditionally setting it
  to false afterwards, which assumed the store started with the journal
  off.

Mutation-checked in throwaway worktrees:
- With version_history.go at ec52d29030^, dolt's WispFlagClearMintsNothing
  now fails ("read participation_generation for pg-flag-clear: sql: no
  rows in result set"). It used to pass there.
- Embedded ClearWispFlag rerouted as a flag-off persistence move followed
  by a flag-on title update: the case fails on the new precondition ("is
  not in the wisps table after flag-clear"). With the precondition
  disabled, the same probe passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(storage): review fixups for the participation_generation fence

The review of d17b716115 approved the PR with no required changes and
raised four minors and three nits. Each one below says what changed and
why, or why it did not.

Minor 1, the upsert create's new-or-existing choice was untested. The
unit-of-work create that is not CreateOnly (bd batch, bd todo, tracker
sync and mol port) inserts through an upsert, and db.Insert's existed
probe picks the mint's shape; forcing it either way shipped green. The
new uow TestParticipationGenerationUpsertCreate pins all three outcomes:
a new row is stamped and begins its history, a legacy row the upsert
resolves to stays legacy with no version, and a participating row mints
exactly one more. No served-route case is added because every served
create is CreateOnly (uow createParams), so the http wire cannot reach
this branch. The cmd/bd proxied-server batch and todo creates call this
same use-case method.

Minor 2, an update-shaped mint still lets issues.updated_at move
through ON UPDATE CURRENT_TIMESTAMP. Not changed here. The maintainer
listed it on the PR as follow-up (2), and the approving review of
d17b716115 accepted carrying it as a follow-up. It does not change
the fence's contract as pinned.

Minor 3, the depadd and waitsfor goldens had been re-recorded to zero
issue_versions rows. Their seeds run with history off, so every seeded
row was legacy and the harness stopped checking the dependency-add,
close and reopen mints. Both scenarios now stamp their seeded
issues-table rows after the seed (stampParticipation; wisps stay
unstamped). Their goldens are back to main's pre-fence files byte for
byte: depadd mints 11 version rows and waitsfor 15. The fenced legacy
outcome stays pinned by small, import458 and import458-reject-stale. A
run scenario that mints no version row now fails rather than recording
an empty digest.

Minor 4, PromoteFromEphemeralInTx always minted create-shaped. When the
issues plane already held the id (the dual presence GH#4455 records),
its upsert resolved to that row and the create-shaped mint stamped a
legacy record (R2.3). It now keeps InsertIssueIfNew's isNew and mints
update-shaped for an existing row, as finishCreateIssueInTx does. The
version guard lists it as a dual-shape mint site and entry point, and
PromoteOverDualPresenceKeepsLegacy pins the behavior at run time.

Nit 1: with versioned history off, db.Insert skips the existed probe,
since both mint entries return before reading anything. A create costs
the round trips it did before the fence.

Nit 2: the MINT COST comment now counts the fence read (about 6 to 7
round trips) and no longer says the snapshot reads current_revision;
the fence read can select it for #6379's ordinal seed. Moving that read
now would only add a column nothing consumes yet.

Nit 3: the http leg's participation-generation wire-gap waiver opens as
PENDING and names the wire mutation that would retire each case.

Validation, in a scrubbed env with -tags=gms_pure_go on embedded and
server Dolt (all pass, none skipped):
- uow: TestParticipationGenerationFence, its FixtureKitIsWired check
  and TestParticipationGenerationUpsertCreate with its 4 subtests.
- embeddeddolt: the create-batch equivalence parts (small/depadd,
  apply/waitsfor), the parts-coverage guard, the large scenarios, and
  the fence and kit.
- dolt: TestCreateBatchFastPathsMatchPerRow_Dolt, the fence,
  FenceIgnoresSchemaSkewHatch and the kit.
- httpclient: TestServedParticipationGenerationFence.
- TestEveryLegWiresEveryRoleContract, the issueops structural guards,
  and the issueops, storage and domain/db packages.

Eleven mutants, graded in 13 runs, each fail the test meant to catch
them:
- upsert always create-shaped or always update-shaped;
- the probe guard inverted;
- promote always create-shaped or always update-shaped;
- the seed left unstamped;
- the dependency-add, close or reopen mint dropped;
- two on-disk single-entry promotes.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): a wisp upsert must not stamp a legacy issues twin

domain/db's Insert chose the mint shape of its upsert (the create that
is not CreateOnly) from an existence probe of the target table. For a
wisps-table insert that probe asks the wrong plane: the seam snapshots
the issues table first (GetIssueInTx), so when the issues plane already
held the id as a legacy row, the wisp create took the create-shaped
mint and stamped and versioned that untouched twin. An ordinary wisp
write promoted a legacy record, which R2.3 forbids. bd mol port and the
proxied mol and wisp writers call CreateWisp this way. It is latent
until versioned history is enabled (#6661).

A wisps-table insert never creates an issues row, so it now mints
update-shaped, without the probe. The fence leaves a legacy twin alone,
a wisp-only id still mints nothing through either entry (FR-8), and a
participating twin mints the one version main already minted for it.
Insert's comment now records that last case: the dual presence
predates the fence, and this upsert does not refuse it the way
issueops' create path does.

TestParticipationGenerationUpsertCreate gains
WispOverLegacyIssuesRowKeepsItLegacy, which fails against the previous
dispatch (twin stamped, one version row). Also from review: the version
shape guard names the dual-shape site its parse cannot see, and the
row-lock exemption comment counts all three of the seam's UPDATE issues
statements.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
2026-10-18 13:10:57 +00:00

2725 lines
118 KiB
Go

package schema
import (
"bytes"
"context"
"crypto/sha256"
"database/sql"
"encoding/csv"
"encoding/hex"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"github.com/DATA-DOG/go-sqlmock"
"github.com/steveyegge/beads/internal/storage/depid"
"github.com/steveyegge/beads/internal/testutil"
)
func TestPendingMigrationDirtyTablesDetectsMigration0043Dependencies(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
expectCursorProbe(mock, "schema_migrations", true)
mock.ExpectQuery(`SELECT COALESCE\(MAX\(version\), 0\) FROM schema_migrations`).
WillReturnRows(sqlmock.NewRows([]string{"version"}).AddRow(42))
touched, err := mainSource.pendingMigrationDirtyTables(context.Background(), db, map[string]dirtyTableState{
"dependencies": {},
})
if err != nil {
t.Fatalf("pendingMigrationDirtyTables: %v", err)
}
if len(touched) != 1 || touched[0] != "dependencies" {
t.Fatalf("touched = %v, want [dependencies]", touched)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
// TestMigrateUpReturnsDirtyTablesErrorForPreExistingDirtyTable exercises the
// full MigrateUp pre-flight guard (gastownhall/beads#4566): a pre-existing
// dirty `dependencies` table collides with pending migration 0043, which
// alters it (confirmed against the real migration content by
// TestPendingMigrationDirtyTablesDetectsMigration0043Dependencies above).
// MigrateUp must report this with the typed *DirtyTablesError so
// working-set-reconcile opens can detect it via errors.As and skip the
// migration instead of failing outright.
func TestMigrateUpReturnsDirtyTablesErrorForPreExistingDirtyTable(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// MigrateUp re-asserts the canonical dolt_ignore patterns before anything
// else (GH#4378); the rows changed, so a scoped commit lands before the
// pass runs (#4566: the seed must not ride the per-step pass commits).
expectIgnorePatternSeed(mock, 42)
// #4356: the open-time untrack reconcile runs right after the seed and
// before the no-work short-circuit. On a healthy database it is two reads
// and no writes.
expectIgnoredCursorHealNoop(mock)
// migrationWorkNeeded: mainSource.atLatest reads the current cursor; v42
// is behind LatestVersion(), so the || short-circuits before checking
// ignoredSource.atLatest or the content-hash/backfill probes.
expectCursorProbe(mock, "schema_migrations", true)
expectScalar(mock, "SELECT COALESCE(MAX(version), 0) FROM schema_migrations", "version", 42)
// dirtyTables(ctx, db, false): `dependencies` has an uncommitted, unstaged
// change in the working set.
expectDirtyDoltStatusRow(mock, "dependencies", false)
// The seed changed rows, so it is committed scoped+labeled right after
// pre-existing tables are unstaged, before the dirty-table guards run.
mock.ExpectQuery(regexp.QuoteMeta("CALL DOLT_ADD('dolt_ignore')")).
WillReturnRows(sqlmock.NewRows([]string{"status"}))
mock.ExpectQuery(regexp.QuoteMeta("CALL DOLT_COMMIT('-m', 'schema: seed dolt_ignore patterns', '--skip-empty')")).
WillReturnRows(sqlmock.NewRows([]string{"hash"}))
// committableDirtyTables -> dirtyTables(ctx, db, true): same dirty state.
expectDirtyDoltStatusRow(mock, "dependencies", false)
// MigrateUp now captures the pre-pass main cursor up front (it drives the
// aux-rekey dirtyBefore exemption); v42 (behind latest) is read here.
expectCursorProbe(mock, "schema_migrations", true)
expectScalar(mock, "SELECT COALESCE(MAX(version), 0) FROM schema_migrations", "version", 42)
// auxRekeyExemptTables reads the ignored cursor to see which passes' markers
// are pending, then each pass's clone-local re-key state. This mocked world
// has no ignored cursor table and no local_metadata, so every read stops at
// its existence probe and nothing aux is exempted — and `dependencies` is
// not an aux table anyway, so it stays in dirtyBefore.
expectCursorProbe(mock, "ignored_schema_migrations", false)
for range auxRekeyPasses {
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES`).
WillReturnRows(sqlmock.NewRows([]string{"count"}).AddRow(0))
}
// pendingMigrationDirtyTables re-reads the current version and finds
// migration 0043 touches the dirty `dependencies` table.
expectCursorProbe(mock, "schema_migrations", true)
expectScalar(mock, "SELECT COALESCE(MAX(version), 0) FROM schema_migrations", "version", 42)
_, err = MigrateUp(context.Background(), db)
if err == nil {
t.Fatal("MigrateUp() error = nil, want *DirtyTablesError")
}
var dirtyErr *DirtyTablesError
if !errors.As(err, &dirtyErr) {
t.Fatalf("MigrateUp() error = %v (%T), want *DirtyTablesError", err, err)
}
if len(dirtyErr.Tables) != 1 || dirtyErr.Tables[0] != "dependencies" {
t.Fatalf("DirtyTablesError.Tables = %v, want [dependencies]", dirtyErr.Tables)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
// expectDirtyDoltStatusRow mocks a dolt_status query returning a single dirty
// table row. The regex matches both the plain and dolt_ignore-filtered forms
// of the dirtyTables query (see lock_test.go's expectDoltStatusRows).
func expectDirtyDoltStatusRow(mock sqlmock.Sqlmock, table string, staged bool) {
mock.ExpectQuery("(?s)SELECT s\\.table_name, s\\.staged\\s+FROM dolt_status s").
WillReturnRows(sqlmock.NewRows([]string{"table_name", "staged"}).AddRow(table, staged))
}
func TestIgnoredPendingMigrationDirtyTablesDetectsWispDependencies(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
expectCursorProbe(mock, "ignored_schema_migrations", true)
mock.ExpectQuery(`SELECT COALESCE\(MAX\(version\), 0\) FROM ignored_schema_migrations`).
WillReturnRows(sqlmock.NewRows([]string{"version"}).AddRow(2))
expectIgnoredSentinelProbes(mock, true)
touched, err := ignoredSource.pendingMigrationDirtyTables(context.Background(), db, map[string]dirtyTableState{
"wisp_dependencies": {},
})
if err != nil {
t.Fatalf("pendingMigrationDirtyTables: %v", err)
}
if len(touched) != 1 || touched[0] != "wisp_dependencies" {
t.Fatalf("touched = %v, want [wisp_dependencies]", touched)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestMigrationSQLTouchesTableStatementForms(t *testing.T) {
tests := []struct {
name string
sql string
want bool
}{
{
name: "rename table source",
sql: "RENAME TABLE dependencies TO old_dependencies",
want: true,
},
{
name: "rename table target",
sql: "RENAME TABLE old_dependencies TO dependencies",
want: true,
},
{
name: "create index on table",
sql: "CREATE INDEX idx_dep_type ON dependencies (type)",
want: true,
},
{
name: "create unique index on quoted table",
sql: "CREATE UNIQUE INDEX idx_dep_type ON `dependencies` (type)",
want: true,
},
{
name: "create view named table",
sql: "CREATE OR REPLACE VIEW dependencies AS SELECT 1",
want: true,
},
{
name: "select only",
sql: "SELECT * FROM dependencies",
want: false,
},
{
name: "unrelated ddl",
sql: "ALTER TABLE comments ADD COLUMN reviewed_at DATETIME",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := migrationSQLTouchesTable(tt.sql, "dependencies"); got != tt.want {
t.Fatalf("migrationSQLTouchesTable(%q) = %v, want %v", tt.sql, got, tt.want)
}
})
}
}
func TestCheckNoDuplicateVersionsPanicsWithBothFilenames(t *testing.T) {
files := []migrationFile{
{version: 7, name: "0007_create_metadata.up.sql"},
{version: 12, name: "0012_create_routes.up.sql"},
{version: 7, name: "0007_create_duplicate.up.sql"},
}
defer func() {
r := recover()
if r == nil {
t.Fatal("expected panic on duplicate version, got none")
}
msg, ok := r.(string)
if !ok {
t.Fatalf("expected string panic, got %T: %v", r, r)
}
for _, want := range []string{
"duplicate migration version 7",
"0007_create_metadata.up.sql",
"0007_create_duplicate.up.sql",
"renumber one before commit",
} {
if !strings.Contains(msg, want) {
t.Errorf("panic message %q missing expected substring %q", msg, want)
}
}
}()
checkNoDuplicateVersions(files)
}
// TestGoEmbedMigrationSourcesHaveNoDuplicateVersions runs discovery over the
// real embedded migration tree for both sources. list() panics on duplicate
// numeric prefixes — at runtime that panic fires at store open, before any
// command's RunE, so a duplicate bricks every bd command. Tests that read a
// migration's SQL file directly bypass list() and cannot catch this; this
// discovery-level check makes the whole failure class fail in CI instead.
func TestGoEmbedMigrationSourcesHaveNoDuplicateVersions(t *testing.T) {
for _, src := range []migrationSource{mainSource, ignoredSource} {
files := src.list() // panics on duplicate versions
if len(files) == 0 {
t.Errorf("migrationSource(%s).list() returned no migrations", src.dir)
}
}
}
func TestDirtyTableSignatureRejectsUnsafeTableName(t *testing.T) {
_, err := dirtyTableSignature(context.Background(), nil, "issues'); SELECT 1; --")
if err == nil {
t.Fatal("expected unsafe table name error")
}
if !strings.Contains(err.Error(), "unsafe dolt status table name") {
t.Fatalf("error = %v, want unsafe table name context", err)
}
}
func TestMigration0035HandlesLegacyWispDependenciesShape(t *testing.T) {
upSQL, err := os.ReadFile("migrations/0035_migrate_infra_to_wisps.up.sql")
if err != nil {
t.Fatalf("read 0035 up migration: %v", err)
}
downSQL, err := os.ReadFile("migrations/0035_migrate_infra_to_wisps.down.sql")
if err != nil {
t.Fatalf("read 0035 down migration: %v", err)
}
up := string(upSQL)
for _, want := range []string{
"@has_split_wisp_dependencies",
"INSERT IGNORE INTO wisp_dependencies (issue_id, depends_on_id, type, created_at, created_by, metadata, thread_id)",
"INSERT IGNORE INTO wisp_dependencies (issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external, type, created_at, created_by, metadata, thread_id)",
} {
if !strings.Contains(up, want) {
t.Fatalf("0035 up migration missing legacy/split branch marker %q", want)
}
}
down := string(downSQL)
for _, want := range []string{
"@has_split_wisp_dependencies",
"SELECT issue_id, depends_on_id, type, created_at, created_by, metadata, thread_id FROM wisp_dependencies",
"SELECT issue_id, COALESCE(depends_on_issue_id, depends_on_wisp_id, depends_on_external), type, created_at, created_by, metadata, thread_id FROM wisp_dependencies",
} {
if !strings.Contains(down, want) {
t.Fatalf("0035 down migration missing legacy/split branch marker %q", want)
}
}
}
func TestMigration0053RepairsRigWispsShape(t *testing.T) {
sql, err := os.ReadFile("migrations/0053_repair_rig_wisps.up.sql")
if err != nil {
t.Fatalf("read 0053 up migration: %v", err)
}
body := string(sql)
for _, want := range []string{
"@has_wisps",
"INFORMATION_SCHEMA.TABLES",
"INSERT IGNORE INTO issues",
"FROM wisps WHERE issue_type = ''rig''",
"SET ephemeral = 0",
"INSERT IGNORE INTO dependencies",
"FROM wisp_dependencies wd",
"REPLACE INTO dependencies",
"REPLACE INTO wisp_dependencies",
"wisp_child_counters",
"INSERT IGNORE INTO child_counters",
"UPDATE child_counters",
"DELETE FROM wisps WHERE issue_type = ''rig''",
} {
if !strings.Contains(body, want) {
t.Fatalf("0053 migration missing rig repair marker %q", want)
}
}
}
func TestEnsureIssuesRigColumnsAddsOnlyMissing(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// #4502: issues tables bootstrapped before the rig/agent columns landed
// in 0001 reach v52 without them; the 0053 pre-repair must add exactly
// the missing ones. Simulate hook_bead present, the other five absent.
countQuery := `SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS`
mock.ExpectQuery(countQuery).WithArgs("issues", "hook_bead").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
for _, col := range []struct{ name, ddl string }{
{"role_bead", "ALTER TABLE issues ADD COLUMN role_bead VARCHAR\\(255\\) DEFAULT ''"},
{"agent_state", "ALTER TABLE issues ADD COLUMN agent_state VARCHAR\\(32\\) DEFAULT ''"},
{"last_activity", "ALTER TABLE issues ADD COLUMN last_activity DATETIME"},
{"role_type", "ALTER TABLE issues ADD COLUMN role_type VARCHAR\\(32\\) DEFAULT ''"},
{"rig", "ALTER TABLE issues ADD COLUMN rig VARCHAR\\(255\\) DEFAULT ''"},
} {
mock.ExpectQuery(countQuery).WithArgs("issues", col.name).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(col.ddl).WillReturnResult(sqlmock.NewResult(0, 0))
}
if err := ensureIssuesRigColumns(context.Background(), db); err != nil {
t.Fatalf("ensureIssuesRigColumns: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureWispDependenciesSplitTargetsAddsMissingAndBackfills(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
ddlByColumn := map[string]string{
"depends_on_issue_id": "ALTER TABLE wisp_dependencies ADD COLUMN depends_on_issue_id VARCHAR\\(255\\) NULL",
"depends_on_wisp_id": "ALTER TABLE wisp_dependencies ADD COLUMN depends_on_wisp_id VARCHAR\\(255\\) NULL",
"depends_on_external": "ALTER TABLE wisp_dependencies ADD COLUMN depends_on_external VARCHAR\\(255\\) NULL",
}
for _, col := range []string{"depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"} {
mock.ExpectQuery(columnQuery).WithArgs("wisp_dependencies", col).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(ddlByColumn[col]).WillReturnResult(sqlmock.NewResult(0, 0))
}
mock.ExpectQuery(columnQuery).WithArgs("wisp_dependencies", "depends_on_id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
for _, update := range []string{
`UPDATE wisp_dependencies SET depends_on_external = depends_on_id`,
`UPDATE wisp_dependencies wd JOIN wisps w ON w\.id = wd\.depends_on_id`,
`UPDATE wisp_dependencies wd JOIN issues i ON i\.id = wd\.depends_on_id`,
`UPDATE wisp_dependencies SET depends_on_external = depends_on_id WHERE depends_on_external IS NULL`,
} {
mock.ExpectExec(update).WillReturnResult(sqlmock.NewResult(0, 1))
}
if err := ensureWispDependenciesSplitTargets(context.Background(), db); err != nil {
t.Fatalf("ensureWispDependenciesSplitTargets: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestPreMigrationRepairScopedToMain0053(t *testing.T) {
// The repair must not fire for other versions or for the ignored source
// (whose cursor table differs); nil DB proves no queries are attempted.
if err := mainSource.preMigrationRepair(context.Background(), nil, 52); err != nil {
t.Fatalf("main v52 repair = %v, want nil no-op", err)
}
if err := ignoredSource.preMigrationRepair(context.Background(), nil, 53); err != nil {
t.Fatalf("ignored v53 repair = %v, want nil no-op", err)
}
}
func TestPreMigrationRepairScopedToMain0047(t *testing.T) {
// #4695/#4176: the repair must not fire for other versions or for the
// ignored source; nil DB proves no queries are attempted.
if err := mainSource.preMigrationRepair(context.Background(), nil, 46); err != nil {
t.Fatalf("main v46 repair = %v, want nil no-op", err)
}
if err := ignoredSource.preMigrationRepair(context.Background(), nil, 47); err != nil {
t.Fatalf("ignored v47 repair = %v, want nil no-op", err)
}
}
func TestPreMigrationRepairDispatchesMain47ToWispTableRepair(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`CREATE TABLE IF NOT EXISTS wisps`).WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`CREATE TABLE IF NOT EXISTS wisp_dependencies`).WillReturnResult(sqlmock.NewResult(0, 0))
if err := mainSource.preMigrationRepair(context.Background(), db, 47); err != nil {
t.Fatalf("main v47 repair: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
// TestRunMigrationsSnapshotsDirtyTablesBeforeRepairSoRepairMutationsCommitAtomically
// is the atomicity fix for the finding-#6 gap: preMigrationRepair (which can
// itself mutate a synced table, e.g. ensureDependenciesIDColumn's ALTERs on
// `dependencies`) must run AFTER the per-step "before" dirty-table snapshot,
// not before it -- otherwise the repair's own mutation is misclassified as
// pre-existing dirt to exclude from this step's commit, and a process killed
// between this step and the pass's later commits would leave history
// claiming the migration applied while the repaired table's change was never
// durably recorded (and the version-gated repair hook can never re-run to
// fix it, since its version is no longer pending).
//
// sqlmock enforces in-order call matching by default, so the exact sequence
// registered below IS the assertion: if runMigrations snapshotted dirty
// tables AFTER calling preMigrationRepair (the pre-fix order), the "before"
// dirtyTables query below would run too late (after the repair's ADD COLUMN
// /MODIFY/ADD PRIMARY KEY execs it's supposed to precede) and this test would
// fail with an out-of-order-call error, not a silent pass.
//
// This is a Go-level proof of the call ordering, not a live Dolt dirty/staged
// commit -- that end-to-end proof belongs in
// internal/storage/embeddeddolt (cgo-gated, see TestEmbeddedMigrateRepairedDependenciesIDColumnCommitsAtomicallyWithVersion53_4690).
func TestRunMigrationsSnapshotsDirtyTablesBeforeRepairSoRepairMutationsCommitAtomically(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
dirtyStatusQuery := `(?s)SELECT s\.table_name, s\.staged\s+FROM dolt_status s\s+WHERE NOT EXISTS`
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
// 1. The "before" snapshot: this step's own repair has not run yet, so
// nothing is dirty.
mock.ExpectQuery(dirtyStatusQuery).
WillReturnRows(sqlmock.NewRows([]string{"table_name", "staged"}))
// 2. preMigrationRepair(53): ensureIssuesRigColumns (all present, no
// exec), ensureWispDependenciesSplitTargets (table absent, no-op), then
// ensureDependenciesIDColumn actually mutating `dependencies`.
for _, col := range []string{"hook_bead", "role_bead", "agent_state", "last_activity", "role_type", "rig"} {
mock.ExpectQuery(columnQuery).WithArgs("issues", col).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
}
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectQuery(columnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies ADD COLUMN id CHAR\(36\) NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectQuery(`(?s)SELECT issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external\s+FROM dependencies\s+WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"issue_id", "depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"}))
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM dependencies WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
keyColumnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.KEY_COLUMN_USAGE.*TABLE_NAME = \? AND COLUMN_NAME = \? AND CONSTRAINT_NAME = 'PRIMARY'`
mock.ExpectQuery(keyColumnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies MODIFY COLUMN id CHAR\(36\) NOT NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
tableConstraintQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLE_CONSTRAINTS.*TABLE_NAME = \? AND CONSTRAINT_TYPE = 'PRIMARY KEY'`
mock.ExpectQuery(tableConstraintQuery).WithArgs("dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies ADD PRIMARY KEY \(id\)`).
WillReturnResult(sqlmock.NewResult(0, 0))
// 3. Migration 0053's own frozen SQL, then its cursor row.
mock.ExpectExec(`SET FOREIGN_KEY_CHECKS = 0`).WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectExec(`INSERT IGNORE INTO schema_migrations`).WithArgs(53, sqlmock.AnyArg()).
WillReturnResult(sqlmock.NewResult(0, 1))
// 4. commitMigrationStep's "after" snapshot sees `dependencies` dirty --
// from the repair's own ALTERs above, now correctly attributed to THIS
// step because the "before" snapshot in (1) predates them -- so it gets
// force-added alongside the cursor table and committed in the same call.
mock.ExpectQuery(dirtyStatusQuery).
WillReturnRows(sqlmock.NewRows([]string{"table_name", "staged"}).AddRow("dependencies", false))
mock.ExpectQuery(`CALL DOLT_ADD\('-f', \?\)`).WithArgs("dependencies").
WillReturnRows(sqlmock.NewRows([]string{"status"}))
mock.ExpectQuery(`CALL DOLT_ADD\('-f', \?\)`).WithArgs("schema_migrations").
WillReturnRows(sqlmock.NewRows([]string{"status"}))
mock.ExpectQuery(`CALL DOLT_COMMIT`).
WillReturnRows(sqlmock.NewRows([]string{"hash"}))
applied, err := runMigrations(context.Background(), db, mainSource, 52, 53, true)
if err != nil {
t.Fatalf("runMigrations: %v", err)
}
if applied != 1 {
t.Fatalf("runMigrations applied = %d, want 1", applied)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations (ordering mismatch would surface here): %v", err)
}
}
func TestEnsureWispTablesForMigration0047CreatesMissingTables(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// #4695/#4176: a clone whose main cursor arrives at 0047 with the wisp
// tables entirely absent (dolt_ignore'd, never synced) must have them
// created at the canonical shape before 0047's frozen recompute runs.
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`CREATE TABLE IF NOT EXISTS wisps \(`).WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`CREATE TABLE IF NOT EXISTS wisp_dependencies \(`).WillReturnResult(sqlmock.NewResult(0, 0))
if err := ensureWispTablesForMixedBlockedRecompute(context.Background(), db); err != nil {
t.Fatalf("ensureWispTablesForMixedBlockedRecompute: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureWispTablesForMigration0047DelegatesSplitTargetRepairWhenWispDependenciesExists(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
// wisps already exists: skip creating it.
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
// wisp_dependencies already exists too: delegate to the same split-target
// repair migration 0053 relies on, rather than recreating it.
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(tableQuery).WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
for _, col := range []string{"depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"} {
mock.ExpectQuery(columnQuery).WithArgs("wisp_dependencies", col).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
}
// The legacy source column is already gone: a prior pass finished the
// backfill (or this database never had it), so there is nothing left to
// re-run and the repair no-ops here.
mock.ExpectQuery(columnQuery).WithArgs("wisp_dependencies", "depends_on_id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
if err := ensureWispTablesForMixedBlockedRecompute(context.Background(), db); err != nil {
t.Fatalf("ensureWispTablesForMixedBlockedRecompute: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestPreMigrationRepairScopedToIgnored15(t *testing.T) {
// The repair must not fire for other versions or for the main source
// (whose cursor table differs); nil DB proves no queries are attempted.
if err := ignoredSource.preMigrationRepair(context.Background(), nil, 14); err != nil {
t.Fatalf("ignored v14 repair = %v, want nil no-op", err)
}
if err := mainSource.preMigrationRepair(context.Background(), nil, 15); err != nil {
t.Fatalf("main v15 repair = %v, want nil no-op", err)
}
}
func TestPreMigrationRepairDispatchesIgnored15ToWispIsBlockedRepair(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
indexQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.STATISTICS.*TABLE_NAME = \? AND INDEX_NAME = \?`
// wisps exists (id/status present) but is_blocked never landed, and its
// index is missing too.
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(columnQuery).WithArgs("wisps", "is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE wisps ADD COLUMN is_blocked TINYINT\(1\) NOT NULL DEFAULT 0`).
WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectQuery(indexQuery).WithArgs("wisps", "idx_wisps_is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`CREATE INDEX idx_wisps_is_blocked ON wisps\(is_blocked, status\)`).
WillReturnResult(sqlmock.NewResult(0, 0))
if err := ignoredSource.preMigrationRepair(context.Background(), db, 15); err != nil {
t.Fatalf("ignored v15 repair: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureWispIsBlockedForRecomputeNoopsWhenWispsAbsent(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// A clone whose ignored/0001 has not yet materialized wisps in this
// pass: the repair takes its early no-op branch rather than attempt to
// ALTER a table that does not exist. That is NOT because an absent wisps
// table is harmless to the pending recompute -- ignored/0007 and 0015's
// own unguarded "UPDATE wisps SET is_blocked = ..." both fail outright
// with a table-not-found error if wisps is genuinely missing when they
// run. The no-op is safe here only because it is never reached with
// wisps absent in a real pass: ignored/0001 (or the cursor-reality
// repair, cursorRealityFloor, when the cursor and schema
// disagree) always materializes wisps before the pass advances this far.
// This test exercises the repair function in isolation and is not a
// claim that 0007/0015 tolerate a missing wisps table.
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
if err := ensureWispIsBlockedForRecompute(context.Background(), db); err != nil {
t.Fatalf("ensureWispIsBlockedForRecompute: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureWispIsBlockedForRecomputeNoopsWhenAlreadyPresent(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// The common (non-drifted) path: a clone that ran ignored/0006 normally
// already has both the column and its index, so the repair issues no
// ALTER or CREATE INDEX -- matching ignored/0006's own guard being a
// no-op here.
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
indexQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.STATISTICS.*TABLE_NAME = \? AND INDEX_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(columnQuery).WithArgs("wisps", "is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(indexQuery).WithArgs("wisps", "idx_wisps_is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
if err := ensureWispIsBlockedForRecompute(context.Background(), db); err != nil {
t.Fatalf("ensureWispIsBlockedForRecompute: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
// wispIsBlockedDriftDB models the drift shape ensureWispIsBlockedForRecompute
// repairs: a wisps table that exists but is missing is_blocked. Unlike a
// loose fake that accepts any statement merely mentioning "is_blocked", this
// one verifies every ExecContext call it receives byte-for-byte against what
// the repair and the pending migration are actually supposed to send: the
// two guarded DDL statements must match ignored/0006's own SQL exactly (the
// same literal strings ensureWispIsBlockedColumn/ensureWispIsBlockedIndex
// issue), the migration body must match the pending file's real embedded
// bytes (read via IgnoredMigrationSQL, so a change to the shipped file
// changes what this test expects too, rather than silently going stale), and
// the cursor-row INSERT must record the exact version this test bounds
// runMigrations to. Anything else reaching ExecContext fails the test
// immediately via t.Fatalf instead of silently succeeding -- unrecognized
// SQL and a wrong cursor version are exactly the two failure modes a
// rubber-stamp fake would hide. QueryRowContext delegates to a real *sql.DB
// (backed by sqlmock, purely for genuine *sql.Row Scan semantics) so
// schemaTableExists/schemaColumnExists/schemaIndexExists behave exactly as
// they would against a live engine; see wispsAbsentDB above for the same
// technique.
type wispIsBlockedDriftDB struct {
t *testing.T
real *sql.DB
mock sqlmock.Sqlmock
cursorTable string
migrationBody string
wantCursorVersion int
hasIsBlockedColumn bool
hasIsBlockedIndex bool
bodyExecuted bool
cursorInserted bool
}
// wispIsBlockedColumnDDL / wispIsBlockedIndexDDL are the exact statements
// ensureWispIsBlockedColumn / ensureWispIsBlockedIndex issue -- ignored/0006's
// own ADD COLUMN and CREATE INDEX, translated to Go. wispIsBlockedDriftDB's
// ExecContext matches against these literal strings, so a future drift
// between the repair's SQL and 0006's shipped SQL fails this test byte-for-
// byte instead of passing under a substring check.
const (
wispIsBlockedColumnDDL = "ALTER TABLE wisps ADD COLUMN is_blocked TINYINT(1) NOT NULL DEFAULT 0"
wispIsBlockedIndexDDL = "CREATE INDEX idx_wisps_is_blocked ON wisps(is_blocked, status)"
)
// newWispIsBlockedDriftDB builds a drift DB bounded to one pending ignored
// migration file (migrationFile, e.g.
// "0015_recompute_null_gate_wisp_is_blocked.up.sql") at wantCursorVersion
// (its numeric version). TestIgnored15RecomputeHardFailsWithoutIsBlockedRepair
// and TestIgnored7RecomputeHardFailsWithoutIsBlockedRepair below share this
// helper for the {ignored_schema_migrations, 15} and {..., 7} registrations
// respectively; only the pending file and version differ between them. The
// returned sqlmock.Sqlmock is exposed via (*wispIsBlockedDriftDB).
// expectationsWereMet so callers can assert it was fully consumed -- but
// only after confirming runMigrations itself succeeded, since an
// unregistered repair never issues the three INFORMATION_SCHEMA probes
// below and would otherwise report a second, unrelated "unmet
// expectations" failure on top of the real one.
func newWispIsBlockedDriftDB(t *testing.T, migrationFile string, wantCursorVersion int) *wispIsBlockedDriftDB {
t.Helper()
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
tableQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
indexQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.STATISTICS.*TABLE_NAME = \? AND INDEX_NAME = \?`
mock.ExpectQuery(tableQuery).WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(columnQuery).WithArgs("wisps", "is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectQuery(indexQuery).WithArgs("wisps", "idx_wisps_is_blocked").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
body, err := IgnoredMigrationSQL(migrationFile)
if err != nil {
t.Fatalf("IgnoredMigrationSQL(%s): %v", migrationFile, err)
}
return &wispIsBlockedDriftDB{
t: t,
real: db,
mock: mock,
cursorTable: ignoredSource.cursorTable,
migrationBody: body,
wantCursorVersion: wantCursorVersion,
}
}
func (d *wispIsBlockedDriftDB) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row {
return d.real.QueryRowContext(ctx, query, args...)
}
func (d *wispIsBlockedDriftDB) QueryContext(context.Context, string, ...any) (*sql.Rows, error) {
panic("wispIsBlockedDriftDB: QueryContext not expected on the ignored pre-repair path")
}
func (d *wispIsBlockedDriftDB) ExecContext(_ context.Context, query string, args ...any) (sql.Result, error) {
d.t.Helper()
switch query {
case wispIsBlockedColumnDDL:
d.hasIsBlockedColumn = true
return execOK{}, nil
case wispIsBlockedIndexDDL:
d.hasIsBlockedIndex = true
return execOK{}, nil
case d.migrationBody:
// The pending migration's real embedded body, byte-for-byte. A wisps
// table still missing is_blocked fails on its first statement with
// the same error a real engine gives; once the column exists (the
// repair ran first) it succeeds -- matching runMigrations' real
// ordering (preMigrationRepair before execMigrationBody).
if !d.hasIsBlockedColumn {
return nil, errors.New("column 'is_blocked' could not be found in any table in scope")
}
d.bodyExecuted = true
return execOK{}, nil
case "INSERT IGNORE INTO " + d.cursorTable + " (version, content_hash) VALUES (?, ?)":
if len(args) == 0 {
d.t.Fatalf("wispIsBlockedDriftDB: cursor INSERT with no version arg")
}
gotVersion, ok := args[0].(int)
if !ok {
d.t.Fatalf("wispIsBlockedDriftDB: cursor INSERT version arg is %T, want int", args[0])
}
if gotVersion != d.wantCursorVersion {
d.t.Fatalf("wispIsBlockedDriftDB: cursor INSERT recorded version %d, want %d", gotVersion, d.wantCursorVersion)
}
d.cursorInserted = true
return execOK{}, nil
default:
d.t.Fatalf("wispIsBlockedDriftDB: unexpected ExecContext query:\n%s", query)
return nil, nil // unreachable: t.Fatalf stops this goroutine
}
}
// expectationsWereMet asserts the three INFORMATION_SCHEMA probe expectations
// registered in newWispIsBlockedDriftDB were consumed. Callers check this
// only once runMigrations has already returned nil: an unregistered repair
// (the RED half of the red-green proof below) never issues those probes at
// all, and asserting this unconditionally would report a second, unrelated
// failure alongside the real one.
func (d *wispIsBlockedDriftDB) expectationsWereMet() error {
return d.mock.ExpectationsWereMet()
}
// execOK is a trivial sql.Result for wispIsBlockedDriftDB's successful exec
// paths above.
type execOK struct{}
func (execOK) LastInsertId() (int64, error) { return 0, nil }
func (execOK) RowsAffected() (int64, error) { return 0, nil }
// TestIgnored15RecomputeHardFailsWithoutIsBlockedRepair is the red-green
// proof for ensureWispIsBlockedForRecompute at the {ignored_schema_migrations,
// 15} registration: it exercises the real runMigrations/preMigrationRepair
// dispatch against the literal embedded ignored/0015 SQL bytes, using
// wispIsBlockedDriftDB to reproduce the exact error a wisps table missing
// is_blocked gets from 0015's frozen UPDATE. With the
// {ignored_schema_migrations, 15} entry in preMigrationRepairs this test
// passes: the column and index are restored before 0015 runs, 0015's own
// embedded body then executes for real, and the version-15 cursor row is
// recorded. With that entry removed it fails with the same "column
// 'is_blocked' could not be found" error a real affected clone hits.
func TestIgnored15RecomputeHardFailsWithoutIsBlockedRepair(t *testing.T) {
orig := stderr
stderr = &bytes.Buffer{}
defer func() { stderr = orig }()
db := newWispIsBlockedDriftDB(t, "0015_recompute_null_gate_wisp_is_blocked.up.sql", 15)
// Bounded to exactly version 15: minVersion=14 skips 1..14 (irrelevant to
// this probe of 0015 alone), upTo=15 stops right after it.
if _, err := runMigrations(context.Background(), db, ignoredSource, 14, 15, false); err != nil {
t.Fatalf("runMigrations(ignoredSource, v15) = %v, want nil: a clone whose wisps table lacks is_blocked must have it restored before 0015's frozen recompute runs", err)
}
if err := db.expectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
if !db.hasIsBlockedColumn {
t.Error("ignored/0015 ran without ever restoring wisps.is_blocked")
}
if !db.hasIsBlockedIndex {
t.Error("ignored/0015 ran without ever restoring idx_wisps_is_blocked")
}
if !db.bodyExecuted {
t.Error("ignored/0015's own embedded recompute body never executed")
}
if !db.cursorInserted {
t.Error("ignored/0015's cursor row (version 15) was never recorded")
}
}
// TestIgnored7RecomputeHardFailsWithoutIsBlockedRepair is
// TestIgnored15RecomputeHardFailsWithoutIsBlockedRepair's twin for the
// {ignored_schema_migrations, 7} registration: a clone stuck at ignored
// cursor 6 (one that never completed 0006) reaches 0007 -- not 0015 -- first,
// and 0007's frozen recompute is the same shape of unguarded UPDATE against
// wisps.is_blocked ignored/0015 carries. Without the
// {ignored_schema_migrations, 7} entry, such a clone hard-fails on every
// single pass at 0007 and can never advance far enough to reach the
// version-15 entry at all: the unrecoverable case
// ensureWispIsBlockedForRecompute's doc comment describes.
func TestIgnored7RecomputeHardFailsWithoutIsBlockedRepair(t *testing.T) {
orig := stderr
stderr = &bytes.Buffer{}
defer func() { stderr = orig }()
db := newWispIsBlockedDriftDB(t, "0007_recompute_wisp_is_blocked.up.sql", 7)
// Bounded to exactly version 7: minVersion=6 (the stuck cursor position)
// skips 1..6, upTo=7 stops right after it.
if _, err := runMigrations(context.Background(), db, ignoredSource, 6, 7, false); err != nil {
t.Fatalf("runMigrations(ignoredSource, v7) = %v, want nil: a cursor-6 clone whose wisps table lacks is_blocked must have it restored before 0007's frozen recompute runs", err)
}
if err := db.expectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
if !db.hasIsBlockedColumn {
t.Error("ignored/0007 ran without ever restoring wisps.is_blocked")
}
if !db.hasIsBlockedIndex {
t.Error("ignored/0007 ran without ever restoring idx_wisps_is_blocked")
}
if !db.bodyExecuted {
t.Error("ignored/0007's own embedded recompute body never executed")
}
if !db.cursorInserted {
t.Error("ignored/0007's cursor row (version 7) was never recorded")
}
}
func TestEnsureDependenciesIDColumnNoopWhenAlreadyFullyBackfilledAndKeyed(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// Steady state: id exists, nothing left to backfill (WHERE id IS NULL
// matches no rows), and id is already the primary key. Column presence
// alone must not short-circuit re-verification (#3/#6 re-entry), but a
// database that has already fully converged still does no work.
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
mock.ExpectQuery(columnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(`(?s)SELECT issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external\s+FROM dependencies\s+WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"issue_id", "depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"}))
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM dependencies WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
keyColumnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.KEY_COLUMN_USAGE.*TABLE_NAME = \? AND COLUMN_NAME = \? AND CONSTRAINT_NAME = 'PRIMARY'`
mock.ExpectQuery(keyColumnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
if err := ensureDependenciesIDColumn(context.Background(), db); err != nil {
t.Fatalf("ensureDependenciesIDColumn: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureDependenciesIDColumnBackfillsMissingIDsDeterministically(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// #4690: dependencies already in split-target/content-derived-key shape
// but never had its surrogate id column added at all (a different
// historical migration path than this repo's 0043). Add it, backfill
// with the same deterministic id every insert path and the post-migration
// rekey pass use, and restore id as the PRIMARY KEY (0043's exact
// canonical shape -- not just a plain NOT NULL column, see #4690's
// duplication risk without a key).
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
mock.ExpectQuery(columnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies ADD COLUMN id CHAR\(36\) NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
rows := sqlmock.NewRows([]string{"issue_id", "depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"}).
AddRow("iss-1", "iss-2", nil, nil).
AddRow("iss-3", nil, "wisp-1", nil)
mock.ExpectQuery(`(?s)SELECT issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external\s+FROM dependencies\s+WHERE id IS NULL`).
WillReturnRows(rows)
id1 := depid.New("iss-1", "iss-2")
mock.ExpectExec(`(?s)UPDATE dependencies SET id = \?\s+WHERE issue_id = \?.*depends_on_issue_id <=> \?.*depends_on_wisp_id <=> \?.*depends_on_external <=> \?`).
WithArgs(id1, "iss-1", "iss-2", nil, nil).
WillReturnResult(sqlmock.NewResult(0, 1))
id2 := depid.New("iss-3", "wisp-1")
mock.ExpectExec(`(?s)UPDATE dependencies SET id = \?\s+WHERE issue_id = \?.*depends_on_issue_id <=> \?.*depends_on_wisp_id <=> \?.*depends_on_external <=> \?`).
WithArgs(id2, "iss-3", nil, "wisp-1", nil).
WillReturnResult(sqlmock.NewResult(0, 1))
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM dependencies WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
keyColumnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.KEY_COLUMN_USAGE.*TABLE_NAME = \? AND COLUMN_NAME = \? AND CONSTRAINT_NAME = 'PRIMARY'`
mock.ExpectQuery(keyColumnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies MODIFY COLUMN id CHAR\(36\) NOT NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
tableConstraintQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLE_CONSTRAINTS.*TABLE_NAME = \? AND CONSTRAINT_TYPE = 'PRIMARY KEY'`
mock.ExpectQuery(tableConstraintQuery).WithArgs("dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies ADD PRIMARY KEY \(id\)`).
WillReturnResult(sqlmock.NewResult(0, 0))
if err := ensureDependenciesIDColumn(context.Background(), db); err != nil {
t.Fatalf("ensureDependenciesIDColumn: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureDependenciesIDColumnDropsExistingPrimaryKeyBeforeAddingID(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// The #4690 drifted shape can leave dependencies keyed some other way
// (e.g. a composite content-derived key predating the id column). A
// table carries only one PRIMARY KEY, so the existing one must be
// dropped before id can become it; the uk_dep_* natural-identity unique
// keys (0043) enforce the real uniqueness regardless.
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
mock.ExpectQuery(columnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectQuery(`(?s)SELECT issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external\s+FROM dependencies\s+WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"issue_id", "depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"}))
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM dependencies WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
keyColumnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.KEY_COLUMN_USAGE.*TABLE_NAME = \? AND COLUMN_NAME = \? AND CONSTRAINT_NAME = 'PRIMARY'`
mock.ExpectQuery(keyColumnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies MODIFY COLUMN id CHAR\(36\) NOT NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
tableConstraintQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLE_CONSTRAINTS.*TABLE_NAME = \? AND CONSTRAINT_TYPE = 'PRIMARY KEY'`
mock.ExpectQuery(tableConstraintQuery).WithArgs("dependencies").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
mock.ExpectExec(`ALTER TABLE dependencies DROP PRIMARY KEY`).
WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectExec(`ALTER TABLE dependencies ADD PRIMARY KEY \(id\)`).
WillReturnResult(sqlmock.NewResult(0, 0))
if err := ensureDependenciesIDColumn(context.Background(), db); err != nil {
t.Fatalf("ensureDependenciesIDColumn: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestEnsureDependenciesIDColumnFailsClearlyOnTargetlessRowInsteadOfBrickingNotNullModify(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
// ck_dep_one_target (0041) should make a targetless row unreachable in
// practice, but if one exists anyway the repair must fail with a clear,
// actionable error instead of leaving id NULL under a column callers
// assume is NOT NULL, or letting a blind MODIFY ... NOT NULL abort with a
// generic, confusing "column cannot be null" error.
columnQuery := `(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.COLUMNS.*TABLE_NAME = \? AND COLUMN_NAME = \?`
mock.ExpectQuery(columnQuery).WithArgs("dependencies", "id").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
mock.ExpectExec(`ALTER TABLE dependencies ADD COLUMN id CHAR\(36\) NULL`).
WillReturnResult(sqlmock.NewResult(0, 0))
rows := sqlmock.NewRows([]string{"issue_id", "depends_on_issue_id", "depends_on_wisp_id", "depends_on_external"}).
AddRow("iss-4", nil, nil, nil)
mock.ExpectQuery(`(?s)SELECT issue_id, depends_on_issue_id, depends_on_wisp_id, depends_on_external\s+FROM dependencies\s+WHERE id IS NULL`).
WillReturnRows(rows)
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM dependencies WHERE id IS NULL`).
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(1))
err = ensureDependenciesIDColumn(context.Background(), db)
if err == nil {
t.Fatal("ensureDependenciesIDColumn = nil error, want a targetless-row error")
}
if !strings.Contains(err.Error(), "1 dependencies row") {
t.Fatalf("ensureDependenciesIDColumn error = %q, want it to name the unbackfillable row count", err)
}
// The MODIFY COLUMN ... NOT NULL was never attempted (no mock expectation
// registered for it above): mock.ExpectationsWereMet catches a stray call.
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestMigration0057GuardsEventValueColumnsIndependently(t *testing.T) {
sql, err := os.ReadFile("migrations/0057_events_value_columns_idempotent_longtext.up.sql")
if err != nil {
t.Fatalf("read 0057 up migration: %v", err)
}
// Each column has its own guard variable and its own MODIFY, not one
// combined check driving both: a single shared check would either skip
// converting the other column (if only one had already drifted to
// LONGTEXT) or re-issue a MODIFY on a column that's already LONGTEXT,
// re-triggering the encoding-flip risk this migration exists to guard
// against.
body := string(sql)
for _, want := range []string{
"@old_value_needs_fix",
"@new_value_needs_fix",
"INFORMATION_SCHEMA.COLUMNS",
"COLUMN_TYPE = 'text'",
"ALTER TABLE events MODIFY COLUMN old_value LONGTEXT",
"ALTER TABLE events MODIFY COLUMN new_value LONGTEXT",
} {
if !strings.Contains(body, want) {
t.Fatalf("0057 migration missing independent per-column guard marker %q", want)
}
}
// The two columns must not be MODIFY'd in one combined ALTER statement
// (that would put them back behind a single guard).
if strings.Contains(body, "MODIFY COLUMN old_value LONGTEXT, MODIFY COLUMN new_value LONGTEXT") {
t.Fatal("0057 migration MODIFYs old_value and new_value in one combined ALTER, want independent guards/ALTERs per column")
}
}
func TestIgnoredMigration0011CleansOrphanedChildCountersShape(t *testing.T) {
sql, err := os.ReadFile("migrations/ignored/0011_cleanup_orphaned_child_counters.up.sql")
if err != nil {
t.Fatalf("read ignored 0011 up migration: %v", err)
}
// #4534: counter rows orphaned while fk_counter_parent was dropped brick
// all inserts once the FK returns; the cleanup must preserve live-wisp
// counters and delete only rows dangling from issues.
body := string(sql)
for _, want := range []string{
"@has_child_counters",
"INSERT IGNORE INTO wisp_child_counters",
"GREATEST(wcc.last_child, cc.last_child)",
"DELETE cc FROM child_counters cc INNER JOIN wisps w ON w.id = cc.parent_id",
"DELETE cc FROM child_counters cc LEFT JOIN issues i ON i.id = cc.parent_id WHERE i.id IS NULL",
} {
if !strings.Contains(body, want) {
t.Fatalf("ignored 0011 migration missing cleanup marker %q", want)
}
}
}
func TestMigration0053NoopsWithoutWispTablesThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "rig-repair-no-wisps")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-wisps dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
seedSQL := fmt.Sprintf(`
DROP TABLE IF EXISTS wisp_child_counters;
DROP TABLE IF EXISTS wisp_comments;
DROP TABLE IF EXISTS wisp_events;
DROP TABLE IF EXISTS wisp_dependencies;
DROP TABLE IF EXISTS wisp_labels;
DROP TABLE IF EXISTS wisps;
DELETE FROM schema_migrations WHERE version = %d;
`, LatestVersion())
migrationSQL, err := mainSource.files.ReadFile("migrations/0053_repair_rig_wisps.up.sql")
if err != nil {
t.Fatalf("read 0053 migration: %v", err)
}
runDoltSQL(t, dir, seedSQL+"\n"+string(migrationSQL))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps'`, "0")
}
// TestMigration0056NoopsWithoutWispCommentsThroughDoltCLI pins the fresh-clone
// guard for the comments-keyset index: main migrations run before the ignored
// chain materializes the clone-local wisp_% tables, so migration 0056 must
// no-op its wisp half when wisp_comments is absent. A bare
// CREATE INDEX ... ON wisp_comments would fail at PREPARE ("table not found")
// and brick the first writable open. The durable comments half still runs.
// The seed below drops wisp_comments before re-applying 0056 in isolation:
// the bundle itself does create the table (main-plane 0021), so the absent
// case has to be staged rather than assumed.
func TestMigration0056NoopsWithoutWispCommentsThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "comments-keyset-no-wisps")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-wisps dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
seedSQL := fmt.Sprintf(`
DROP TABLE IF EXISTS wisp_comments;
DELETE FROM schema_migrations WHERE version = %d;
`, LatestVersion())
migrationSQL, err := mainSource.files.ReadFile("migrations/0056_add_comments_keyset_index.up.sql")
if err != nil {
t.Fatalf("read 0056 migration: %v", err)
}
runDoltSQL(t, dir, seedSQL+"\n"+string(migrationSQL))
// Wisp half no-oped: no wisp_comments table was created, no error raised.
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisp_comments'`, "0")
// Comments half ran: the durable composite index is present (one distinct
// index name; a composite spans three STATISTICS rows).
requireDoltCount(t, dir,
`SELECT COUNT(DISTINCT INDEX_NAME) AS c FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'comments' AND INDEX_NAME = 'idx_comments_issue_created_id'`, "1")
}
// TestMigration0065NoopsWithoutWispCommentsThroughDoltCLI is what licenses
// cliSubstituteAssumesWispTables: the fresh-bundle substitute for 0065 is a
// bare MODIFY that aborts the batch with "table not found" on a database that
// never synced wisp_comments (#4695/#4176), so replay callers use the frozen
// source text instead -- and this pins that the frozen text really does
// tolerate the absent table rather than merely being assumed to. A missing
// table makes the INFORMATION_SCHEMA probe yield NULL, and IF(NULL = 1, ...)
// takes the 'SELECT 1' branch.
func TestMigration0065NoopsWithoutWispCommentsThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "widen-wisp-comments-no-wisps")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-wisps dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
seedSQL := fmt.Sprintf(`
DROP TABLE IF EXISTS wisp_comments;
DELETE FROM schema_migrations WHERE version = %d;
`, LatestVersion())
migrationSQL, err := mainSource.files.ReadFile("migrations/0065_widen_wisp_comments_text.up.sql")
if err != nil {
t.Fatalf("read 0065 migration: %v", err)
}
runDoltSQL(t, dir, seedSQL+"\n"+string(migrationSQL))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisp_comments'`, "0")
// Control: the same absent table under the registered substitute is the
// failure this test exists to route around. Without it, "the frozen text
// tolerates it" would be indistinguishable from "nothing here can fail".
substitute := cliCompatibleMigrationSQL("0065_widen_wisp_comments_text.up.sql", string(migrationSQL))
if substitute == string(migrationSQL) {
t.Fatal("0065 has no registered CLI substitute; this test's control is vacuous")
}
if err := runDoltSQLExpectingError(t, dir, substitute); err == nil {
t.Fatal("0065 CLI substitute unexpectedly succeeded against an absent wisp_comments")
}
}
func TestMigration0053RepairsRigWispsThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "rig-repair")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create rig repair dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
const rigID = "schema-cli-rig"
const targetID = "schema-cli-target"
const sourceID = "schema-cli-source"
seedSQL := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS wisp_child_counters (
parent_id VARCHAR(255) PRIMARY KEY,
last_child INT NOT NULL DEFAULT 0
);
DELETE FROM schema_migrations WHERE version = %d;
INSERT INTO issues (id, title, description, design, acceptance_criteria, notes, status, priority, issue_type)
VALUES (%s, 'target', '', '', '', '', 'open', 2, 'task'),
(%s, 'source', '', '', '', '', 'open', 2, 'task');
INSERT INTO wisps (id, title, description, design, acceptance_criteria, notes, status, priority, issue_type, ephemeral)
VALUES (%s, 'Rig identity', '', '', '', '', 'open', 1, 'rig', 1);
INSERT INTO wisp_labels (issue_id, label) VALUES (%s, 'gt:rig');
INSERT INTO wisp_dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by, metadata)
VALUES (%s, %s, %s, 'blocks', NOW(), 'tester', JSON_OBJECT());
INSERT INTO dependencies (id, issue_id, depends_on_wisp_id, type, created_at, created_by, metadata)
VALUES (%s, %s, %s, 'blocks', NOW(), 'tester', JSON_OBJECT());
INSERT INTO wisp_events (id, issue_id, event_type, actor, created_at)
VALUES ('11111111-1111-1111-1111-111111111111', %s, 'created', 'tester', NOW());
INSERT INTO wisp_comments (id, issue_id, author, text, created_at)
VALUES ('22222222-2222-2222-2222-222222222222', %s, 'tester', 'durable identity', NOW());
INSERT INTO wisp_child_counters (parent_id, last_child) VALUES (%s, 7);
`, LatestVersion(),
doltSQLString(targetID), doltSQLString(sourceID), doltSQLString(rigID),
doltSQLString(rigID), doltSQLString(depid.New(rigID, targetID)),
doltSQLString(rigID), doltSQLString(targetID), doltSQLString(depid.New(sourceID, rigID)),
doltSQLString(sourceID), doltSQLString(rigID), doltSQLString(rigID),
doltSQLString(rigID), doltSQLString(rigID))
migrationSQL, err := mainSource.files.ReadFile("migrations/0053_repair_rig_wisps.up.sql")
if err != nil {
t.Fatalf("read 0053 migration: %v", err)
}
runDoltSQL(t, dir, seedSQL+"\n"+cliCompatibleMigrationSQL("0053_repair_rig_wisps.up.sql", string(migrationSQL)))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM issues WHERE id = 'schema-cli-rig' AND issue_type = 'rig' AND ephemeral = 0`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM wisps WHERE id = 'schema-cli-rig'`, "0")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM labels WHERE issue_id = 'schema-cli-rig' AND label = 'gt:rig'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM dependencies WHERE issue_id = 'schema-cli-rig' AND depends_on_issue_id = 'schema-cli-target'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM dependencies WHERE issue_id = 'schema-cli-source' AND depends_on_issue_id = 'schema-cli-rig' AND depends_on_wisp_id IS NULL`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM comments WHERE issue_id = 'schema-cli-rig'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM child_counters WHERE parent_id = 'schema-cli-rig' AND last_child = 7`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM wisp_child_counters WHERE parent_id = 'schema-cli-rig'`, "0")
}
func TestMigration0047RepairsMissingWispTablesThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "mixed-blocked-repair-no-wisps")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-wisps dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
// Reproduce #4695/#4176: a clone whose main cursor arrives at 0047 with
// the (dolt_ignore'd, never-synced) wisp tables entirely absent. Seed a
// normal issue-to-issue block edge so the recompute has real work to do.
const blockerID = "schema-cli-0047-blocker"
const blockedID = "schema-cli-0047-blocked"
seedSQL := fmt.Sprintf(`
DROP TABLE IF EXISTS wisp_child_counters;
DROP TABLE IF EXISTS wisp_comments;
DROP TABLE IF EXISTS wisp_events;
DROP TABLE IF EXISTS wisp_dependencies;
DROP TABLE IF EXISTS wisp_labels;
DROP TABLE IF EXISTS wisps;
DELETE FROM schema_migrations WHERE version = 47;
INSERT INTO issues (id, title, description, design, acceptance_criteria, notes, status, priority, issue_type)
VALUES (%s, 'blocker', '', '', '', '', 'open', 2, 'task'),
(%s, 'blocked', '', '', '', '', 'open', 2, 'task');
INSERT INTO dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by, metadata)
VALUES (%s, %s, %s, 'blocks', NOW(), 'tester', JSON_OBJECT());
`,
doltSQLString(blockerID), doltSQLString(blockedID),
doltSQLString(depid.New(blockedID, blockerID)), doltSQLString(blockedID), doltSQLString(blockerID))
migrationSQL, err := mainSource.files.ReadFile("migrations/0047_recompute_mixed_is_blocked.up.sql")
if err != nil {
t.Fatalf("read 0047 migration: %v", err)
}
// The repair DDL that ensureWispTablesForMixedBlockedRecompute issues in
// Go (plain CREATE TABLE IF NOT EXISTS, no PREPARE) precedes the frozen
// migration text, exactly mirroring preMigrationRepair's ordering.
repairSQL := wispsTableDDLForMigration0047 + "\n" + wispDependenciesTableDDLForMigration0047 + "\n"
runDoltSQL(t, dir, seedSQL+"\n"+repairSQL+string(migrationSQL))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisp_dependencies'`, "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM issues WHERE id = %s AND is_blocked = 1`, doltSQLString(blockedID)), "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM issues WHERE id = %s AND is_blocked = 0`, doltSQLString(blockerID)), "1")
}
// dependenciesIDRepairSQLForTest mirrors ensureDependenciesIDColumn's Go
// logic (migration_repairs.go) for a single known row, since plain SQL
// cannot reproduce depid.New's UUIDv5 derivation -- CLI tests seed exactly
// one dependency edge and precompute its id in Go. Unlike the real Go
// function, the "is this step already done" checks run here in Go (via a
// live query against dir) rather than as SET/IF/PREPARE dynamic SQL: the
// Dolt CLI does not reliably apply some prepared ALTER TABLE statements
// (see cli_migrations.go's cliCompatibleMigrationSQL doc and its
// per-migration direct-DDL substitutes for exactly this reason), so a
// PREPARE'd ADD COLUMN here is invisible to the very next statement in the
// same dolt sql -f batch. Deciding state in Go and emitting only direct
// (unprepared) DDL sidesteps that CLI limitation while still being safe to
// call more than once against the same database.
func dependenciesIDRepairSQLForTest(t *testing.T, dir, issueID, targetIssueID, precomputedID string) string {
t.Helper()
hasID := queryDoltCSV(t, dir, `
SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id'`)[0]["c"] != "0"
idIsPK := hasID && queryDoltCSV(t, dir, `
SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id' AND CONSTRAINT_NAME = 'PRIMARY'`)[0]["c"] != "0"
var b strings.Builder
if !hasID {
b.WriteString("ALTER TABLE dependencies ADD COLUMN id CHAR(36) NULL;\n")
fmt.Fprintf(&b, `UPDATE dependencies SET id = %s
WHERE issue_id = %s
AND depends_on_issue_id <=> %s
AND depends_on_wisp_id <=> NULL
AND depends_on_external <=> NULL
AND id IS NULL;
`, doltSQLString(precomputedID), doltSQLString(issueID), doltSQLString(targetIssueID))
}
if !idIsPK {
b.WriteString("ALTER TABLE dependencies MODIFY COLUMN id CHAR(36) NOT NULL;\n")
hasAnyPK := queryDoltCSV(t, dir, `
SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND CONSTRAINT_TYPE = 'PRIMARY KEY'`)[0]["c"] != "0"
if hasAnyPK {
b.WriteString("ALTER TABLE dependencies DROP PRIMARY KEY;\n")
}
b.WriteString("ALTER TABLE dependencies ADD PRIMARY KEY (id);\n")
}
return b.String()
}
func TestMigration0053NoopsWhenDependenciesMissingIDColumnThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "rig-repair-no-dep-id")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-dep-id dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
// Reproduce #4690: dependencies already in split-target/content-derived
// key shape (the uk_dep_* unique keys on issue_id + typed target, added
// by 0043, already give it a natural identity), but its surrogate id
// column was never added at all by this clone's history (a different
// migration path than this repo's 0043, which both drops the older
// generated depends_on_id column AND adds id in the same step). Seed one
// ordinary (non-rig) edge -- zero rig wisps exist, matching the
// reporter's confirmed-safe-to-no-op precondition -- and drop id,
// leaving the table keyless (Dolt supports this; the uk_dep_* unique
// keys still enforce the real identity).
const sourceID = "schema-cli-0053-source"
const targetID = "schema-cli-0053-target"
seedSQL := fmt.Sprintf(`
INSERT INTO issues (id, title, description, design, acceptance_criteria, notes, status, priority, issue_type)
VALUES (%s, 'source', '', '', '', '', 'open', 2, 'task'),
(%s, 'target', '', '', '', '', 'open', 2, 'task');
INSERT INTO dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by, metadata)
VALUES (%s, %s, %s, 'blocks', NOW(), 'tester', JSON_OBJECT());
ALTER TABLE dependencies DROP PRIMARY KEY;
ALTER TABLE dependencies DROP COLUMN id;
DELETE FROM schema_migrations WHERE version = 53;
`,
doltSQLString(sourceID), doltSQLString(targetID),
doltSQLString(depid.New(sourceID, targetID)), doltSQLString(sourceID), doltSQLString(targetID))
runDoltSQL(t, dir, seedSQL)
// dependenciesIDRepairSQLForTest mirrors ensureDependenciesIDColumn's Go
// logic (add the column, backfill, then restore it as the PRIMARY KEY --
// #2's fix, not just a plain NOT NULL column) for this one known edge.
repairID := depid.New(sourceID, targetID)
runDoltSQL(t, dir, dependenciesIDRepairSQLForTest(t, dir, sourceID, targetID, repairID))
migrationSQL, err := mainSource.files.ReadFile("migrations/0053_repair_rig_wisps.up.sql")
if err != nil {
t.Fatalf("read 0053 migration: %v", err)
}
runDoltSQL(t, dir, string(migrationSQL))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id' AND CONSTRAINT_NAME = 'PRIMARY'`, "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM dependencies WHERE issue_id = %s AND id = %s`, doltSQLString(sourceID), doltSQLString(repairID)), "1")
}
func TestMigration0057IsIdempotentOnAlreadyLongtextEventsThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "events-longtext-idempotent")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create events-longtext-idempotent dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
migrationSQL, err := mainSource.files.ReadFile("migrations/0057_events_value_columns_idempotent_longtext.up.sql")
if err != nil {
t.Fatalf("read 0057 migration: %v", err)
}
// events.old_value/new_value are already LONGTEXT after the fresh chain
// (0048's unconditional MODIFY). Re-running 0057's guarded SQL a second
// time -- simulating any repeat pass through this version -- must take
// the no-op branch for BOTH columns independently rather than re-issuing
// MODIFY on either one, which is exactly the drift 0048 itself lacks a
// guard against (#4353). This does not, and cannot, reproduce a raw
// re-execution of 0048's own frozen SQL by tooling outside bd's
// cursor-gated migrate chain; see 0057's header comment for that
// boundary.
runDoltSQL(t, dir, string(migrationSQL))
runDoltSQL(t, dir, string(migrationSQL))
requireDoltColumnShape(t, dir, "events", "old_value", "longtext", "YES")
requireDoltColumnShape(t, dir, "events", "new_value", "longtext", "YES")
}
// wispIsBlockedColumnSQLForTest decides, from live DB state, whether
// wisps.is_blocked (added by ignored/0006_add_wisp_is_blocked.up.sql) and its
// index still need adding, and emits direct DDL if so. Same rationale as
// dependenciesIDRepairSQLForTest: 0006's own ADD COLUMN/CREATE INDEX are
// PREPARE'd, and the Dolt CLI does not reliably apply a prepared ALTER TABLE
// in this path, so a fresh wisps table (created by this test's v47 repair
// injection, not through the ignored sequence) never actually gains the
// column before ignored/0007 needs it.
func wispIsBlockedColumnSQLForTest(t *testing.T, dir string) string {
t.Helper()
hasColumn := queryDoltCSV(t, dir, `
SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps' AND COLUMN_NAME = 'is_blocked'`)[0]["c"] != "0"
hasIndex := queryDoltCSV(t, dir, `
SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps' AND INDEX_NAME = 'idx_wisps_is_blocked'`)[0]["c"] != "0"
var b strings.Builder
if !hasColumn {
b.WriteString("ALTER TABLE wisps ADD COLUMN is_blocked TINYINT(1) NOT NULL DEFAULT 0;\n")
}
if !hasIndex {
b.WriteString("CREATE INDEX idx_wisps_is_blocked ON wisps(is_blocked, status);\n")
}
return b.String()
}
// TestFullChainFromPreWispsAndMissingDependenciesIDConvergesThroughDoltCLI is
// the acceptance test for the migration-chain-hardening branch: it seeds a
// database simultaneously in BOTH drifted shapes this branch fixes --
// #4695/#4176 (cursor arrives with the wisp tables absent) and #4690
// (dependencies never got its surrogate id column) -- and drives the
// migration chain all the way through the main sequence AND the ignored
// sequence to the latest version, the 0047 -> 0053 -> 0056 -> 0057 -> ignored path
// the narrower per-fix CLI tests above stop short of.
//
// Like every other CLI test in this file, this applies migration SQL text
// directly (via cliCompatibleMigrationSQL, the same substitution
// AllMigrationsSQL() uses) rather than calling schema.MigrateUp: the Dolt
// CLI test path here proves the frozen migration files plus the repair SQL
// converge to the correct end state, not the Go orchestration around them
// (runMigrations' commit atomicity is covered separately, and by a real cgo
// embedded-dolt test: see
// internal/storage/embeddeddolt's TestEmbeddedMigrateRepairedDependenciesIDColumnCommitsAtomicallyWithVersion53_4690).
func TestFullChainFromPreWispsAndMissingDependenciesIDConvergesThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "full-chain-convergence")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
// Build the schema genuinely bounded at v46 -- the real shape those
// migrations produce, not a hand-simplified stand-in -- using the same
// cliCompatibleMigrationSQL substitution AllMigrationsSQL() itself uses.
const seedCursor = 46
recordCursorSQL := func(cursorTable, name string, version int, data []byte) string {
sum := sha256.Sum256(data)
return fmt.Sprintf("INSERT IGNORE INTO %s (version, content_hash) VALUES (%d, %s);\n",
cursorTable, version, doltSQLString(hex.EncodeToString(sum[:])))
}
var bounded strings.Builder
bounded.WriteString(mainSource.bootstrapSQL())
bounded.WriteString(";\n")
for _, f := range mainSource.list() {
if f.version > seedCursor {
continue
}
data, err := mainSource.files.ReadFile(mainSource.dir + "/" + f.name)
if err != nil {
t.Fatalf("read %s: %v", f.name, err)
}
bounded.WriteString(cliCompatibleMigrationSQL(f.name, string(data)))
bounded.WriteString("\n")
bounded.WriteString(recordCursorSQL("schema_migrations", f.name, f.version, data))
}
runDoltSQL(t, dir, bounded.String())
// Reproduce #4695/#4176 AND #4690 simultaneously: the wisp tables never
// synced (dolt_ignore'd, absent even though the cursor is past the
// migrations that create them), and dependencies lost its surrogate id
// column (a different historical migration path than 0043).
const blockerID = "full-chain-blocker"
const blockedID = "full-chain-blocked"
depID := depid.New(blockedID, blockerID)
seedSQL := fmt.Sprintf(`
DROP TABLE IF EXISTS wisp_child_counters;
DROP TABLE IF EXISTS wisp_comments;
DROP TABLE IF EXISTS wisp_events;
DROP TABLE IF EXISTS wisp_dependencies;
DROP TABLE IF EXISTS wisp_labels;
DROP TABLE IF EXISTS wisps;
INSERT INTO issues (id, title, description, design, acceptance_criteria, notes, status, priority, issue_type)
VALUES (%s, 'blocker', '', '', '', '', 'open', 2, 'task'),
(%s, 'blocked', '', '', '', '', 'open', 2, 'task');
INSERT INTO dependencies (id, issue_id, depends_on_issue_id, type, created_at, created_by, metadata)
VALUES (%s, %s, %s, 'blocks', NOW(), 'tester', JSON_OBJECT());
ALTER TABLE dependencies DROP PRIMARY KEY;
ALTER TABLE dependencies DROP COLUMN id;
`,
doltSQLString(blockerID), doltSQLString(blockedID),
doltSQLString(depID), doltSQLString(blockedID), doltSQLString(blockerID))
runDoltSQL(t, dir, seedSQL)
// Drive the main sequence 47..latest (injecting each affected version's
// repair immediately before its own frozen SQL, mirroring
// preMigrationRepair's dispatch), then the ignored sequence 1..latest.
applyRemainingChain := func() {
var b strings.Builder
flush := func() {
if b.Len() == 0 {
return
}
runDoltSQL(t, dir, b.String())
b.Reset()
}
for _, f := range mainSource.list() {
if f.version <= seedCursor {
continue
}
switch f.version {
case 47:
b.WriteString(wispsTableDDLForMigration0047)
b.WriteString("\n")
b.WriteString(wispDependenciesTableDDLForMigration0047)
b.WriteString("\n")
case 53:
// dependenciesIDRepairSQLForTest queries live DB state, so
// the batch built so far must actually be applied first --
// it decides "hasID"/"idIsPK" from what's really in dir, not
// from statements still sitting unexecuted in b.
flush()
b.WriteString(dependenciesIDRepairSQLForTest(t, dir, blockedID, blockerID, depID))
b.WriteString("\n")
}
data, err := mainSource.files.ReadFile(mainSource.dir + "/" + f.name)
if err != nil {
t.Fatalf("read %s: %v", f.name, err)
}
if cliSubstituteAssumesWispTables(f.name) {
// These substitutes assume every wisp_* table already exists
// -- true for a fresh AllMigrationsSQL() bundle, where the
// ignored sequence's final committed shape is all that
// matters, but not here: this test's ordering deliberately
// matches real MigrateUp (main before ignored), so at this
// point only wisps and wisp_dependencies exist yet. The raw
// frozen text's own guards handle that correctly (0053's
// @has_wisps/@has_wisp_labels/... proven by
// TestMigration0053NoopsWithoutWispTablesThroughDoltCLI;
// 0065's by TestMigration0065NoopsWithoutWispCommentsThroughDoltCLI),
// so use it unsubstituted here.
b.WriteString(string(data))
} else {
b.WriteString(cliCompatibleMigrationSQL(f.name, string(data)))
}
b.WriteString("\n")
b.WriteString(recordCursorSQL("schema_migrations", f.name, f.version, data))
}
b.WriteString(ignoredSource.bootstrapSQL())
b.WriteString(";\n")
for _, f := range ignoredSource.list() {
if f.name == "0006_add_wisp_is_blocked.up.sql" {
// Same Dolt-CLI prepared-ALTER limitation as the v53 repair
// above: 0006's ADD COLUMN/CREATE INDEX are PREPARE'd, and a
// prepared ALTER TABLE is not reliably applied through this
// path (cli_migrations.go), so 0007 (which needs
// wisps.is_blocked to already exist) fails right after it.
// Decide state in Go and emit direct DDL instead, same
// technique as dependenciesIDRepairSQLForTest.
flush()
b.WriteString(wispIsBlockedColumnSQLForTest(t, dir))
b.WriteString("\n")
}
data, err := ignoredSource.files.ReadFile(ignoredSource.dir + "/" + f.name)
if err != nil {
t.Fatalf("read ignored %s: %v", f.name, err)
}
b.WriteString(cliCompatibleMigrationSQL(f.name, string(data)))
b.WriteString("\n")
b.WriteString(recordCursorSQL("ignored_schema_migrations", f.name, f.version, data))
}
flush()
}
applyRemainingChain()
assertConverged := func() {
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisp_dependencies'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps' AND COLUMN_NAME = 'no_history'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps' AND COLUMN_NAME = 'started_at'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisp_dependencies' AND INDEX_NAME = 'idx_wisp_dep_type'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.KEY_COLUMN_USAGE WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'dependencies' AND COLUMN_NAME = 'id' AND CONSTRAINT_NAME = 'PRIMARY'`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM dependencies WHERE id IS NULL`, "0")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM dependencies WHERE issue_id = %s AND id = %s`, doltSQLString(blockedID), doltSQLString(depID)), "1")
requireDoltCount(t, dir, `SELECT COUNT(*) AS c FROM dependencies`, "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM issues WHERE id = %s AND is_blocked = 1`, doltSQLString(blockedID)), "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM issues WHERE id = %s AND is_blocked = 0`, doltSQLString(blockerID)), "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM schema_migrations WHERE version = %d`, LatestVersion()), "1")
requireDoltCount(t, dir,
fmt.Sprintf(`SELECT COUNT(*) AS c FROM ignored_schema_migrations WHERE version = %d`, LatestIgnoredVersion()), "1")
}
assertConverged()
// Re-open: a real second schema.MigrateUp on this database would see
// both cursors already at latest and do nothing at all -- this harness
// applies raw migration SQL text directly (see the package doc comment
// above) rather than calling MigrateUp, so it has no cursor gate to
// reproduce that no-op with. Re-running every migration file
// unconditionally is NOT an equivalent stand-in: some of the
// cliCompatibleMigrationSQL substitutes this test also relies on (0054's
// and 0055's direct-DDL lease-column bundles, in particular) are
// deliberately single-application fresh-bundle DDL, not idempotent, and
// re-running them is a false failure orthogonal to this branch's fixes.
//
// What a re-open must prove for THIS branch is narrower and more
// precise: (1) both repair-detection helpers, queried fresh against live
// state, find nothing left to do -- proving ensureDependenciesIDColumn's
// and the wisp-tables repair's target state is fully converged, not just
// "ran once and happened to look right"; and (2) the three frozen
// migration files this branch's fixes actually touch (0047, 0053, 0057)
// are safe to re-run on the now-converged database without error or
// drift, which their own guards (not this test) are responsible for.
if repair := dependenciesIDRepairSQLForTest(t, dir, blockedID, blockerID, depID); repair != "" {
t.Fatalf("dependencies id repair has more work after convergence, want none:\n%s", repair)
}
if repair := wispIsBlockedColumnSQLForTest(t, dir); repair != "" {
t.Fatalf("wisps.is_blocked repair has more work after convergence, want none:\n%s", repair)
}
for _, name := range []string{
"0047_recompute_mixed_is_blocked.up.sql",
"0053_repair_rig_wisps.up.sql",
"0057_events_value_columns_idempotent_longtext.up.sql",
} {
data, err := mainSource.files.ReadFile(mainSource.dir + "/" + name)
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
runDoltSQL(t, dir, string(data))
}
assertConverged()
}
func TestWispDependenciesSplitTargetBackfillPrefersWispOverIssueThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "wisp-dependency-split")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create wisp dependency split dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
var repairSQL strings.Builder
for _, col := range wispDependenciesSplitTargetColumns() {
fmt.Fprintf(&repairSQL, "ALTER TABLE wisp_dependencies ADD COLUMN %s %s;\n", col.name, col.definition)
}
for _, stmt := range wispDependenciesSplitTargetBackfillSQL() {
repairSQL.WriteString(stmt)
repairSQL.WriteString(";\n")
}
const sourceID = "source-wisp"
const ambiguousID = "ambiguous-target"
seedSQL := fmt.Sprintf(`
CREATE TABLE issues (
id VARCHAR(255) PRIMARY KEY
);
CREATE TABLE wisps (
id VARCHAR(255) PRIMARY KEY
);
CREATE TABLE wisp_dependencies (
issue_id VARCHAR(255) NOT NULL,
depends_on_id VARCHAR(255) NOT NULL,
PRIMARY KEY (issue_id, depends_on_id)
);
INSERT INTO issues (id) VALUES (%s);
INSERT INTO wisps (id) VALUES (%s), (%s);
INSERT INTO wisp_dependencies (issue_id, depends_on_id) VALUES (%s, %s);
`, doltSQLString(ambiguousID),
doltSQLString(sourceID), doltSQLString(ambiguousID),
doltSQLString(sourceID), doltSQLString(ambiguousID))
runDoltSQL(t, dir, seedSQL+repairSQL.String())
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM wisp_dependencies WHERE issue_id = 'source-wisp' AND depends_on_wisp_id = 'ambiguous-target' AND depends_on_issue_id IS NULL AND depends_on_external IS NULL`, "1")
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM wisp_dependencies WHERE issue_id = 'source-wisp' AND depends_on_issue_id = 'ambiguous-target'`, "0")
}
func TestMigration0047HandlesLegacyWispDependenciesShape(t *testing.T) {
sql, err := os.ReadFile("migrations/0047_recompute_mixed_is_blocked.up.sql")
if err != nil {
t.Fatalf("read 0047 up migration: %v", err)
}
body := string(sql)
for _, want := range []string{
"@wisp_dependencies_needs_split",
"ALTER TABLE wisp_dependencies ADD COLUMN depends_on_issue_id",
"ALTER TABLE wisp_dependencies ADD COLUMN depends_on_wisp_id",
"ALTER TABLE wisp_dependencies ADD COLUMN depends_on_id VARCHAR(255) AS",
"cd.depends_on_issue_id",
"d.depends_on_wisp_id",
} {
if !strings.Contains(body, want) {
t.Fatalf("0047 migration missing legacy wisp_dependencies compatibility marker %q", want)
}
}
}
func TestCLICompatibleMigration0046UsesFreshSchemaDDLOnly(t *testing.T) {
got := cliCompatibleMigrationSQL("0046_add_is_blocked.up.sql", "source migration")
for _, want := range []string{
"ALTER TABLE issues ADD COLUMN is_blocked TINYINT(1) NOT NULL DEFAULT 0",
"CREATE INDEX idx_issues_is_blocked ON issues(is_blocked, status)",
} {
if !strings.Contains(got, want) {
t.Fatalf("0046 CLI migration missing %q", want)
}
}
for _, forbidden := range []string{
"UPDATE issues",
"WITH RECURSIVE",
"directly_blocked",
"recursively_blocked",
"parent-child",
} {
if strings.Contains(got, forbidden) {
t.Fatalf("0046 CLI migration contains dead backfill marker %q", forbidden)
}
}
}
func TestCLICompatibleMigration0008MatchesRuntimeChildCountersFK(t *testing.T) {
got := cliCompatibleMigrationSQL("0008_create_child_counters.up.sql", "source migration")
if want := "CONSTRAINT fk_counter_parent FOREIGN KEY (parent_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE"; !strings.Contains(got, want) {
t.Fatalf("0008 CLI migration missing %q", want)
}
}
func TestCLICompatibleMigration0032UsesDirectDropColumn(t *testing.T) {
got := cliCompatibleMigrationSQL("0032_drop_schema_migrations_applied_at.up.sql", "source migration")
if want := "ALTER TABLE schema_migrations DROP COLUMN applied_at"; !strings.Contains(got, want) {
t.Fatalf("0032 CLI migration missing %q", want)
}
for _, forbidden := range []string{
"PREPARE",
"EXECUTE",
"DEALLOCATE",
"@needs_drop",
} {
if strings.Contains(got, forbidden) {
t.Fatalf("0032 CLI migration contains prepared-DDL marker %q", forbidden)
}
}
}
func TestCLICompatibleMigration0049UsesDirectLongtextDDL(t *testing.T) {
got := cliCompatibleMigrationSQL("0049_longtext_large_content_columns.up.sql", "source migration")
for _, want := range []string{
"ALTER TABLE issues MODIFY COLUMN description LONGTEXT NOT NULL",
"MODIFY COLUMN design LONGTEXT NOT NULL",
"MODIFY COLUMN acceptance_criteria LONGTEXT NOT NULL",
"MODIFY COLUMN notes LONGTEXT NOT NULL",
"ALTER TABLE issues MODIFY COLUMN close_reason LONGTEXT DEFAULT ''",
"ALTER TABLE wisps MODIFY COLUMN description LONGTEXT NOT NULL DEFAULT ''",
"ALTER TABLE wisps MODIFY COLUMN close_reason LONGTEXT DEFAULT ''",
"ALTER TABLE comments MODIFY COLUMN text LONGTEXT NOT NULL",
} {
if !strings.Contains(got, want) {
t.Fatalf("0049 CLI migration missing %q", want)
}
}
for _, forbidden := range []string{
"PREPARE",
"EXECUTE",
"DEALLOCATE",
"@issues_needs_fix",
"@comments_needs_fix",
} {
if strings.Contains(got, forbidden) {
t.Fatalf("0049 CLI migration contains prepared-DDL marker %q", forbidden)
}
}
}
func TestCLICompatibleMigration0039PreservesRuntimeChildCountersFK(t *testing.T) {
got := cliCompatibleMigrationSQL("0039_drop_child_counters_fk.up.sql", "source migration")
if strings.TrimSpace(got) != "SELECT 1;" {
t.Fatalf("0039 CLI migration = %q, want SELECT 1", got)
}
}
func TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities(t *testing.T) {
got := AllMigrationsSQL()
for _, want := range []string{
"CONSTRAINT fk_counter_parent FOREIGN KEY (parent_id) REFERENCES issues(id) ON DELETE CASCADE ON UPDATE CASCADE",
"ALTER TABLE schema_migrations DROP COLUMN applied_at",
"ALTER TABLE issues MODIFY COLUMN close_reason LONGTEXT DEFAULT ''",
"ALTER TABLE comments MODIFY COLUMN text LONGTEXT NOT NULL",
// Trailing semicolons matter: without them these substrings also
// match the quoted DDL inside the source migrations' SET @sql
// guards, and the assertion would pass whether or not the direct
// statement is in the bundle.
"ALTER TABLE issues ADD COLUMN storage_class VARCHAR(16);",
"ALTER TABLE wisps ADD COLUMN storage_class VARCHAR(16);",
"ALTER TABLE wisp_comments MODIFY COLUMN text LONGTEXT NOT NULL;",
// 0066: same prepared-ALTER shape as 0060, same CLI no-op on 2.2.x.
"ALTER TABLE bd_events_journal ADD COLUMN actor VARCHAR(255) NOT NULL DEFAULT '';",
// 0067: two-plane prepared ADD COLUMN, same shape as 0060.
"ALTER TABLE issues ADD COLUMN current_revision BIGINT NOT NULL DEFAULT 1;",
"ALTER TABLE wisps ADD COLUMN current_revision BIGINT NOT NULL DEFAULT 1;",
// 0068: single-plane prepared ADD COLUMN, same shape as 0066 (no
// wisps twin — issue_versions has none), plus step 7's prepared
// MODIFY COLUMN (durable_state JSON -> LONGBLOB), same shape as 0065.
"ALTER TABLE issue_versions ADD COLUMN attribution_status VARCHAR(20) NOT NULL;",
"ALTER TABLE issue_versions MODIFY COLUMN durable_state LONGBLOB;",
// 0069: two prepared MODIFY COLUMNs on issue_versions (change_at and
// removed_at to DATETIME(6)), same shape as 0068's step 7.
"ALTER TABLE issue_versions MODIFY COLUMN change_at DATETIME(6) NOT NULL;",
"ALTER TABLE issue_versions MODIFY COLUMN removed_at DATETIME(6);",
// 0070: two-plane prepared ADD COLUMN, same shape as 0067.
"ALTER TABLE issues ADD COLUMN participation_generation BIGINT NULL;",
"ALTER TABLE wisps ADD COLUMN participation_generation BIGINT NULL;",
} {
if !strings.Contains(got, want) {
t.Fatalf("AllMigrationsSQL missing direct CLI DDL %q", want)
}
}
for _, forbidden := range []string{
"COLUMN_NAME = 'applied_at'",
"ALTER TABLE child_counters DROP FOREIGN KEY fk_counter_parent",
"@issues_cr_needs_fix",
"@comments_needs_fix",
// 0060 guards its ALTERs with PREPARE/EXECUTE for idempotent re-runs
// on upgraded databases. Only the main series is bundled, so this
// token can only come from that file's source text.
"COLUMN_NAME = 'storage_class'",
// Likewise 0065's guard variable. The generic check that no NEW
// migration reaches the bundle with a prepared ALTER lives in
// cli_prepared_ddl.go; these stay as per-migration anchors.
"@wisp_comments_needs_fix",
// 0066 guards its ALTER the same way; only its source text carries
// this probe (the events table's actor column is a bare CREATE).
"COLUMN_NAME = 'actor'",
// 0067 guards both planes' ALTERs the same way. Its two guard
// variables are the per-migration anchors; the probe token would
// also appear in the ignored twin, which is not bundled.
"@issues_cr_needs_add",
"@wisps_cr_needs_add",
// 0068 guards both of its ALTERs the same way; only its source text
// carries these probes.
"@issue_versions_as_needs_add",
"@issue_versions_ds_needs_retype",
// 0069 guards both of its MODIFYs the same way.
"@issue_versions_change_at_needs_widen",
"@issue_versions_removed_at_needs_widen",
// 0070 guards both planes' ALTERs the same way, mirroring 0067's pair.
"@issues_pg_needs_add",
"@wisps_pg_needs_add",
} {
if strings.Contains(got, forbidden) {
t.Fatalf("AllMigrationsSQL contains source prepared-DDL guard %q", forbidden)
}
}
}
func TestAllMigrationsSQLAppliesThroughDoltCLIAndRecordsLatestVersion(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "cli-bundle")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create CLI bundle dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
rows := queryDoltCSV(t, dir, `
SELECT COALESCE(MAX(version), 0) AS max_version, COUNT(*) AS version_count
FROM schema_migrations`)
if len(rows) != 1 {
t.Fatalf("schema_migrations query returned %d rows, want 1", len(rows))
}
want := strconv.Itoa(LatestVersion())
if got := rows[0]["max_version"]; got != want {
t.Fatalf("MAX(version) = %s, want %s", got, want)
}
if got := rows[0]["version_count"]; got != want {
t.Fatalf("COUNT(*) = %s, want %s", got, want)
}
requireDoltNoRows(t, dir, `
SELECT column_name
FROM information_schema.columns
WHERE table_schema = DATABASE()
AND table_name = 'schema_migrations'
AND column_name = 'applied_at'`, "schema_migrations.applied_at")
requireDoltFKRules(t, dir, "fk_comments_issue", "CASCADE", "CASCADE")
requireDoltColumnShape(t, dir, "comments", "text", "longtext", "NO")
requireDoltColumnShape(t, dir, "bd_events_journal", "actor", "varchar(255)", "NO")
requireDoltColumnShape(t, dir, "issues", "description", "longtext", "NO")
requireDoltColumnShape(t, dir, "wisps", "description", "longtext", "NO")
requireDoltColumnShape(t, dir, "wisps", "no_history", "tinyint(1)", "YES")
requireDoltColumnShape(t, dir, "wisps", "started_at", "datetime", "YES")
requireDoltColumnShape(t, dir, "wisps", "wisp_type", "varchar(32)", "YES")
requireDoltColumnShape(t, dir, "issues", "storage_class", "varchar(16)", "YES")
requireDoltColumnShape(t, dir, "wisps", "storage_class", "varchar(16)", "YES")
}
func runDoltCommand(t *testing.T, dir string, args ...string) {
t.Helper()
cmd := exec.Command("dolt", args...)
cmd.Dir = dir
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("dolt %v failed in %s: %v\nOutput: %s", args, dir, err, output)
}
}
func runDoltSQL(t *testing.T, dir, query string) {
t.Helper()
sqlFile := filepath.Join(t.TempDir(), "migration-bundle.sql")
if err := os.WriteFile(sqlFile, []byte(query), 0o644); err != nil {
t.Fatalf("write dolt sql file: %v", err)
}
runDoltCommand(t, dir, "sql", "-f", sqlFile)
}
// runDoltSQLExpectingError runs query the same way runDoltSQL does but returns
// the error instead of failing the test, so a caller can assert that a
// statement really is rejected.
func runDoltSQLExpectingError(t *testing.T, dir, query string) error {
t.Helper()
sqlFile := filepath.Join(t.TempDir(), "migration-bundle.sql")
if err := os.WriteFile(sqlFile, []byte(query), 0o644); err != nil {
t.Fatalf("write dolt sql file: %v", err)
}
cmd := exec.Command("dolt", "sql", "-f", sqlFile)
cmd.Dir = dir
_, err := cmd.CombinedOutput()
return err
}
func queryDoltCSV(t *testing.T, dir, query string) []map[string]string {
t.Helper()
cmd := exec.Command("dolt", "sql", "-q", query, "-r", "csv")
cmd.Dir = dir
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("dolt sql query failed in %s: %v\nQuery: %s\nOutput: %s", dir, err, query, output)
}
trimmed := strings.TrimSpace(string(output))
if trimmed == "" {
return nil
}
records, err := csv.NewReader(strings.NewReader(trimmed)).ReadAll()
if err != nil {
t.Fatalf("parse dolt CSV output: %v\nRaw: %s", err, output)
}
if len(records) < 2 {
return nil
}
headers := records[0]
rows := make([]map[string]string, 0, len(records)-1)
for _, record := range records[1:] {
row := make(map[string]string, len(headers))
for i, header := range headers {
if i < len(record) {
row[header] = record[i]
}
}
rows = append(rows, row)
}
return rows
}
func requireDoltNoRows(t *testing.T, dir, query, subject string) {
t.Helper()
if rows := queryDoltCSV(t, dir, query); len(rows) != 0 {
t.Fatalf("%s query returned %d rows, want none: %v", subject, len(rows), rows)
}
}
func requireDoltCount(t *testing.T, dir, query, want string) {
t.Helper()
rows := queryDoltCSV(t, dir, query)
if len(rows) != 1 {
t.Fatalf("count query returned %d rows, want 1: %v", len(rows), rows)
}
if got := rows[0]["c"]; got != want {
t.Fatalf("count query returned %s, want %s\nQuery: %s", got, want, query)
}
}
func requireDoltFKRules(t *testing.T, dir, constraintName, wantUpdate, wantDelete string) {
t.Helper()
rows := queryDoltCSV(t, dir, fmt.Sprintf(`
SELECT update_rule AS update_rule, delete_rule AS delete_rule
FROM information_schema.referential_constraints
WHERE constraint_schema = DATABASE()
AND constraint_name = %s`, doltSQLString(constraintName)))
if len(rows) != 1 {
t.Fatalf("%s FK query returned %d rows, want 1: %v", constraintName, len(rows), rows)
}
if got := rows[0]["update_rule"]; got != wantUpdate {
t.Fatalf("%s UPDATE_RULE = %s, want %s", constraintName, got, wantUpdate)
}
if got := rows[0]["delete_rule"]; got != wantDelete {
t.Fatalf("%s DELETE_RULE = %s, want %s", constraintName, got, wantDelete)
}
}
func requireDoltColumnShape(t *testing.T, dir, tableName, columnName, wantType, wantNullable string) {
t.Helper()
rows := queryDoltCSV(t, dir, fmt.Sprintf(`
SELECT column_type AS column_type, is_nullable AS is_nullable
FROM information_schema.columns
WHERE table_schema = DATABASE()
AND table_name = %s
AND column_name = %s`, doltSQLString(tableName), doltSQLString(columnName)))
if len(rows) != 1 {
t.Fatalf("%s.%s column query returned %d rows, want 1: %v", tableName, columnName, len(rows), rows)
}
if got := rows[0]["column_type"]; got != wantType {
t.Fatalf("%s.%s COLUMN_TYPE = %s, want %s", tableName, columnName, got, wantType)
}
if got := rows[0]["is_nullable"]; got != wantNullable {
t.Fatalf("%s.%s IS_NULLABLE = %s, want %s", tableName, columnName, got, wantNullable)
}
}
func doltSQLString(value string) string {
return "'" + strings.ReplaceAll(value, "'", "''") + "'"
}
func TestStageSchemaTablesSkipsIgnoredTables(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
mock.ExpectQuery(`(?s)SELECT s\.table_name, s\.staged\s+FROM dolt_status s\s+WHERE NOT EXISTS`).
WillReturnRows(sqlmock.NewRows([]string{"table_name", "staged"}).
AddRow("schema_migrations", false))
mock.ExpectQuery(`(?s)SELECT t\.TABLE_NAME\s+FROM INFORMATION_SCHEMA\.TABLES t\s+WHERE .*NOT EXISTS`).
WillReturnRows(sqlmock.NewRows([]string{"TABLE_NAME"}).
AddRow("schema_migrations"))
mock.ExpectQuery(`CALL DOLT_ADD\('-f', \?\)`).
WithArgs("schema_migrations").
WillReturnRows(sqlmock.NewRows([]string{"status"}))
staged, err := stageSchemaTables(context.Background(), db, map[string]dirtyTableState{})
if err != nil {
t.Fatalf("stageSchemaTables: %v", err)
}
if !staged {
t.Fatal("stageSchemaTables staged = false, want true")
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
func TestUnstageIgnoredTablesResetsExistingIgnoredTables(t *testing.T) {
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
mock.ExpectQuery(`(?s)SELECT s\.table_name, s\.staged\s+FROM dolt_status s\s+WHERE EXISTS`).
WillReturnRows(sqlmock.NewRows([]string{"table_name", "staged"}).
AddRow("ignored_schema_migrations", true).
AddRow("wisp_dependencies", true).
AddRow("wisps", false))
mock.ExpectQuery(`CALL DOLT_RESET\(\?\)`).
WithArgs("ignored_schema_migrations").
WillReturnRows(sqlmock.NewRows([]string{"status"}))
mock.ExpectQuery(`CALL DOLT_RESET\(\?\)`).
WithArgs("wisp_dependencies").
WillReturnRows(sqlmock.NewRows([]string{"status"}))
if err := unstageIgnoredTables(context.Background(), db); err != nil {
t.Fatalf("unstageIgnoredTables: %v", err)
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
}
// showColumnsRows builds a SHOW COLUMNS result with one row per supplied field
// name, mirroring the Field/Type/Null/Key/Default/Extra shape Dolt returns.
func showColumnsRows(fields ...string) *sqlmock.Rows {
rows := sqlmock.NewRows([]string{"Field", "Type", "Null", "Key", "Default", "Extra"})
for _, f := range fields {
rows.AddRow(f, "char(64)", "YES", "", nil, "")
}
return rows
}
func TestHasContentHashColumnUsesShowColumns(t *testing.T) {
t.Run("column present", func(t *testing.T) {
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
defer db.Close()
mock.ExpectQuery(`SHOW COLUMNS FROM schema_migrations LIKE 'content_hash'`).
WillReturnRows(showColumnsRows("content_hash"))
has, err := mainSource.hasContentHashColumn(context.Background(), db)
if err != nil {
t.Fatalf("hasContentHashColumn: %v", err)
}
if !has {
t.Fatal("has = false, want true when SHOW COLUMNS returns the column")
}
if err := mock.ExpectationsWereMet(); err != nil {
t.Fatalf("unmet sql expectations: %v", err)
}
})
t.Run("column absent", func(t *testing.T) {
db, mock, _ := sqlmock.New()
defer db.Close()
mock.ExpectQuery(`SHOW COLUMNS FROM schema_migrations`).
WillReturnRows(showColumnsRows())
has, err := mainSource.hasContentHashColumn(context.Background(), db)
if err != nil {
t.Fatalf("hasContentHashColumn: %v", err)
}
if has {
t.Fatal("has = true, want false when SHOW COLUMNS returns no rows")
}
})
t.Run("missing table reports false without error", func(t *testing.T) {
// The old INFORMATION_SCHEMA probe returned count 0 for an absent table;
// SHOW COLUMNS errors with 1146 instead. That error must be swallowed so a
// not-yet-created cursor table still reports "no content_hash column".
db, mock, _ := sqlmock.New()
defer db.Close()
mock.ExpectQuery(`SHOW COLUMNS FROM schema_migrations`).
WillReturnError(errors.New("Error 1146: Table 'beads.schema_migrations' doesn't exist"))
has, err := mainSource.hasContentHashColumn(context.Background(), db)
if err != nil {
t.Fatalf("hasContentHashColumn returned error for missing table, want nil: %v", err)
}
if has {
t.Fatal("has = true, want false for a missing cursor table")
}
})
t.Run("non-matching field is rejected", func(t *testing.T) {
// '_' is a LIKE single-char wildcard, so 'contentXhash' could slip past
// the server-side filter; the exact Field comparison must reject it.
db, mock, _ := sqlmock.New()
defer db.Close()
mock.ExpectQuery(`SHOW COLUMNS FROM schema_migrations`).
WillReturnRows(showColumnsRows("contentXhash"))
has, err := mainSource.hasContentHashColumn(context.Background(), db)
if err != nil {
t.Fatalf("hasContentHashColumn: %v", err)
}
if has {
t.Fatal("has = true, want false for a column that only matches the LIKE wildcard")
}
})
t.Run("propagates unexpected errors", func(t *testing.T) {
db, mock, _ := sqlmock.New()
defer db.Close()
mock.ExpectQuery(`SHOW COLUMNS FROM schema_migrations`).
WillReturnError(errors.New("connection refused"))
if _, err := mainSource.hasContentHashColumn(context.Background(), db); err == nil {
t.Fatal("expected unexpected error to propagate, got nil")
}
})
}
// TestHasContentHashColumnMatchesInformationSchemaOnDolt proves on a real Dolt
// database that the SHOW COLUMNS probe returns the same answer as the retired
// INFORMATION_SCHEMA.COLUMNS probe, in both the present and absent states.
func TestHasContentHashColumnMatchesInformationSchemaOnDolt(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "content-hash-probe")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create probe dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
const table = "schema_migrations"
// The retired probe: COUNT(*) over INFORMATION_SCHEMA.COLUMNS.
infoSchemaHas := func() bool {
rows := queryDoltCSV(t, dir, fmt.Sprintf(`
SELECT COUNT(*) AS cnt
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = '%s' AND COLUMN_NAME = 'content_hash'`, table))
return len(rows) == 1 && rows[0]["cnt"] == "1"
}
// The new probe: SHOW COLUMNS ... LIKE, matching the Field name exactly.
showColumnsHas := func() bool {
rows := queryDoltCSV(t, dir, fmt.Sprintf("SHOW COLUMNS FROM %s LIKE 'content_hash'", table))
for _, r := range rows {
for _, v := range r {
if v == "content_hash" {
return true
}
}
}
return false
}
// State 1: cursor table carries content_hash (matches bootstrapSQL).
runDoltSQL(t, dir, fmt.Sprintf(
"CREATE TABLE %s (version INT PRIMARY KEY, applied_at DATETIME, content_hash CHAR(64))", table))
if !showColumnsHas() {
t.Fatal("SHOW COLUMNS reported no content_hash on a table that has it")
}
if got, want := showColumnsHas(), infoSchemaHas(); got != want {
t.Fatalf("with content_hash: SHOW COLUMNS=%v, INFORMATION_SCHEMA=%v", got, want)
}
// State 2: same table without content_hash.
runDoltSQL(t, dir, fmt.Sprintf("ALTER TABLE %s DROP COLUMN content_hash", table))
if showColumnsHas() {
t.Fatal("SHOW COLUMNS reported content_hash on a table that lacks it")
}
if got, want := showColumnsHas(), infoSchemaHas(); got != want {
t.Fatalf("without content_hash: SHOW COLUMNS=%v, INFORMATION_SCHEMA=%v", got, want)
}
}
type mockDB struct{}
func (m *mockDB) ExecContext(_ context.Context, _ string, _ ...any) (sql.Result, error) {
return nil, nil
}
func (m *mockDB) QueryContext(_ context.Context, _ string, _ ...any) (*sql.Rows, error) {
panic("not called")
}
func (m *mockDB) QueryRowContext(_ context.Context, _ string, _ ...any) *sql.Row {
panic("not called")
}
// wispsAbsentDB extends mockDB's blind-success ExecContext / panicking
// QueryContext with TWO answerable QueryRowContext queries, same pattern:
// the ignored/7 and ignored/15 pre-migration repairs (both dispatch to
// ensureWispIsBlockedForRecompute) each probe whether wisps exists before
// touching it, and *sql.Row has no public constructor outside a real
// database/sql driver -- mockDB itself cannot fake a scan result. A small
// embedded sqlmock answers exactly those two queries with "table absent"
// (0), so the repair takes its early no-op branch on both firings; any
// OTHER QueryRowContext this pass might issue still surfaces loudly, via
// sqlmock's own unmet/unexpected-call error rather than a silent success.
type wispsAbsentDB struct {
mockDB
real *sql.DB
}
func newWispsAbsentDB(t *testing.T) *wispsAbsentDB {
t.Helper()
failOnSwallowedAdvisory(t)
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("sqlmock.New: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
// Registered twice, once per firing of the {ignored_schema_migrations, 7}
// and {ignored_schema_migrations, 15} repairs; go-sqlmock v1.5.2 predates
// ExpectedQuery.Times, so the same expectation is set up twice instead of
// asking for a repeat count.
for i := 0; i < 2; i++ {
mock.ExpectQuery(`(?s)SELECT COUNT\(\*\) FROM INFORMATION_SCHEMA\.TABLES.*TABLE_NAME = \?`).
WithArgs("wisps").
WillReturnRows(sqlmock.NewRows([]string{"c"}).AddRow(0))
}
t.Cleanup(func() {
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("wispsAbsentDB: unmet sql expectations: %v", err)
}
})
return &wispsAbsentDB{real: db}
}
func (w *wispsAbsentDB) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row {
return w.real.QueryRowContext(ctx, query, args...)
}
func TestRunMigrationsStderrOutput(t *testing.T) {
var buf bytes.Buffer
orig := stderr
stderr = &buf
defer func() { stderr = orig }()
// mockDB.QueryRowContext panics, so stub issueRowCounter (be-8ja's
// large-rig check, which now runs unconditionally at the top of
// runMigrations) to report a small rig — no warning line, so this test's
// line-count assertion stays scoped to the per-migration progress lines.
origCounter := issueRowCounter
issueRowCounter = func(context.Context, DBConn) (int64, error) { return 0, nil }
defer func() { issueRowCounter = origCounter }()
// Bounded below migration 40: migrations 40/41 carry preMigrationRepairs and
// CALL DOLT_* bodies that issue real queries via mockDB.QueryRowContext /
// QueryContext, and the repairs at 47 and 53 issue real INFORMATION_SCHEMA
// probes (see TestPreMigrationRepairScopedToMain0047 /
// TestPreMigrationRepairScopedToMain0053) — none of which the panic-stub mock
// supports. This test only exercises the stderr lines, not the repair or
// proc-drain paths.
n, err := runMigrations(context.Background(), &mockDB{}, mainSource, 0, 39, false)
if err != nil {
t.Fatalf("runMigrations: %v", err)
}
if n == 0 {
t.Fatal("expected at least one migration to run")
}
got := buf.String()
if !strings.Contains(got, "Applying migration ") {
t.Errorf("expected stderr to contain 'Applying migration ', got: %q", got)
}
if !strings.Contains(got, " done (") {
t.Errorf("expected stderr to contain a ' done (Ns)' timing line, got: %q", got)
}
// One "Applying" line + one "done" line per migration.
lines := strings.Split(strings.TrimRight(got, "\n"), "\n")
if len(lines) != 2*n {
t.Errorf("expected %d stderr lines (2 per migration), got %d", 2*n, len(lines))
}
}
// TestRunMigrationsUsesProvidedSource verifies that runMigrations operates on
// the supplied migrationSource rather than always falling back to mainSource.
// Regression test for the bug where ignoredSource.migrate() silently ran main
// migrations and left ignored_schema_migrations empty (no wisp tables).
func TestRunMigrationsUsesProvidedSource(t *testing.T) {
orig := stderr
stderr = &bytes.Buffer{}
defer func() { stderr = orig }()
// Stub the large-rig row counter for the same reason as
// TestRunMigrationsStderrOutput: mockDB.QueryRowContext panics.
origCounter := issueRowCounter
issueRowCounter = func(context.Context, DBConn) (int64, error) { return 0, nil }
defer func() { issueRowCounter = origCounter }()
// Bounded below migration 40 for the same reason as
// TestRunMigrationsStderrOutput: mockDB can't answer the queries the
// preMigrationRepairs at versions 40/41/47/53 (and 40/41's CALL DOLT_*
// bodies) issue on the pinned connection.
main, err := runMigrations(context.Background(), &mockDB{}, mainSource, 0, 39, false)
if err != nil {
t.Fatalf("runMigrations(mainSource): %v", err)
}
// Same upTo cap as the mainSource call: the source-threading regression
// this test guards (hardcoding mainSource) is only detectable when both
// calls share a bound, so their counts collapse to equal under the bug.
// ignoredSource has 24 migrations, so 39 is a no-op on correct behavior.
// A shared bound below 15 would dodge the ignored/15 repair too, but main
// and ignored both number their first 14 files contiguously (1..14, no
// gaps in either series), so any bound < 15 gives main==ignored and would
// fail this test's own regression check for an unrelated reason.
// wispsAbsentDB answers both the {ignored_schema_migrations, 7} and
// {..., 15} repairs' probes (ensureWispIsBlockedForRecompute dispatches
// at both) instead of dodging them.
ignored, err := runMigrations(context.Background(), newWispsAbsentDB(t), ignoredSource, 0, 39, false)
if err != nil {
t.Fatalf("runMigrations(ignoredSource): %v", err)
}
if main == 0 || ignored == 0 {
t.Fatalf("expected non-zero counts; main=%d ignored=%d", main, ignored)
}
if main == ignored {
t.Errorf("runMigrations ignored its source argument: main and ignored both returned %d", main)
}
}
// TestRunMigrationsLargeRigNoticeOnlyOnMainSource pins the round-2 review
// fix: MigrateUp calls runMigrations once for mainSource and once for
// ignoredSource in the same pass (schema.go's MigrateUp). Without gating the
// large-rig notice to the main-source pass, a large rig with pending
// migrations in both sources would print the "one-shot" warning twice (and
// issue the COUNT(*) query twice). The notice must fire on the mainSource
// call and stay silent on the ignoredSource call.
func TestRunMigrationsLargeRigNoticeOnlyOnMainSource(t *testing.T) {
origCounter := issueRowCounter
issueRowCounter = func(context.Context, DBConn) (int64, error) { return 49_187, nil }
defer func() { issueRowCounter = origCounter }()
var mainBuf bytes.Buffer
orig := stderr
stderr = &mainBuf
// Bounded below migration 40 for the same reason as
// TestRunMigrationsStderrOutput: the panic-stub mock can't answer the
// queries versions 40/41/47/53 issue on the pinned connection.
if _, err := runMigrations(context.Background(), &mockDB{}, mainSource, 0, 39, false); err != nil {
stderr = orig
t.Fatalf("runMigrations(mainSource): %v", err)
}
if !strings.Contains(mainBuf.String(), "Large rig detected") {
stderr = orig
t.Errorf("expected mainSource pass to emit the large-rig notice; got: %q", mainBuf.String())
}
var ignoredBuf bytes.Buffer
stderr = &ignoredBuf
// This assertion (no large-rig notice on the ignored-source pass) does
// not depend on applying every ignored migration, so bound below the
// {ignored_schema_migrations, 7} repair rather than answering it (7, not
// 15: ensureWispIsBlockedForRecompute now dispatches at both versions).
_, err := runMigrations(context.Background(), &mockDB{}, ignoredSource, 0, 6, false)
stderr = orig
if err != nil {
t.Fatalf("runMigrations(ignoredSource): %v", err)
}
if strings.Contains(ignoredBuf.String(), "Large rig detected") {
t.Errorf("expected ignoredSource pass to stay silent on the large-rig notice (main-source pass already warned); got: %q", ignoredBuf.String())
}
}
func TestIgnoredMigration0017AddsWispsDeferUntilIndexThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "wisps-defer-until-index")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create wisps defer_until index dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
migrationSQL, err := ignoredSource.files.ReadFile("migrations/ignored/0017_add_wisps_defer_until_index.up.sql")
if err != nil {
t.Fatalf("read ignored 0017 up migration: %v", err)
}
const indexCountQuery = `SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps' AND INDEX_NAME = 'idx_wisps_defer_until'`
runDoltSQL(t, dir, string(migrationSQL))
requireDoltCount(t, dir, indexCountQuery, "1")
// Idempotent: re-applying must not error or duplicate the index.
runDoltSQL(t, dir, string(migrationSQL))
requireDoltCount(t, dir, indexCountQuery, "1")
}
func TestIgnoredMigration0017NoopsWithoutWispsTableThroughDoltCLI(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "wisps-defer-until-index-no-wisps")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create no-wisps dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
runDoltSQL(t, dir, AllMigrationsSQL())
runDoltSQL(t, dir, `
DROP TABLE IF EXISTS wisp_child_counters;
DROP TABLE IF EXISTS wisp_comments;
DROP TABLE IF EXISTS wisp_events;
DROP TABLE IF EXISTS wisp_dependencies;
DROP TABLE IF EXISTS wisp_labels;
DROP TABLE IF EXISTS wisps;
`)
migrationSQL, err := ignoredSource.files.ReadFile("migrations/ignored/0017_add_wisps_defer_until_index.up.sql")
if err != nil {
t.Fatalf("read ignored 0017 up migration: %v", err)
}
runDoltSQL(t, dir, string(migrationSQL))
requireDoltCount(t, dir,
`SELECT COUNT(*) AS c FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'wisps'`, "0")
}
// TestShowTableExistsMatchesInformationSchemaOnDolt proves the per-open table
// probe (the unfiltered SHOW TABLES listing, names compared case-insensitively)
// agrees with the INFORMATION_SCHEMA.TABLES count it replaced, on a real Dolt.
// It covers every table the open path probes — both cursor tables, the
// sentinel and floored-sentinel tables, the column sentinel's carrier and the
// untrack scratch table — under bd's own dolt_ignore patterns, so all but
// schema_migrations live only in the working set. It also covers a name whose
// '_' LIKE-matches a different table, and a table stored in another case: Dolt
// resolves and counts table names case-insensitively there, but its SHOW
// TABLES LIKE filter does not.
func TestShowTableExistsMatchesInformationSchemaOnDolt(t *testing.T) {
testutil.RequireDoltBinary(t)
dir := filepath.Join(t.TempDir(), "show-tables-probe")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatalf("create probe dir: %v", err)
}
runDoltCommand(t, dir, "init", "--name", "test", "--email", "test@example.com")
infoSchemaHas := func(table string) bool {
rows := queryDoltCSV(t, dir, fmt.Sprintf(`
SELECT COUNT(*) AS cnt
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = '%s'`, table))
return len(rows) == 1 && rows[0]["cnt"] == "1"
}
// The same listing and comparison showTableExists makes.
showTablesHas := func(table string) bool {
for _, r := range queryDoltCSV(t, dir, "SHOW TABLES") {
for _, v := range r {
if strings.EqualFold(v, table) {
return true
}
}
}
return false
}
agree := func(state, table string, want bool) {
t.Helper()
if got := showTablesHas(table); got != want {
t.Fatalf("%s: SHOW TABLES reported %s present=%v, want %v", state, table, got, want)
}
if got := infoSchemaHas(table); got != want {
t.Fatalf("%s: INFORMATION_SCHEMA reported %s present=%v, want %v", state, table, got, want)
}
}
probed := []string{mainSource.cursorTable, ignoredSource.cursorTable, ignoredCursorUntrackTempTable}
probed = append(probed, ignoredSource.sentinelTables...)
for _, st := range ignoredSource.sentinelFlooredTables {
probed = append(probed, st.table)
}
for _, sc := range ignoredSource.sentinelColumns {
probed = append(probed, sc.table)
}
for _, table := range probed {
agree("empty database", table, false)
}
for _, pattern := range doltIgnorePatterns {
runDoltSQL(t, dir, fmt.Sprintf("REPLACE INTO dolt_ignore VALUES ('%s', true)", pattern))
}
for _, gated := range versionGatedDoltIgnorePatterns {
runDoltSQL(t, dir, fmt.Sprintf("REPLACE INTO dolt_ignore VALUES ('%s', true)", gated.pattern))
}
runDoltSQL(t, dir, "CREATE TABLE "+mainSource.cursorTable+" (version INT PRIMARY KEY)")
runDoltSQL(t, dir, "CALL DOLT_COMMIT('-Am', 'committed cursor')")
for _, table := range probed {
if table == mainSource.cursorTable || table == ignoredCursorUntrackTempTable {
continue
}
runDoltSQL(t, dir, "CREATE TABLE "+table+" (id VARCHAR(64) PRIMARY KEY)")
}
requireDoltNoRows(t, dir, "SELECT table_name FROM dolt_status", "dolt_status after creating only dolt_ignored tables")
for _, table := range probed {
agree("migrated shape, dolt_ignored tables in the working set", table, table != ignoredCursorUntrackTempTable)
}
// 'schemaxmigrations' LIKE-matches 'schema_migrations'; only the name
// comparison keeps it from reading as the cursor table once that is gone.
runDoltSQL(t, dir, "CREATE TABLE schemaxmigrations (id INT PRIMARY KEY)")
runDoltSQL(t, dir, "DROP TABLE "+mainSource.cursorTable)
agree("LIKE wildcard lookalike", mainSource.cursorTable, false)
// A table stored in another case is the same table to Dolt.
runDoltSQL(t, dir, "RENAME TABLE bd_events_seq TO bd_events_seq_case")
runDoltSQL(t, dir, "RENAME TABLE bd_events_seq_case TO BD_EVENTS_SEQ")
agree("table stored in another case", "bd_events_seq", true)
}