beads/internal/storage/issueops/row_lock_guard_test.go
Jim Wordelman ce70b261c5
storage: add participation_generation (migration 0070) and the versioned-history write fence (#6943)
* schema+storage: restore 0068, move participation_generation to 0070 (be-blvmg)

MAYOR RULING 2026-09-29 on tracker be-waare: migration 0068 shipped to
origin/main as attribution_status-only (be-v33pa). be-h89oq's lineage had
appended participation_generation steps 4-5 to that same file while it was
still unmerged, which check-migration-hygiene.sh Check C correctly rejects
once 0068 is frozen on main. This restores 0068 to origin/main's exact
content and re-homes the participation_generation column (issues + wisps,
design section 16.3 steps 4-5, be-dt74u amendment) in a new migration 0070,
with its clone-local ignored/0028 twin for wisps. Ports back the
participation_generation write-fence consumer code (RecordVersionInTx in
internal/storage/issueops/version_history.go and its callers) and the
dualwrite contract tests that exercise it.

RED: the CLI-bundle direct-DDL override for 0070's guarded PREPARE/ALTER
pair is not yet registered in cli_migrations.go, so
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities fail --
correctly: a fresh CLI-built bundle would silently omit the column
(dolthub/dolt#11345) while a runtime-migrated store has it.

git diff origin/main -- internal/storage/schema/migrations/0068_*.sql is
empty, confirming 0068 is untouched.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* schema: register CLI-bundle direct-DDL override for migration 0070 (be-blvmg)

GREEN: adds cliMigration0070AddParticipationGeneration and its dispatch
entries (cliCompatibleMigrationSQL, cliSubstituteAssumesWispTables), the
same pre-2.3 Dolt CLI PREPARE/ALTER escape hatch 0067's wisps.current_revision
and 0068's attribution_status already use (dolthub/dolt#11345). Makes
TestBundleMigrationsWithPreparedALTERAreOverriddenOrJustified,
TestMigration0070AddsParticipationGeneration, and
TestAllMigrationsSQLUsesDirectDDLForKnownCLIIncompatibilities pass: a
fresh CLI-built bundle now carries participation_generation on both
issues and wisps, matching what a runtime-migrated store gets.

scripts/check-migration-hygiene.sh checks A-D pass; 0068 remains
byte-identical to origin/main.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* build: bazel-sync BUILD.bazel for migration 0070 files (be-blvmg)

make bazel-sync (gazelle) picked up two files the RED commit added that
the hand-ported BUILD.bazel hadn't listed yet: 0070's embedded .up.sql in
the go_library embedsrcs and migration_0070_add_participation_generation_test.go
in the go_test srcs.

Agent-Signature: model=claude-sonnet-5; reasoning=high

* test(feat): red — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3) (refs be-fyaja)

* feat: green — Fix: enable history before fixtures in TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit — PR #6943's §16.2b write fence leaves its flag-off neighbor legacy, so the delete mints 0 rows (stacked on 44418b3)

(refs be-fyaja)

* test(conformance): pin that a wisp reaching the issues plane begins its history (be-31mnz)

Review feedback on #6943, blocker 1: with the flag on, a promoted wisp is
never versioned. None of the five RunParticipationGeneration* cases reaches
a promotion, so nothing caught it.

Two contract cases, wired on all three legs:

- RunParticipationGenerationWispPromotionBeginsHistory: create a wisp with
  the flag on, promote it, and require a stamped participation_generation,
  one version row, and +1 on the next update.
- RunParticipationGenerationWispPersistenceMoveBeginsHistory: the same
  check for a persistence-mode update (bd update --persistent). This mints
  at a different site than promotion on every leg: the guarded update's one
  end-of-update mint on dolt and embeddeddolt, and
  MoveIssuePersistenceInTx's own on uow.

"Wisp" is in the names because this contract already says "promoted" in
R2.3's sense (ProceedsForPromotedRecord), which is a different operation.
The fixture gains CreateWisp, PromoteWisp and MovePersistent closures, and
each leg's FixtureKitIsWired checks them.

Red at this commit on the embedded leg: after the arrival there are 0 rows
and participation_generation is NULL, and after the update there are still
0 rows. A flag-on durable create goes 1 -> 2 in the same store.

* fix(issueops): mint a wisp's first issues-plane version create-shaped (be-31mnz)

Review feedback on #6943, blocker 1. A promotion, or a persistence move out
of the wisps table, inserts a new issues row. A wisp never declares
participation (FR-8), so that row's participation_generation is NULL. Both
sites minted through the update-shaped RecordVersionInTx, and the section
16.2b fence read the NULL as legacy and skipped the mint. Nothing stamps the
column later, so every later update skipped too, and a record created with
versioned history on was never versioned.

These sites now use RecordVersionForCreateInTx, which stamps the column and
mints version 1, as a create on the issues plane does:

- PromoteFromEphemeralInTx (promote.go).
- MoveIssuePersistenceInTx's move out of the wisps table.
- ExecuteUpdate's one end-of-update mint (execution.go), when its
  persistence patch did that move. This is the site bd update --persistent
  reaches on the dolt and embeddeddolt legs, and the review did not list
  it. PersistenceMoveResult.EnteredIssuesPlane tells ExecuteUpdate which
  entry point to use.

One site the review named stays update-shaped on purpose:
MoveIssuePersistenceInTx's same-plane repair, which clears a stray wisp
flag on a row already in issues. That row keeps its own
participation_generation. A row created with the flag on still mints. A
legacy row stays legacy, because a repair is an ordinary write and R2.3
says an ordinary write does not promote it. A create-shaped mint there
would stamp the legacy row and start versioning it.

* docs(schema): 0070 must precede turning versioned history on (be-31mnz)

Review note on #6943 (schema skew). The write fence reads
participation_generation on every flag-on mint of an issues-plane row. On a
store that has not applied 0070 the read fails, and the mutation fails with
it instead of skipping. The migration header now says so, and says that the
flag-off path (the default) returns before the read, so a store below 0070
writes exactly as before.

Comment only. 0070 is not on main, so check-migration-hygiene's frozen-file
check does not apply to it, and no test pins its bytes.

* test(dolt): a persistence repair keeps the row's own participation (be-31mnz)

Pins why MoveIssuePersistenceInTx's same-plane repair stays update-shaped
while promotion and a move out of the wisps table became create-shaped (the
previous commit). The repair clears a stray wisp flag on a row that already
lives in issues, so the row keeps its own participation_generation.

TestPersistenceRepairKeepsTheRowsOwnParticipation runs the repair with
versioned history on for a legacy row (created with the flag off) and a
participating row (created with it on), through both repair entry points:
MoveIssuePersistenceInTx and a guarded update with a persistence patch. They
mint at different sites. The legacy row must stay NULL and mint nothing,
since R2.3 says an ordinary write does not promote it. The participating row
must mint one version. participation_generation must not change in either
case.

It fails if the repair mint is create-shaped, which would stamp and start
versioning the legacy row.

* test(dolt): a delete's rewrite of a legacy neighbor mints nothing (be-31mnz)

Folded in from bee's R1 on #6961, note 1. Once
TestDualWriteDeleteNeighborRewriteRidesTheDeletesDoltCommit started creating
its neighbor with history on, nothing covered a delete whose neighbor was
created with history off. Under the section 16.2b fence, the expected result
for that case is different: the rewrite still runs, but it mints nothing.

TestDualWriteDeleteSkipsALegacyNeighborsRewrite creates the fixtures with
history off, turns history on, and force-deletes the cited target. It then
checks four things: the rewrite ran (ReferencesUpdated=1), the neighbor has
no issue_versions rows, its participation_generation is still NULL, and
dolt_status is clean for issue_versions, store_epoch and issues.

* test(embeddeddolt): batch-created rows are stamped with the participation generation

A batch create writes its rows first and mints each created row's first
version last, after the creation-time edges land. The mint has to stay
create-shaped for rows the batch inserted (stamping participation_generation
from the store epoch) and update-shaped for rows it only upserted over
(skipped while the row is a legacy row), so the per-row isNew has to survive
the deferral to the batch's final mint pass.

Pin that on the embedded engine:

- brand-new explicit-id rows are stamped with the store epoch and get one
  version row each, and match the same input run one row at a time with the
  batch-create fast paths off (5 rows, and 250 rows across three INSERT
  statements);
- a legacy row the batch upserts over keeps a NULL participation_generation
  and gets no version row, while the new rows beside it are stamped;
- with versioned history off, no row is stamped and no version row is written.

* ci(bazel): enroll the participation-generation tests in the embedded shards

The embedded-Dolt shard manifest has to name every top-level Test* in
internal/storage/embeddeddolt, or the exact-once coverage check fails.
The two participation-generation fence tests were written before that
check landed on main, and the batch-create stamping test is new, so none
of the three is listed yet. Add them to the 15-shard block as the
generator's incremental output: three inserted lines, no existing
assignment moves.

Also list the new batch-create test file in the embeddeddolt_test srcs,
in sorted position, so the Bazel target compiles it alongside the Go
build. Its imports are already among that target's deps.

Rebase note (maintainer): on current main the embedded storage manifest
uses a 40-shard block; the generator's incremental output placed these
three tests in shards 18, 19 and 39 and re-sorted one existing shard-12
line.

* fix(issueops): keep a created row's updated_at when its first version is minted

Minting a version is bookkeeping. It must not change a user-visible column
of the row it versions.

The create-shaped mint stamps participation_generation with an UPDATE of the
issues row. issues.updated_at is ON UPDATE CURRENT_TIMESTAMP, so an UPDATE
that changes any other column and does not name updated_at rewrites it with
the clock. Before the stamp, a row's first mint left current_revision at its
column default of 1, so the UPDATE changed nothing and the clause never
fired. The stamp moves participation_generation from NULL to the store
epoch, so the clause fires on every row whose first version is minted and
replaces the updated_at an import supplies.

Set updated_at to itself in the stamp, as the blocked-state recompute in
issueops/blocked_state.go already does, so the timestamp the row was created
with survives. Add a regression that batch-creates rows with a supplied
updated_at and requires both the stamp and the unchanged timestamp.

* test(createbatchequiv): re-record the digests the participation-generation fence moves

The fence does not re-mint rows that predate the stamp: a row whose
participation_generation is still NULL gets no new version when a batch
touches it. Main mints one, so six recorded entries move.

The issue_versions entry of five scenarios loses the versions main mints
for rows that predate the stamp: three rows in small, all eleven in
depadd, all fifteen in waitsfor, four in import458 and two in
import458-reject-stale. In depadd, eq-db1 and eq-dl are the only rows
main mints more than once, and each of those mints moves their updated_at
off the fixture date. The fence skips them, so the two rows keep the
fixture date and join the issues_updated_at entry.

No other table, list or counter digest changes, and the fast and per-row
bodies already agree in every scenario. The digests were recorded with
the embedded backend. The doc comment on GoldenDirEnv now says which
entries were re-recorded.

* test(storage): waive the participation-generation contracts on the http leg

main's #7247 (6d21d337ad, "a served http conformance leg") registered a
fourth leg, http, after this PR's original base. The PR's seven
RunParticipationGeneration* entrypoints are neither wired nor waived there,
so TestEveryLegWiresEveryRoleContract/http fails on the rebased branch.

ParticipationGenerationFixture toggles the store's own versioned-history
write flag and reads the raw participation_generation column back: store
write-fence bookkeeping (design §16.2b), not a publicops role the HTTP
client serves. Waive the seven cases on http with a named reason, the same
waiver the donnabox fork carries (09e7ce05af), and keep the gap visible
until a supported HTTP surface is designed and tested.

go test ./internal/storage/ -run TestEveryLegWiresEveryRoleContract now
passes on dolt, embeddeddolt, http and uow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): decide the fence's plane by the table that holds the row

recordVersionInTx chose the plane from the row's flags (IsWisp: Ephemeral
|| NoHistory) and then read participation_generation from the issues table
only. A wisps-TABLE row can carry neither flag, so it passed the IsWisp
return and the issues read failed with sql.ErrNoRows, aborting the whole
mutation whenever versioned history is on. Two producers:

  - bd mol squash clears an ephemeral root's flag in place
    (UpdateIssue {"wisp": false}, after the root's close, in the squash's
    one transaction); update.go and the uow repository's Update both mint
    after it unconditionally.
  - import's "wisp_plane" marker (WispPlaneOverride = &true, flags false)
    routes the row to wisps, and the batch create mints it create-shaped.

On origin/main the same paths do not fail but mint a stray issue_versions
row for a wisps-table row, an FR-8 leak (wisps are never versioned).

Fix: no issues row means nothing to version. ErrNoRows on the
participation_generation read now returns nil (no version, no error) for
update- and create-shaped calls alike. That read is the right signal and
IsActiveWispInTx is not: GetIssueInTx has already resolved the row, so
ErrNoRows on the issues read can only mean "not in issues", which is
exactly the question the fence has to answer before it stamps or advances
an issues row. It costs no extra query. IsActiveWispInTx asks the inverse
question ("is it in wisps?") with one more query per mint, and it folds
every error, including a transient one, into "not a wisp".

Tests come first, as three new conformance contracts with three new
ParticipationGenerationFixture hooks (ClearWispFlag, SquashWispRoot,
CreateUnflaggedWisp). They are wired on the dolt, embeddeddolt and uow legs
as subtests of each leg's TestParticipationGenerationFence (no new
top-level test, so the shard manifests do not change) and waived on http
with participationGenerationRawStoreWaiverReason:

  RunParticipationGenerationWispFlagClearMintsNothing      (create wisp,
      UpdateIssue {"wisp": false}: no error, 0 versions)
  RunParticipationGenerationWispRootSquashMintsNothing     (close +
      UpdateIssue {"wisp": false} in one transaction: no error, 0 versions)
  RunParticipationGenerationUnflaggedWispCreateMintsNothing (CreateIssues
      with WispPlaneOverride=&true, flags false: no error, 0 versions)

Red before the fix, green after, run both ways:
  embeddeddolt (BEADS_TEST_EMBEDDED_DOLT=1): all three FAIL with
    "versioned history: read participation_generation for pg-...: sql: no
    rows in result set"; all three PASS after.
  uow: all three FAIL with the same error; all three PASS after.
  dolt (BEADS_TEST_DOLT_SERVER=local): WispRootSquash and UnflaggedWispCreate
    FAIL with the same error; both PASS after. WispFlagClear passes both
    ways on this leg. The store-level wisp update (updateWisp) runs in a
    transaction versioned history never scopes, so it never reaches the
    mint. For the same reason the leg's SquashWispRoot hook turns the events
    journal on for its one transaction: that pins the wisp writes to the
    scoped regular transaction, as in a journal-enabled workspace, so that
    case does reach the seam.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(issueops): a create stamps participation even when the row is flagged no-history

A create-shaped mint (RecordVersionForCreateInTx) of a brand-new
ISSUES-TABLE row that carries NoHistory or Ephemeral returned at the IsWisp
check before the isCreate stamp, so participation_generation stayed NULL
although the row was created with versioned history on.

Import produces exactly that row: bd-r9uce keeps a promoted no-history
wisp's durable row on the issues plane with its stray no_history=true
(import_shared.go applyImportWispPlane: WispPlaneOverride = &false). When
bd update --persistent later repairs the flag, it is a same-plane normalize
whose mint is update-shaped by design (persistence.go: "A row created with
the flag on proceeds; a legacy row stays legacy"). The NULL column made the
fence read the row as legacy, so the repair and every later update were
fenced and the row was never versioned. origin/main versions it after the
repair (0, then 1, then 2 rows).

Fix: on a create-shaped call, a flagged row that the issues table holds
gets participation_generation stamped from store_epoch.epoch, the same
source the minting stamp uses, read in the same transaction. No version is
minted and updated_at is kept (SET ..., updated_at = updated_at), then the
call returns. Update-shaped calls on flagged rows still return before any
read. The table check from the previous commit now runs ahead of the IsWisp
branch for create-shaped calls, so a flagged wisps-table row (the common
wisp create) still mints and stamps nothing. That costs a create-shaped
call on a flagged wisp one primary-key miss on issues, and only while
versioned history is on. The store_epoch read/seed moves into
readStoreEpochInTx so both stamps share it.

Tests first:
  - New conformance contract
    RunParticipationGenerationNoHistoryIssueCreateDeclaresParticipation and
    hook CreateNoHistoryIssue. Wired on dolt, embeddeddolt and uow as a
    TestParticipationGenerationFence subtest and waived on http. With
    history on: create {NoHistory: true, WispPlaneOverride: &false} gives
    participation_generation non-NULL and 0 versions. The persistent repair
    (MovePersistent) then gives 1 version, and an update-shaped edit
    (MutateExisting, a title edit) gives 2.
  - New subtest
    TestCreateBatchParticipationGeneration/NoHistoryRowIsStampedUnversionedAndKeepsTheCallersUpdatedAt
    (embeddeddolt): a batch of such rows with a supplied updated_at comes
    out stamped, unversioned, and with updated_at unchanged.

Red before the fix, green after, run both ways:
  dolt (BEADS_TEST_DOLT_SERVER=local), embeddeddolt
  (BEADS_TEST_EMBEDDED_DOLT=1), uow: the contract FAILS on all three legs
    ("participation_generation for pg-no-history-create is NULL ...", then
    "= 0 after the persistent repair, want 1", then "went from 0 to 0 ...
    want exactly +1"). It PASSES on all three after.
  embeddeddolt subtest: FAILS before the fix ("store_epoch has no row after
    the batch, so nothing stamped the 5 rows"). With the fix minus the
    updated_at = updated_at clause it FAILS on every row's updated_at. It
    PASSES with the full fix.
No new top-level Test*, so the shard manifests do not change. The
createbatchequiv goldens are unchanged: the dolt and embeddeddolt
equivalence tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: fix comments the rebase left stale

Comment-only.

- createbatchequiv.GoldenDirEnv said "One deliberate behavior change since
  ... nothing else moved" and then listed a second re-record, the
  participation-generation fence's. It now says there are two deliberate
  changes, puts each in its own paragraph, and scopes "nothing else moved"
  to apply.json, the only golden the first change touched.
- issueops CreateIssueResult.isNew and deferredVersion still described one
  write path. After main's #7205, a row written by deferredCreates'
  multi-row INSERT reaches finishCreateIssueInTx(..., true) directly
  (create_batch_cache.go flush) and never passes through InsertIssueIfNew
  or CreateIssueInTxWithResult. isNew is InsertIssueIfNew's result on the
  per-row path and always true for a multi-row-INSERT row. deferredVersion
  is filled from both paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): pin the create/update shape split in the version guard

TestEveryVersionedEntryPointMints counts RecordVersionInTx and
RecordVersionForCreateInTx as one seam. A create-shaped site that fell back
to the update-shaped entry therefore stayed green, though the fence then
reads the new row's NULL participation_generation as legacy and the record
is never versioned. The reverse case also stayed green: an update path that
reached the create-shaped entry would stamp an existing legacy row and
promote it, which R2.3 forbids.

Add the split to the structural guard. It is two small tests over the same
journalscan fixpoint and mintVersion edge discipline, with no AST changes:

- TestCreateShapedEntryPointsMintCreateShaped: the create-shaped mint sites
  (finishCreateIssueInTx, CreateIssuesInTxWithContext,
  PromoteFromEphemeralInTx, moveIssuePersistenceInTx, executeUpdate) must
  each call RecordVersionForCreateInTx themselves. A delegate is not enough
  here, because executeUpdate runs its move with mintVersion=false and a
  batch switches finishCreateIssueInTx's mint off at run time. Every
  create-shaped entry point, the delegates and #7205's
  deferredCreates.flush included, must reach it.
- TestUpdateShapedEntryPointsNeverMintCreateShaped: every other
  versionedEntryPoint must not reach RecordVersionForCreateInTx. The
  exception is ApplyBatchInTx, which runs create and update items together
  and is listed as mixed with a reason.

Mutation-checked, each probe reverted afterwards. executeUpdate's
EnteredIssuesPlane mint, PromoteFromEphemeralInTx's mint, or the batch's
deferred create-shaped mint switched to RecordVersionInTx: each fails the
create half. update.go's end-of-update mint switched to
RecordVersionForCreateInTx: fails the update half for UpdateIssueInTx and
every entry point that writes through it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(issueops): hold the dual-shape mint sites to both halves of the split

f006c871c5 listed executeUpdate, moveIssuePersistenceInTx,
finishCreateIssueInTx and CreateIssuesInTxWithContext (and the
ExecuteUpdate/MoveIssuePersistenceInTx/create wrappers that delegate to
them) as create-shaped, so TestUpdateShapedEntryPointsNeverMintCreateShaped
skipped them. Each one also mints update-shaped, on its existing-row or
same-plane branch, which is the common path: a guarded update of an
existing row, the bd update --persistent repair of an issues-plane row's
flags, a create or batch create that resolved to a row already there.
Switching that branch to RecordVersionForCreateInTx stamped a legacy row
out of legacy (R2.3), and every guard stayed green.

Split the lists by what each function actually mints:

- versionCreateShapedMintSites keeps only the pure create-shaped site,
  PromoteFromEphemeralInTx. It must call RecordVersionForCreateInTx and
  must not call RecordVersionInTx.
- versionDualShapeMintSites names the four, each with its two branches.
  TestDualShapeMintSitesCallBothEntries requires each to call BOTH entries
  directly. A delegate does not count, for the same mintVersion=false and
  DeferVersionMint reasons as before.
- versionCreateShapedEntryPoints keeps PromoteFromEphemeralInTx and
  deferredCreates.flush, which passes isNew=true for every row.
- versionDualShapeEntryPoints replaces versionMixedShapeEntryPoints. It
  holds ApplyBatchInTx and the create, persistence-move and guarded-update
  entries that delegate to a dual-shape site, each with a reason. Each must
  reach both entries transitively.

The update half skips the create-shaped and dual-shape entry points, as
before. The update-shaped branch inside a dual-shape site is now the new
test's to pin.

Mutation-checked in a throwaway worktree. Switching the update-shaped mint
to RecordVersionForCreateInTx at execution.go:301, persistence.go:97,
create.go:260 or create.go:499 now fails
TestDualShapeMintSitesCallBothEntries for that site. Before this commit,
the whole issueops package stayed green under each of these switches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(httpclient): run the participation-generation fence over the served wire

edd3b72a1a waived all eleven ParticipationGeneration contracts on the http
leg, saying the client had "no write flag to toggle and no raw column to
read". That reason was wrong. served_dualwrite_history_test.go already
flips the reference store's VersionedHistoryConfigurer and reads its raw
columns, with the client driving the mutation. The waiver also hid the
one harness that runs the fence behind the update a connected workspace
actually sends. Every store leg's MutateExisting is store.UpdateIssue,
but the served Lifecycle.Update reaches the guarded update,
ExecuteUpdate.

Add served_participation_generation_test.go on the same pattern:

- Bound to the client: Mutate is Lifecycle.Create, MutateExisting is
  Lifecycle.Update, MovePersistent is the persistence-mode
  Lifecycle.Update that bd update --persistent sends, and PromoteWisp is
  issueops.Promote over the client's Reader/Lifecycle/Commenter, the
  entry bd promote dials over http.
- Bound to the reference store, as before: SetFlag, the CreateWisp seed,
  and the raw reads.

TestServedParticipationGenerationFence runs seven cases:
CreateWithFlagOnStampsAValue, CreateWithFlagOffLeavesLegacyNull,
SkipsUpdateOnLegacyRecord, ProceedsForPromotedRecord,
FenceIgnoresSchemaSkewHatch, WispPromotionBeginsHistory and
WispPersistenceMoveBeginsHistory. All seven pass, and no case turned up a
server-side fence bug.

Four cases stay waived, now with a reason that names the actual wire gap
(participationGenerationWireGapWaiverReason):

- WispFlagClearMintsNothing: the in-place {"wisp": false} is an untyped
  UpdateIssue, and IssuePatch.Persistence moves the row instead.
- WispRootSquashMintsNothing: it needs RunInTransaction. That and
  UpdateIssue are both on the client's unsupported allowlist (D8).
- UnflaggedWispCreateMintsNothing and
  NoHistoryIssueCreateDeclaresParticipation: both need WispPlaneOverride,
  which every create shape on the wire drops
  (roleIgnoredCreateIssueMembers).

The new file is listed in httpclient_test's srcs. The served lane deals
unpinned tests round-robin, so no shard manifest names it.

Mutation-checked in a throwaway worktree. With execution.go:301 (the
guarded update's update-shaped mint) switched to
RecordVersionForCreateInTx, SkipsUpdateOnLegacyRecord and
FenceIgnoresSchemaSkewHatch fail here ("version row count for
hpg-legacy-skip went from 0 to 1"). The embeddeddolt and uow
Participation|DualWrite|Version suites stay green under the same probe.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): an update never stamps a legacy no-history row

7c00d65fa6 made the seam write participation_generation on a flagged
(NoHistory or Ephemeral) issues-table row when the call is create-shaped.
Only the early `if IsWisp(issue) && !isCreate { return nil }` keeps an
update-shaped call out of that branch, and nothing pinned it. The new
contract and its subtest covered only the positive, create-shaped side.
With the early return dropped, an ordinary update stamped a legacy
no-history row out of legacy (R2.3), and every suite stayed green.

Add RunParticipationGenerationLegacyNoHistoryIssueStaysLegacy beside
NoHistoryIssueCreateDeclaresParticipation as its negative twin:

1. Create the no-history issues-table row with the flag OFF.
2. Turn the flag on and run MutateExisting. participation_generation must
   stay NULL, with 0 versions and current_revision unchanged.
3. Run MovePersistent, the bd update --persistent repair that clears the
   flag in place. The repair is update-shaped too, so the same three
   things must still hold.

Wired on all four legs: dolt, embeddeddolt, uow and http. On http the
case exercises the client's Lifecycle.Update and persistence update.
Their precondition, the flag-off no-history create, is seeded through
the reference store, because no client create can land that row in the
issues table. That gap is also why the case whose SUBJECT is that create
stays waived on http.

Mutation-checked in a throwaway worktree, with the early return made
`if false && IsWisp(issue) && !isCreate`. The new case fails on all four
legs and is the only case that fails:
"participation_generation for pg-no-history-legacy = 1 after an
update-shaped mutation with the flag on, want NULL" (dolt server leg,
embeddeddolt, uow and http). At tip it passes on all four.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(issueops): say which table and which mint the fence comments mean

Comment and doc corrections. No code changes.

- RecordVersionForCreateInTx's doc comment covered a flagged issues-table
  row but not a wisps-table row. A row the wisps table holds mints and
  stamps nothing, whatever its flags say: there is no issues row to stamp
  (FR-8). That holds for an ordinary wisp create and for import's
  unflagged "wisp_plane" row.
- The no-history stamp branch said the bd update --persistent repair mints
  update-shaped "(MoveIssuePersistenceInTx)". That is true only on the
  unit-of-work path. On dolt and embeddeddolt the repair goes through the
  guarded update: executeUpdate runs moveIssuePersistenceInTx with
  mintVersion=false and mints once at its own end. The fixture's
  MovePersistent doc already says this.
- VERSIONED-BEADS-WRITE-PATHS.md row 13 still credited the whole wisp
  exclusion to `RecordVersionInTx returns on IsWisp` and called every case
  a NO-OP. After ec52d29030, the table holding the row decides its plane:
  only an update-shaped call on a flagged wisp returns on IsWisp. Every
  other wisps-table row, including an unflagged one, is excluded by the
  issues-table participation_generation read, which finds no row. After
  7c00d65fa6, a create-shaped call on a flagged issues-table row is
  stamped but not versioned, so the row says that too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(conformance): the wisps-table cases check the row is still a wisp

runWispTableRowMintsNothing asserted 0 versions but never checked its own
precondition, that the wisps table still holds the row. Suppose a step
moved the row onto the issues plane through an update-shaped mint. That
mint would hit the new row's NULL participation_generation and be fenced,
so the count stayed 0 and the case passed without saying anything about
a wisps-table row.

- Add an InWispsTable hook to ParticipationGenerationFixture and bind it
  on dolt, embeddeddolt and uow as a raw wisps-table count. The
  wisps-table body now Fatals if the row is not in the wisps table after
  its steps. Each leg's KitIsWired test checks the hook. On http it
  stays nil with the other wisps-table hooks, whose cases are waived
  there.
- The dolt ClearWispFlag hook now bites. A bare store.UpdateIssue on a
  wisp goes through updateWisp, whose transaction versioned history never
  scopes. So WispFlagClearMintsNothing passed on dolt with or without the
  ec52d29030 fix, as that commit's message admits. The hook now runs the
  lone flag clear in a RunInTransaction with the events journal on: the
  journal-enabled shape the SquashWispRoot hook already uses.
- Both hooks go through a new withEventsJournal helper. It restores the
  store's previous journal setting rather than unconditionally setting it
  to false afterwards, which assumed the store started with the journal
  off.

Mutation-checked in throwaway worktrees:
- With version_history.go at ec52d29030^, dolt's WispFlagClearMintsNothing
  now fails ("read participation_generation for pg-flag-clear: sql: no
  rows in result set"). It used to pass there.
- Embedded ClearWispFlag rerouted as a flag-off persistence move followed
  by a flag-on title update: the case fails on the new precondition ("is
  not in the wisps table after flag-clear"). With the precondition
  disabled, the same probe passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(storage): review fixups for the participation_generation fence

The review of d17b716115 approved the PR with no required changes and
raised four minors and three nits. Each one below says what changed and
why, or why it did not.

Minor 1, the upsert create's new-or-existing choice was untested. The
unit-of-work create that is not CreateOnly (bd batch, bd todo, tracker
sync and mol port) inserts through an upsert, and db.Insert's existed
probe picks the mint's shape; forcing it either way shipped green. The
new uow TestParticipationGenerationUpsertCreate pins all three outcomes:
a new row is stamped and begins its history, a legacy row the upsert
resolves to stays legacy with no version, and a participating row mints
exactly one more. No served-route case is added because every served
create is CreateOnly (uow createParams), so the http wire cannot reach
this branch. The cmd/bd proxied-server batch and todo creates call this
same use-case method.

Minor 2, an update-shaped mint still lets issues.updated_at move
through ON UPDATE CURRENT_TIMESTAMP. Not changed here. The maintainer
listed it on the PR as follow-up (2), and the approving review of
d17b716115 accepted carrying it as a follow-up. It does not change
the fence's contract as pinned.

Minor 3, the depadd and waitsfor goldens had been re-recorded to zero
issue_versions rows. Their seeds run with history off, so every seeded
row was legacy and the harness stopped checking the dependency-add,
close and reopen mints. Both scenarios now stamp their seeded
issues-table rows after the seed (stampParticipation; wisps stay
unstamped). Their goldens are back to main's pre-fence files byte for
byte: depadd mints 11 version rows and waitsfor 15. The fenced legacy
outcome stays pinned by small, import458 and import458-reject-stale. A
run scenario that mints no version row now fails rather than recording
an empty digest.

Minor 4, PromoteFromEphemeralInTx always minted create-shaped. When the
issues plane already held the id (the dual presence GH#4455 records),
its upsert resolved to that row and the create-shaped mint stamped a
legacy record (R2.3). It now keeps InsertIssueIfNew's isNew and mints
update-shaped for an existing row, as finishCreateIssueInTx does. The
version guard lists it as a dual-shape mint site and entry point, and
PromoteOverDualPresenceKeepsLegacy pins the behavior at run time.

Nit 1: with versioned history off, db.Insert skips the existed probe,
since both mint entries return before reading anything. A create costs
the round trips it did before the fence.

Nit 2: the MINT COST comment now counts the fence read (about 6 to 7
round trips) and no longer says the snapshot reads current_revision;
the fence read can select it for #6379's ordinal seed. Moving that read
now would only add a column nothing consumes yet.

Nit 3: the http leg's participation-generation wire-gap waiver opens as
PENDING and names the wire mutation that would retire each case.

Validation, in a scrubbed env with -tags=gms_pure_go on embedded and
server Dolt (all pass, none skipped):
- uow: TestParticipationGenerationFence, its FixtureKitIsWired check
  and TestParticipationGenerationUpsertCreate with its 4 subtests.
- embeddeddolt: the create-batch equivalence parts (small/depadd,
  apply/waitsfor), the parts-coverage guard, the large scenarios, and
  the fence and kit.
- dolt: TestCreateBatchFastPathsMatchPerRow_Dolt, the fence,
  FenceIgnoresSchemaSkewHatch and the kit.
- httpclient: TestServedParticipationGenerationFence.
- TestEveryLegWiresEveryRoleContract, the issueops structural guards,
  and the issueops, storage and domain/db packages.

Eleven mutants, graded in 13 runs, each fail the test meant to catch
them:
- upsert always create-shaped or always update-shaped;
- the probe guard inverted;
- promote always create-shaped or always update-shaped;
- the seed left unstamped;
- the dependency-add, close or reopen mint dropped;
- two on-disk single-entry promotes.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): a wisp upsert must not stamp a legacy issues twin

domain/db's Insert chose the mint shape of its upsert (the create that
is not CreateOnly) from an existence probe of the target table. For a
wisps-table insert that probe asks the wrong plane: the seam snapshots
the issues table first (GetIssueInTx), so when the issues plane already
held the id as a legacy row, the wisp create took the create-shaped
mint and stamped and versioned that untouched twin. An ordinary wisp
write promoted a legacy record, which R2.3 forbids. bd mol port and the
proxied mol and wisp writers call CreateWisp this way. It is latent
until versioned history is enabled (#6661).

A wisps-table insert never creates an issues row, so it now mints
update-shaped, without the probe. The fence leaves a legacy twin alone,
a wisp-only id still mints nothing through either entry (FR-8), and a
participating twin mints the one version main already minted for it.
Insert's comment now records that last case: the dual presence
predates the fence, and this upsert does not refuse it the way
issueops' create path does.

TestParticipationGenerationUpsertCreate gains
WispOverLegacyIssuesRowKeepsItLegacy, which fails against the previous
dispatch (twin stamped, one version row). Also from review: the version
shape guard names the dual-shape site its parse cannot see, and the
row-lock exemption comment counts all three of the seam's UPDATE issues
statements.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
2026-10-18 13:10:57 +00:00

773 lines
37 KiB
Go

package issueops
// Build-time completeness guard for the row_lock optimistic-concurrency token,
// absorbed from gastownhall/beads#4682's revision guard and retargeted at the
// row_lock column current main actually ships (see the freshRowLock invariant
// in lease.go and types.Issue.RowVersion). The PR's substring stamp check
// (strings.Contains(body, "revision")) was false-negative-prone — any comment
// or identifier mentioning the column name counted as a stamp — so this port
// tightens it to the actual mint forms: a write's own literal must show a
// genuine row_lock rewrite (literalStampsRowLock), and only the routed
// funnels whose SET clause is itself assembled elsewhere (a %s placeholder)
// fall back to an AST-based scan of the enclosing function for a
// freshRowLock()/FreshRowLock()/RowLockClause() call (containsStampCall) —
// never a text/regex scan, so a stamp mentioned only in a comment does not
// count (comments are never part of the expression tree).
import (
"go/ast"
"go/parser"
"go/token"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
)
// issueTableWriteRe matches the start of an UPDATE or INSERT to an issue-bearing
// table: literal `issues`/`wisps` (optionally backtick-quoted — MySQL identifier
// quoting, used by versioncontrolops's manual conflict-resolution writes,
// e.g. UPDATE `issues` SET ...), or a `%s`-templated table (the routed
// issues/wisps funnels build the table name with WispTableRouting/pickIssueTable
// and interpolate it as %s, itself sometimes backtick-quoted — conflicts.go's
// UPDATE `%s` SET ...). Auxiliary tables reached through the same %s
// templating are filtered out by their distinguishing columns (auxOrExemptMarkers).
//
// This alone over-matches: fmt.Errorf("db: Update %s: %w", id, err) and
// similar error-format strings echo the verb+%s shape without being SQL at
// all. sqlWriteKeywordRe below requires a real clause keyword before a match
// is counted.
//
// The trailing `\b` needs no backtick counterpart: it is a word/non-word
// boundary check anchored right after the bare identifier ("issues", "wisps",
// or the "s" of "%s"), which already holds whether the next real character is
// a backtick, whitespace, or anything else non-word — adding an optional
// backtick there would make it FALSE when one is actually present (backtick
// and the whitespace after it are both non-word, so no boundary exists
// between them).
var issueTableWriteRe = regexp.MustCompile("(?i)\\b(?:UPDATE|INSERT\\s+INTO)\\s+`?(?:issues|wisps|%s)\\b")
// sqlWriteKeywordRe requires a genuine SQL clause keyword in the same
// enclosing literal as an issueTableWriteRe match, so an error-format string
// that merely names the verb for a message — "db: Update %s: %w", "db:
// insert into %s: %w" (internal/storage/domain/db/issue.go) — is not counted
// as a table write. Every real UPDATE has a SET; every real INSERT has
// VALUES; the one aux-table copy that uses neither verb's usual keyword
// (persistence.go's `INSERT INTO %s (%s) SELECT %s FROM %s ...`) has SELECT.
var sqlWriteKeywordRe = regexp.MustCompile(`(?i)\b(?:SET|VALUES|SELECT)\b`)
// auxOrExemptMarkers identify a matched write that legitimately does NOT stamp
// row_lock. The exemption set is main's freshRowLock invariant (lease.go):
// row_lock guards status/assignee/started_at against the reclaim/close races,
// and paths touching only orthogonal cells are safe to merge with a reclaim.
//
// - is_blocked: the denormalized is_blocked recompute deliberately preserves
// updated_at (blocked_state.go x4, dependencies.go, domain/db/dependency.go)
// and must not remint row_lock either — an aux-marker refresh that bumped
// the token would clobber a concurrent whole-row CAS (ExpectedVersion) for
// no content change. Analysis absorbed from gastownhall/beads#4682.
// (The PR's other exemption, the lease heartbeat, is moot on main: since
// bd-lrgn1 a heartbeat refresh lives entirely in the ephemeral leases
// table and never touches the issues row, and a re-arm remints row_lock on
// purpose; see the freshRowLock invariant.)
// - "compaction_level = ": compaction apply/restore rewrites bookkeeping and
// compacted body text only — cells the reclaim/close races don't care
// about, exempted by name in the freshRowLock invariant. The assignment
// form (not the bare column name) keeps the create INSERT, whose column
// list also names compaction_level, checkable.
// - "SET id = ?": rename (updateIssueIDInTx/updateWispIDInTx) is the only
// writer of the primary key, exempted by name in the freshRowLock
// invariant.
// - "SELECT %s FROM": the persistence move's fully-templated aux-table copy
// (INSERT INTO %s (%s) SELECT %s FROM %s ...) — it copies snapshot/event
// side tables verbatim; the issues/wisps row itself is re-inserted through
// InsertIssueStrictInTx, which stamps.
// - the remainder are columns unique to AUXILIARY tables reached through the
// same %s templating (events / dependencies / child_counters / snapshots /
// comments). Issue rows key on `id` and never carry these, so their
// presence proves a non-issue table.
//
// If a future write trips this guard: stamp row_lock with freshRowLock() (or
// RowLockClause()) when it is a real issues/wisps content write; add the
// distinguishing column/marker here — with the WHY — when it is a new
// auxiliary-table or deliberately-orthogonal write.
//
// A marker match is refused outright, regardless of which marker fired, when
// the write's own SET clause assigns status, assignee, or started_at — the
// three columns the freshRowLock invariant exists to guard (lease.go). Those
// markers are legitimate only because is_blocked/compaction/rename writes
// never touch that trio; a hypothetical future write that assigned e.g.
// status in the same statement as an is_blocked-marked WHERE predicate must
// still stamp. See setClauseAssignsLifecycleField.
var auxOrExemptMarkers = []string{
"is_blocked", // is_blocked recompute (exempt by design)
"compaction_level = ", // compaction bookkeeping/restore (exempt by design)
"SET id = ?", // rename: primary-key rewrite (exempt by design)
"SELECT %s FROM", // persistence move's verbatim aux-table copy
"issue_id", // events / dependencies / comments / snapshots FK
"depends_on", // dependency edge + retarget writes
"parent_id", // child_counters
"last_child", // child_counters
"event_type", // events / wisp_events
}
// funcNameExemptions are functions whose issues/wisps write is deliberately
// exempt from the row_lock stamp requirement, keyed by the enclosing
// function's identity rather than by a substring marker in the SQL text.
//
// This is a DIFFERENT mechanism from auxOrExemptMarkers, and deliberately
// bypasses setClauseAssignsLifecycleField's refusal: that refusal exists
// because a text marker can be matched incidentally (e.g. in a WHERE clause
// unrelated to the exemption's real reason), so a marker match must not wave
// through a write whose SET clause happens to also assign a lifecycle
// column. A function-name match carries no such risk — it names one specific,
// reviewed function, not a string that could recur by coincidence — so it is
// allowed to exempt a write even when that write's SET clause is fully
// %s-templated and could, at runtime, assign status/assignee/started_at.
//
// Add an entry here only for a write that is genuinely correct without a
// fresh stamp — never as a shortcut past a write that should stamp but
// doesn't.
var funcNameExemptions = map[string]string{
// resolveOneConflictRow's `theirs` branch (conflicts.go) is whole-row
// adoption for `bd conflicts resolve --theirs`: every their_* column,
// including their row_lock, is copied verbatim over ours. The resulting
// row IS their row, byte for byte, so their row_lock already vouches for
// exactly this content — the same "settled row equals a parent, that
// parent's token still vouches for it" argument automerge.go's
// mergeIssuesConflictRow makes for the no-op (nothing written) case.
// Reminting here would be pointless content-wise and would let a stale
// ExpectedVersion CAS reject a row it should still recognize.
"resolveOneConflictRow": "whole-row `theirs` adoption: the adopted row_lock already vouches for the (identical) adopted content",
// recordVersionInTx (version_history.go) is the private seam both public
// entry points — RecordVersionInTx (update-shaped) and
// RecordVersionForCreateInTx (create-shaped, design §16.2b's write fence,
// be-h89oq) — delegate to; the three UPDATE issues statements this guard
// would otherwise flag live here, not in either public wrapper, so the
// exemption is keyed to this name. It runs immediately after the mutation
// it is snapshotting — create.go/update.go's own writes, or domain/db's
// Insert/Update — and those callers have already minted a fresh row_lock
// in that same INSERT/UPDATE, in the same transaction. Every UPDATE
// issues statement recordVersionInTx issues writes only bookkeeping
// columns: the update-shaped and create-shaped UPDATE issues SET
// current_revision = ... statements advance the denormalized pointer to
// the issue_versions row just inserted (the create-shaped one also stamps
// participation_generation in the same statement), and the no-history
// stamp sets participation_generation alone on a create-shaped row it does
// not version. None touches row_lock at all (not even to carry the
// existing value forward) because reminting here would bump the token a
// second time within one transaction, invalidating the RowVersion/CAS
// value the primary mutation's own caller already captured and could
// cause a legitimate subsequent CAS check to fail as a spurious lost
// update.
"recordVersionInTx": "runs after the primary mutation already minted a fresh row_lock in the same transaction; reminting here would double-bump the token and invalidate the RowVersion/CAS value already returned to that mutation's caller — covers all three of its UPDATE issues statements (the update-shaped and create-shaped current_revision advances and the no-history participation_generation stamp), since both public entry points delegate to this one private seam",
}
// TestAllIssueRowWritesStampRowLock is the load-bearing completeness guard for
// the freshRowLock invariant: it proves, at build time, that EVERY issues/wisps
// row write across both whole-row write stacks (issueops and the proxied
// internal/storage/domain/db) either stamps a fresh row_lock or matches a
// documented exemption. A forgotten write path is a test failure, catching an
// accidental reintroduction of the zombie-merge bug before it ships.
//
// This does not mean every documented exemption is itself free of a stale-CAS
// hole in RowVersion — two of them knowingly are one: compaction rewrites
// bookkeeping and compacted body text while deliberately preserving row_lock
// (documented at storage.go:319-325, the same "lifecycle/ownership only"
// contract this guard enforces), and a rename (updateIssueIDInTx) changes the
// primary key without reminting the old row's token — but that is safe by
// construction: the CAS predicate is keyed on id, so a stale ExpectedVersion
// read against the pre-rename id matches no row at all rather than winning
// against renamed content.
func TestAllIssueRowWritesStampRowLock(t *testing.T) {
// versioncontrolops is included alongside issueops and the proxied
// domain/db: it writes issues/wisps rows too, both from automerge's
// field-level conflict resolution (resolveIssuesFieldMerge) and from
// conflicts.go's manual `bd conflicts resolve` path — both of which this
// branch's own resolveIssuesFieldMerge/freshRowLockDistinctFrom pairing
// can assign status/assignee. Absorbed from a dual-vendor review of
// d10544c96, which found the original two-directory scan missed this
// package entirely.
dirs := []string{".", filepath.Join("..", "domain", "db"), filepath.Join("..", "versioncontrolops")}
checked := 0
for _, dir := range dirs {
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("read dir %s: %v", dir, err)
}
for _, e := range entries {
name := e.Name()
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
path := filepath.Join(dir, name)
src, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
n, violations := scanIssueWriteRowLockStamps(t, path, src)
checked += n
for _, v := range violations {
t.Error(v)
}
}
}
if checked == 0 {
t.Fatal("guard verified zero issue-table writes — the scan regex or directory set is wrong")
}
t.Logf("verified %d issues/wisps row writes across both stacks stamp row_lock", checked)
}
// scanIssueWriteRowLockStamps parses one Go source file and returns the number
// of issue-table writes it verified plus a violation message for each such write
// whose enclosing function does not mint a fresh row_lock. Split out so the
// guard's teeth can be tested against synthetic sources (TestRowLockGuardHasTeeth).
func scanIssueWriteRowLockStamps(t *testing.T, path string, src []byte) (checked int, violations []string) {
t.Helper()
fset := token.NewFileSet()
file, err := parser.ParseFile(fset, path, src, 0)
if err != nil {
t.Fatalf("parse %s: %v", path, err)
}
ast.Inspect(file, func(n ast.Node) bool {
fn, ok := n.(*ast.FuncDecl)
if !ok || fn.Body == nil {
return true
}
start := fset.Position(fn.Body.Pos()).Offset
end := fset.Position(fn.Body.End()).Offset
body := string(src[start:end])
// Computed lazily, and only once per function: the AST-based
// whole-function scan is needed only for the funnel fallback below,
// and most functions never reach it.
checkedStampCall, hasStampCall := false, false
funcStampsRowLock := func() bool {
if !checkedStampCall {
// Function-scope first: the original, tighter guarantee for
// a funnel whose mint call and %s-templated SET clause live
// in the SAME function (issueops' Claim/Update).
hasStampCall = containsStampCall(fn.Body) ||
// versioncontrolops' automerge plan/apply split puts the
// mint call in a SIBLING function of the same file
// (mergeIssuesConflictRow builds the plan resolveIssuesFieldMerge
// later applies generically), so a per-function scan can
// never see it; recognition for those two call forms is
// intentionally file-scoped instead. See
// crossFuncStampCallNames and containsSetColumnRowLockStamp.
containsCrossFuncStamp(file)
checkedStampCall = true
}
return hasStampCall
}
for _, loc := range issueTableWriteRe.FindAllStringIndex(body, -1) {
matchText := body[loc[0]:loc[1]]
isInsert := strings.HasPrefix(strings.ToUpper(strings.TrimSpace(matchText)), "INSERT")
// Classify by the tight enclosing SQL literal so an adjacent
// statement's columns can't bleed into this write's markers.
stmt := enclosingSQLLiteral(body, loc[0])
if !sqlWriteKeywordRe.MatchString(stmt) {
// Matched the UPDATE/INSERT-INTO verb shape but names no SET/
// VALUES/SELECT: an error-format string, not SQL.
continue
}
if why, exempt := funcNameExemptions[fn.Name.Name]; exempt {
_ = why // documented at the definition; nothing to check further
continue
}
if hasAnyMarker(stmt, auxOrExemptMarkers) && !setClauseAssignsLifecycleField(stmt) {
continue // auxiliary table or documented exemption: no stamp
}
checked++
stamped := literalStampsRowLock(stmt, isInsert)
if !stamped && !isInsert && setClauseHasPercentSPlaceholder(stmt) {
// The write's own literal doesn't show the mint (its SET
// clause is itself a %s placeholder assembled elsewhere —
// update.go / domain-db's Update and Claim funnels, or
// versioncontrolops' resolveIssuesFieldMerge), so fall back
// to the funnel-fallback check above.
stamped = funcStampsRowLock()
}
if !stamped {
violations = append(violations, path+": "+funcDisplayName(fn)+
" performs an issues/wisps row write that does not stamp row_lock:\n\t"+
firstSQLLine(stmt)+
"\nEvery issues/wisps content write must mint a fresh token via freshRowLock()/RowLockClause() (see the freshRowLock invariant in lease.go), or carry a documented exemption marker.")
}
}
return true
})
return checked, violations
}
// setKeywordRe and whereKeywordRe bound an UPDATE statement's SET clause
// (setClause below), scoping the finding-2 lifecycle-field refusal and the
// finding-3 %s-placeholder funnel signal to the columns actually being
// assigned — not the WHERE predicate, which can legitimately reuse the same
// column names for an unrelated condition (blocked_state.go's mark/unmark
// templates test `i.status <> 'closed'` in WHERE while only ever assigning
// is_blocked/updated_at in SET).
var (
setKeywordRe = regexp.MustCompile(`(?i)\bSET\b`)
whereKeywordRe = regexp.MustCompile(`(?i)\bWHERE\b`)
)
// setClause extracts stmt's SET-to-WHERE fragment, or "" for a statement with
// no SET keyword (an INSERT's column list, or a malformed match).
func setClause(stmt string) string {
idx := setKeywordRe.FindStringIndex(stmt)
if idx == nil {
return ""
}
clause := stmt[idx[1]:]
if w := whereKeywordRe.FindStringIndex(clause); w != nil {
clause = clause[:w[0]]
}
return clause
}
// lifecycleFieldAssignRe matches an assignment to one of the three columns
// the freshRowLock invariant (lease.go) exists to guard: status, assignee,
// started_at (see storage.go's ExpectedVersion doc). A write assigning any of
// these must stamp row_lock regardless of an aux/exempt marker matched
// elsewhere in the statement.
var lifecycleFieldAssignRe = regexp.MustCompile(`(?i)\b(?:status|assignee|started_at)\s*=`)
// setClauseAssignsLifecycleField reports whether stmt's SET clause (not its
// WHERE predicate) assigns status, assignee, or started_at. Used to refuse an
// aux/exempt marker match outright when the write is really a lifecycle
// write that merely happens to share a marker word — e.g. a hypothetical
// `UPDATE issues SET status = ?, assignee = NULL WHERE is_blocked = 0` must
// not be waved through by the is_blocked marker just because that word
// appears in WHERE.
func setClauseAssignsLifecycleField(stmt string) bool {
return lifecycleFieldAssignRe.MatchString(setClause(stmt))
}
// setClauseHasPercentSPlaceholder reports whether stmt's SET clause itself
// contains a %s token — the signal that the clause (or part of it, e.g.
// Claim's trailing row_lock fragment) is assembled elsewhere and interpolated
// in, rather than written out in this literal. Used to gate the finding-3
// function-wide fallback to only the funnel shape it is meant for.
func setClauseHasPercentSPlaceholder(stmt string) bool {
return strings.Contains(setClause(stmt), "%s")
}
// rowLockAssignRe matches a genuine row_lock rewrite in an UPDATE's SET
// clause: a bind-parameter assignment ("row_lock = ?", close/reopen/
// unclaim/reclaim/update's generic path/persistence's storage-class move) or
// the upsert refresh ("row_lock = VALUES(row_lock)", insertIssueRow's ON
// DUPLICATE KEY branch). It deliberately does NOT match the token-preserving
// self-assignment "row_lock = row_lock" — TestRowLockGuardHasTeeth's
// mention-only case: that shape names the column without minting a fresh
// value, exactly the false-positive shape this guard exists to catch.
var rowLockAssignRe = regexp.MustCompile(`(?i)\brow_lock\s*=\s*(?:\?|VALUES\s*\(\s*row_lock\s*\))`)
// insertRowLockColumnRe matches row_lock named in an INSERT's column list
// (helpers.go's classic insertIssueIntoTable, domain/db's insertIssueRow):
// every create path pairs it with FreshRowLock() in the same ExecContext
// call's argument list.
var insertRowLockColumnRe = regexp.MustCompile(`(?i)\brow_lock\b`)
// literalStampsRowLock reports whether a write's OWN SQL literal demonstrates
// a fresh row_lock rewrite, without needing to look at the surrounding Go
// code at all. insert selects which literal shape to check: an INSERT names
// row_lock in its column list, an UPDATE rewrites it in its SET clause — the
// same text ("row_lock") means something different in each.
func literalStampsRowLock(stmt string, insert bool) bool {
if insert {
return insertRowLockColumnRe.MatchString(stmt)
}
return rowLockAssignRe.MatchString(setClause(stmt))
}
// stampCallNames are the forms that actually mint a fresh token: a direct
// freshRowLock()/FreshRowLock() call (close, reopen, unclaim, reclaim,
// insert) or RowLockClause() (claim and the proxied domain/db paths, which
// interpolate the returned "row_lock = ?" fragment into their SET clause).
var stampCallNames = map[string]bool{
"freshRowLock": true,
"FreshRowLock": true,
"RowLockClause": true,
}
// containsStampCall walks node looking for a CallExpr naming one of
// stampCallNames. It is AST-based, not a text/regex scan over source bytes,
// so a stamp name that appears only in a comment — or in a string literal,
// or as part of an unrelated identifier — does not count: comments are never
// part of an expression tree, and the match below requires the name to be
// the callee of an actual call, not merely mentioned. Used only for the
// finding-3 fallback (a %s-templated SET clause whose mint call lives
// elsewhere in the same function), never as the primary per-write check.
func containsStampCall(node ast.Node) bool {
found := false
ast.Inspect(node, func(n ast.Node) bool {
if found {
return false
}
ce, ok := n.(*ast.CallExpr)
if !ok {
return true
}
if name, ok := stampCallName(ce.Fun); ok && stampCallNames[name] {
found = true
return false
}
return true
})
return found
}
// stampCallName extracts the callee name from a plain call (freshRowLock())
// or a package/method-qualified call (issueops.RowLockClause()).
func stampCallName(expr ast.Expr) (string, bool) {
switch f := expr.(type) {
case *ast.Ident:
return f.Name, true
case *ast.SelectorExpr:
return f.Sel.Name, true
}
return "", false
}
// crossFuncStampCallNames are mint forms whose CallExpr is deliberately NOT
// required to live in the same function as the write it stamps.
// versioncontrolops' automerge conflict resolution splits row_lock's
// plan-building step from its generic plan-apply step: mergeIssuesConflictRow
// (automerge.go) computes the fresh token and stores it into the write-back
// plan with freshRowLockDistinctFrom, and resolveIssuesFieldMerge (also
// automerge.go) later writes whatever columns the plan names — including
// row_lock, whenever present — without ever mentioning the column by name in
// its own source. containsStampCall's function-scoped search can never see
// that pairing, so recognition for this name is intentionally file-scoped
// (see containsCrossFuncStamp) rather than added to stampCallNames, which
// stays function-scoped so the single-function funnels in issueops keep the
// tighter guarantee finding 3 of the earlier review restored.
var crossFuncStampCallNames = map[string]bool{
"freshRowLockDistinctFrom": true,
}
// containsCrossFuncStamp reports whether file, taken as a whole (every
// function, not just one), contains evidence that SOME function in it mints
// row_lock in a form crossFuncStampCallNames or containsSetColumnRowLockStamp
// recognizes. Called only as a second-level fallback after containsStampCall
// on the single enclosing function has already failed, so it never weakens
// the primary per-write check or the same-function funnel guarantee — it
// only recognizes the specific plan/apply split described above.
func containsCrossFuncStamp(file *ast.File) bool {
found := false
ast.Inspect(file, func(n ast.Node) bool {
if found {
return false
}
ce, ok := n.(*ast.CallExpr)
if !ok {
return true
}
if name, ok := stampCallName(ce.Fun); ok && crossFuncStampCallNames[name] {
found = true
return false
}
return true
})
return found || containsSetColumnRowLockStamp(file)
}
// containsSetColumnRowLockStamp reports whether node contains a call of the
// shape `<x>.setColumn("row_lock", ...)` — issuesRowMerge.setColumn
// (automerge.go), the OTHER form mergeIssuesConflictRow uses to stamp
// row_lock into the write-back plan resolveIssuesFieldMerge later applies.
// AST-based, like containsStampCall: the literal "row_lock" must be the
// call's own first argument, not merely mentioned nearby.
func containsSetColumnRowLockStamp(node ast.Node) bool {
found := false
ast.Inspect(node, func(n ast.Node) bool {
if found {
return false
}
ce, ok := n.(*ast.CallExpr)
if !ok {
return true
}
name, ok := stampCallName(ce.Fun)
if !ok || name != "setColumn" || len(ce.Args) == 0 {
return true
}
lit, ok := ce.Args[0].(*ast.BasicLit)
if ok && lit.Kind == token.STRING && lit.Value == `"row_lock"` {
found = true
return false
}
return true
})
return found
}
// TestRowLockGuardHasTeeth proves the guard actually flags an unstamped
// issues/wisps write and passes a correctly-stamped one — so a green
// TestAllIssueRowWritesStampRowLock means something. The cases below also
// cover the false positives closed by the adversarial review of ea1256462:
// a noise match on an error-format string (finding 1), a lifecycle write
// hiding behind an aux marker matched only in WHERE (finding 2), a direct
// write riding an unrelated sibling write's stamp in the same function
// (finding 3), and a stamp name that appears only in a comment (finding 4).
func TestRowLockGuardHasTeeth(t *testing.T) {
stamped := []byte(`package p
func w(tx T, id, s string) {
tx.ExecContext(ctx, "UPDATE issues SET status = ?, row_lock = ? WHERE id = ?", s, freshRowLock(), id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "stamped.go", stamped); len(v) != 0 || n != 1 {
t.Errorf("stamped write: got checked=%d violations=%v; want checked=1 violations=none", n, v)
}
unstamped := []byte(`package p
func w(tx T, id, s string) {
tx.ExecContext(ctx, "UPDATE issues SET status = ? WHERE id = ?", s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "unstamped.go", unstamped); n != 1 || len(v) != 1 {
t.Errorf("unstamped write: got checked=%d violations=%d; want checked=1 violations=1", n, len(v))
}
// The tightened matcher: a body that merely MENTIONS row_lock — here a
// token-preserving self-assignment, but a comment would do — without
// minting a fresh one must still be flagged. The PR's substring check
// (strings.Contains(body, "revision")) passed exactly this shape.
mentionOnly := []byte(`package p
func w(tx T, id, s string) {
// row_lock intentionally not reminted (WRONG: status writes must mint).
tx.ExecContext(ctx, "UPDATE issues SET status = ?, row_lock = row_lock WHERE id = ?", s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "mention.go", mentionOnly); n != 1 || len(v) != 1 {
t.Errorf("mention-only write: got checked=%d violations=%d; want checked=1 violations=1 (substring match must not count as a stamp)", n, len(v))
}
// An auxiliary-table write (has issue_id) must be ignored, stamped or not.
aux := []byte(`package p
func w(tx T, id, s string) {
tx.ExecContext(ctx, "INSERT INTO %s (id, issue_id, event_type) VALUES (?, ?, ?)", a, b, c)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "aux.go", aux); n != 0 || len(v) != 0 {
t.Errorf("aux write: got checked=%d violations=%v; want checked=0 violations=none", n, v)
}
// An error-format string that echoes the verb+%s shape — like domain/db's
// issue.go fmt.Errorf("db: Update %s: %w", id, err) — must not be counted
// as a table write at all (finding 1 of the adversarial review of
// ea1256462): it names no SET/VALUES/SELECT.
errorFormatString := []byte(`package p
func w(id string, err error) error {
return fmt.Errorf("db: Update %s: %w", id, err)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "errfmt.go", errorFormatString); n != 0 || len(v) != 0 {
t.Errorf("error-format string: got checked=%d violations=%v; want checked=0 violations=none", n, v)
}
// A lifecycle write (assigns status/assignee) must not escape via an
// aux/exempt marker matched only in its WHERE predicate (finding 2): a
// hypothetical `... WHERE is_blocked = 0` must not wave through a status
// change the way the real is_blocked-recompute exemption (which never
// assigns status/assignee/started_at) legitimately does.
lifecycleEscapeAttempt := []byte(`package p
func w(tx T, id, s string) {
tx.ExecContext(ctx, "UPDATE issues SET status = ?, assignee = NULL WHERE is_blocked = 0", s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "lifecycle.go", lifecycleEscapeAttempt); n != 1 || len(v) != 1 {
t.Errorf("lifecycle write behind an is_blocked WHERE marker: got checked=%d violations=%d; want checked=1 violations=1 (a marker in WHERE must not exempt a SET that assigns status/assignee/started_at)", n, len(v))
}
// finding 3: the stamp check used to be computed once per FUNCTION, so an
// unrelated funnel write's stamp call covered a sibling DIRECT write that
// never stamps at all. The two writes below share one function; the
// first is a real funnel (RowLockClause() interpolated via %s, stamped)
// and the second is a plain literal SET with no row_lock anywhere.
mixedFuncDirectUnstamped := []byte(`package p
func w(tx T, id, s, table string) {
rowLockClause, rowLockArgs := RowLockClause()
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, %s WHERE id = ?", table, rowLockClause), append([]interface{}{s}, rowLockArgs...)...)
tx.ExecContext(ctx, "UPDATE issues SET status = ? WHERE id = ?", s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "mixeddirect.go", mixedFuncDirectUnstamped); n != 2 || len(v) != 1 {
t.Errorf("mixed function (one funnel write stamped, one direct write not): got checked=%d violations=%d; want checked=2 violations=1 (the direct write must not ride the funnel write's stamp)", n, len(v))
}
// finding 4: the funnel fallback itself must be AST-based, not a text
// scan over the function body — a stamp name mentioned only in a comment
// must not count. The write below has a %s-templated SET clause (so it
// is eligible for the fallback) and no real freshRowLock/RowLockClause
// call anywhere in the function, only a comment naming one.
commentOnlyStampCall := []byte(`package p
func w(tx T, id, s, table string) {
// freshRowLock() is minted elsewhere (WRONG: no such call exists here).
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, %s WHERE id = ?", table, "row_lock = ?"), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "commentonly.go", commentOnlyStampCall); n != 1 || len(v) != 1 {
t.Errorf("comment-only stamp mention: got checked=%d violations=%d; want checked=1 violations=1 (a stamp name mentioned only in a comment must not satisfy the funnel fallback)", n, len(v))
}
// Finding 2 of the dual-vendor review of d10544c96: MySQL backtick-quoted
// identifiers (versioncontrolops's UPDATE `issues` SET ...) escaped the
// original bare-identifier regex entirely — a write in that shape was
// never even matched, let alone checked. Built with string concatenation
// (not a raw-string literal) because the payload itself needs a literal
// backtick, which a backtick-delimited Go raw string cannot contain.
backtickUnstamped := []byte("package p\n" +
"func w(tx T, id, s string) {\n" +
"\ttx.ExecContext(ctx, \"UPDATE `issues` SET status = ? WHERE id = ?\", s, id)\n" +
"}")
if n, v := scanIssueWriteRowLockStamps(t, "backtickunstamped.go", backtickUnstamped); n != 1 || len(v) != 1 {
t.Errorf("backtick-quoted unstamped lifecycle write: got checked=%d violations=%d; want checked=1 violations=1 (UPDATE `issues` must match despite backtick quoting)", n, len(v))
}
// The backtick-quoting fix must not introduce a false positive on a
// legitimately stamped backtick-quoted write.
backtickStamped := []byte("package p\n" +
"func w(tx T, id, s string) {\n" +
"\ttx.ExecContext(ctx, \"UPDATE `issues` SET status = ?, row_lock = ? WHERE id = ?\", s, freshRowLock(), id)\n" +
"}")
if n, v := scanIssueWriteRowLockStamps(t, "backtickstamped.go", backtickStamped); n != 1 || len(v) != 0 {
t.Errorf("backtick-quoted stamped write: got checked=%d violations=%v; want checked=1 violations=none", n, v)
}
// Finding 1 of the dual-vendor review of d10544c96: versioncontrolops'
// automerge splits row_lock's plan-building step (which mints the token)
// from its generic plan-apply step (which writes whatever the plan names,
// including row_lock, without ever mentioning the column). The two live
// in SIBLING functions of one file, so the mint call is invisible to a
// per-function scan; recognition needs to be file-scoped instead
// (containsCrossFuncStamp), gated on the two call forms that pattern
// actually uses: freshRowLockDistinctFrom() and setColumn("row_lock", ...).
crossFuncStampedViaMintCall := []byte(`package p
func buildsPlan(a, b int) int {
return freshRowLockDistinctFrom(a, b)
}
func w(tx T, id, s, table string) {
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, %s WHERE id = ?", table, "row_lock = ?"), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "crossfuncmint.go", crossFuncStampedViaMintCall); n != 1 || len(v) != 0 {
t.Errorf("cross-function stamp via a sibling's freshRowLockDistinctFrom call: got checked=%d violations=%v; want checked=1 violations=none", n, v)
}
crossFuncStampedViaSetColumn := []byte(`package p
func buildsPlan(m *plan, v int) {
m.setColumn("row_lock", v)
}
func w(tx T, id, s, table string) {
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, %s WHERE id = ?", table, "row_lock = ?"), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "crossfuncsetcolumn.go", crossFuncStampedViaSetColumn); n != 1 || len(v) != 0 {
t.Errorf("cross-function stamp via a sibling's setColumn(\"row_lock\", ...) call: got checked=%d violations=%v; want checked=1 violations=none", n, v)
}
// The cross-function fallback must still refuse a %s-templated write with
// no stamp evidence ANYWHERE in the file — a sibling that calls
// setColumn with an unrelated column name must not be mistaken for a
// row_lock stamp.
crossFuncNoStamp := []byte(`package p
func buildsPlan(m *plan, v string) {
m.setColumn("title", v)
}
func w(tx T, id, s, table string) {
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, %s WHERE id = ?", table, "not_row_lock = ?"), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "crossfuncnostamp.go", crossFuncNoStamp); n != 1 || len(v) != 1 {
t.Errorf("no stamp evidence anywhere in file: got checked=%d violations=%d; want checked=1 violations=1 (an unrelated setColumn call must not satisfy the cross-function fallback)", n, len(v))
}
// funcNameExemptions is keyed to the enclosing function's identity, not a
// text marker, so it must exempt the exact named function (conflicts.go's
// whole-row `theirs` adoption) even though its SET clause could assign
// status/assignee at runtime...
exemptByName := []byte(`package p
func resolveOneConflictRow(tx T, id, s, table string) {
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, assignee = ? WHERE id = ?", table), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "exemptbyname.go", exemptByName); n != 0 || len(v) != 0 {
t.Errorf("resolveOneConflictRow func-name exemption: got checked=%d violations=%v; want checked=0 violations=none (a whole-row-adoption write named by function identity must be exempt)", n, v)
}
// ...but must NOT exempt a differently-named function with the identical
// write shape: the exemption is the specific reviewed function, not the
// SQL shape.
notExemptByName := []byte(`package p
func someOtherWrite(tx T, id, s, table string) {
tx.ExecContext(ctx, fmt.Sprintf("UPDATE %s SET status = ?, assignee = ? WHERE id = ?", table), s, id)
}`)
if n, v := scanIssueWriteRowLockStamps(t, "notexemptbyname.go", notExemptByName); n != 1 || len(v) != 1 {
t.Errorf("non-exempt function with the same write shape: got checked=%d violations=%d; want checked=1 violations=1 (the exemption must be keyed to the specific function name, not the SQL shape)", n, len(v))
}
}
func hasAnyMarker(s string, markers []string) bool {
for _, m := range markers {
if strings.Contains(s, m) {
return true
}
}
return false
}
func funcDisplayName(fn *ast.FuncDecl) string {
if fn.Recv != nil && len(fn.Recv.List) > 0 {
if t, ok := receiverTypeName(fn.Recv.List[0].Type); ok {
return "(" + t + ")." + fn.Name.Name
}
}
return fn.Name.Name
}
func receiverTypeName(expr ast.Expr) (string, bool) {
switch t := expr.(type) {
case *ast.StarExpr:
return receiverTypeName(t.X)
case *ast.Ident:
return t.Name, true
}
return "", false
}
func firstSQLLine(window string) string {
for _, line := range strings.Split(window, "\n") {
line = strings.TrimSpace(line)
if line != "" {
return line
}
}
return strings.TrimSpace(window)
}
// enclosingSQLLiteral returns the Go string literal (backtick-raw or
// double-quoted) that contains the byte at matchStart, without its delimiters.
// The SQL write statements here never span more than one literal, and neither
// delimiter appears inside these SQL bodies (which use single quotes), so a
// nearest-delimiter scan bounds the statement exactly — no bleed into adjacent
// statements. Falls back to a fixed window if no delimiter is found.
func enclosingSQLLiteral(body string, matchStart int) string {
open, delim := -1, byte(0)
for i := matchStart; i >= 0; i-- {
if body[i] == '`' || body[i] == '"' {
open, delim = i, body[i]
break
}
}
if open < 0 {
end := matchStart + 400
if end > len(body) {
end = len(body)
}
return body[matchStart:end]
}
for i := open + 1; i < len(body); i++ {
if body[i] == delim && (delim == '`' || body[i-1] != '\\') {
return body[open+1 : i]
}
}
return body[open+1:]
}