beads/internal
Rome Thorstenson b4ac361989
fix(schema): drain proc result sets and heal partial 0040/0041 via repairs (#4504)
A fresh `bd init` against a shared Dolt sql-server under concurrent load could
permanently brick the database at migration 0040. Two compounding causes:

1. An undrained `CALL DOLT_*` result set on the pinned migration connection
   ("busy buffer" -> "driver: bad connection") failed the SEPARATE version-record
   INSERT that follows it, so schema_migrations never advanced past the
   partially-applied migration.
2. Migrations 0040/0041 are non-idempotent (0040 bare-INSERTs its
   dolt_nonlocal_tables rows; 0041 DELETEs then commits them without
   --skip-empty), so the init retry loop's replay died on "duplicate primary key
   given: [wisps]" (0040) or "nothing to commit" (0041).

drainCall() fixes cause 1 (pure Go, no migration-content change). For cause 2,
the shipped 0040/0041 bytes are frozen and content-hashed
(scripts/check-migration-hygiene.sh check C): editing them would change the
recorded content_hash and fork every already-upgraded clone (the #4259 skew the
smart remote-migrate gate, `bd doctor`, and merge-settle all detect as a genuine
schema fork). Per the migration_repairs.go doctrine, a failing migration also
cannot be fixed forward with a NEW migration (a database bricked at 40 never
reaches it). So idempotency is delivered as pre-migration repairs keyed to
versions 40 and 41 that normalise dolt_nonlocal_tables so each frozen body
replays cleanly — leaving 0040/0041's released bytes (and their content hashes)
untouched.

- schema.go: drainCall() consumes every CALL DOLT_* result set on the pinned
  migration connection — migration bodies via execMigrationBody, the
  apply-migrations DOLT_COMMIT, DOLT_RESET, DOLT_ADD, commitSeededDoltIgnore's
  add/commit pair, and commitMigrationStep's per-step add/commit;
  GET_LOCK/RELEASE_LOCK stay single-result SELECTs.
- lock.go: the fresh-bootstrap heal's DOLT_RESET('--hard') drains too — it is
  immediately followed by a full MigrateUp pass on that same pinned conn.
- migration_repairs.go: keyed preMigrationRepairs registry;
  repairPartial0040NonlocalInsert / repairPartial0041NonlocalDelete.
- migration_body_drain_test.go: drain-classification guards +
  TestFrozenNonIdempotentMigrationsHaveRepairs (a frozen non-idempotent
  nonlocal-table migration must have a registered repair, not edited SQL).
- initschema_idempotent_test.go (integration): forge the committed partial-0040
  and partial-0041 brick states on a real sql-server; assert re-init recovers.
- lock_test.go / schema_test.go: mock the converted DOLT_ADD/RESET/COMMIT sites
  as queries because drainCall uses QueryContext.

Rebased onto current main and reworked per maphew's review on #4504:

- Corrected the drainCall doc-comment mechanism. go-sql-driver's mysqlConn.exec
  DOES end in handleOk.discardResults(), so the happy path was never the gap.
  The real asymmetry is the error path: exec returns early — without discarding
  — when readResultSetHeaderPacket/skipColumns/skipRows fails mid-stream, while
  mysqlRows.Close() drains unconditionally. That asymmetry is reachable here
  precisely because MigrateUp and commitMigrationStep swallow "nothing to
  commit" and keep using the same pinned connection.
- Extended the conversion to the CALL DOLT_* sites main added since the original
  base (commitMigrationStep, commitSeededDoltIgnore, lock.go's
  DOLT_RESET('--hard')), which are the hot ones.
- Lowered TestRunMigrationsLargeRigNoticeOnlyOnMainSource's mock bounds to 39,
  matching the two tests already lowered. It merges silently outside every
  conflict region and would otherwise reach 40/41 and panic on the mockDB stubs.
- Scoped the repair commits: they stage dolt_nonlocal_tables BY NAME instead of
  DOLT_COMMIT('-Am', ...), so on the bounded-migrate path (upTo != 0, per-step
  commit off) they no longer sweep migrations 1..upTo's uncommitted working set
  into a repair-labelled commit. Verified against a real dolt server that a
  scoped add stages only that table and leaves the rest of the working set
  untouched.
- Conflict resolution kept main's repairs at versions 47 and 53 (including
  #4690's ensureDependenciesIDColumn) in the new keyed registry.

Verified: go build, go vet, scripts/check-migration-hygiene.sh, the full
internal/storage/schema suite, and both integration tests in
initschema_idempotent_test.go against a real dolt sql-server.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Executed-By: beads/crew/codd
2026-07-28 13:29:08 -07:00
..
ado fix(ado): preserve unmanaged links on push instead of clobbering them (#4523) 2026-07-05 10:53:05 -07:00
atomicfile /internal/atomicfile/atomicfile.go: use io.Copy 2026-05-01 17:03:30 -07:00
audit fix(audit): make JSONL sidecar opt-in (#4688) 2026-07-11 00:09:17 -07:00
beads fix(beads): allow /var/tmp in repo-context safety boundary (#5028) 2026-07-25 14:02:41 -07:00
compact fix(compact): archive content before destructive compaction; make restore work (#4464) 2026-06-20 14:12:07 -07:00
config feat(dolt): make the shared pool's per-I/O read/write deadlines configurable (bd-vz0y9) (#5089) 2026-07-27 15:46:59 -07:00
configfile refactor(storage)!: remove the SQLite backend and dialect 2026-07-18 02:43:56 +00:00
creds fix(backends): address red-team review findings before merge 2026-07-10 18:58:00 +00:00
debug feat(verbosity): wire --quiet into create/close/update (bd-au0.10) (#4058) 2026-07-06 22:35:42 -07:00
doltremote fix(doltremote): fold host case, strip creds, recognize user-less SCP in CanonicalForComparison (#4576) 2026-07-07 12:59:18 -07:00
doltserver fix(doltserver): sweep Darwin test servers (wy-s4zdj) 2026-07-25 20:48:10 -07:00
formula feat(formula): make primitives discoverable to coding agents (GH#3621) (#3781) 2026-07-06 22:26:41 -07:00
git test(git): restore cwd before temp cleanup (#4935) (#4936) 2026-07-23 18:40:44 -07:00
github fix(github) bd github sync (GitHub issues integration) (#3929) 2026-05-15 08:47:39 -07:00
gitlab feat(gitlab): sync issue dependency links on push (#4294) 2026-07-08 08:00:41 -07:00
hooks test: port Windows hardening fixes (#4600) 2026-07-07 12:59:50 -07:00
idgen fix(idgen): correct test vectors in hash_test.go 2025-12-19 18:13:56 -08:00
jira feat(storage): keyset reads — comments page, issue-list cursor, IsBlocked filter (#4922) 2026-07-21 22:29:09 -07:00
linear fix(windows): detect exited processes in liveness helpers (#4980) (#4981) 2026-07-23 18:24:49 -07:00
lockfile Triage and clean up an unreviewed automated commit stream on main (#4418) 2026-06-15 20:24:57 -07:00
metrics feat: honor DO_NOT_TRACK as a metrics opt-out (#4938) 2026-07-21 08:35:20 -07:00
molecules feat(storage): refuse test DBs on production dolt servers (AD-01) (#3632) 2026-07-24 01:28:51 -07:00
notion fix(notion): drop in_trash from data-source query body (Notion 400) (#4543) 2026-07-05 10:53:01 -07:00
procid fix(lint): check deferred windows.CloseHandle returns (errcheck) (#5088) 2026-07-26 14:47:37 -07:00
query fix(query): make slash and mixed-case metadata keys queryable (#4896) 2026-07-24 17:52:22 -07:00
recipes [codex] Add Kiro setup recipe (#4535) 2026-07-27 22:28:43 -07:00
remotecache fix: pass remote auth user during clone (#4484) 2026-07-05 11:28:24 -07:00
routing fix(routing): warn when contributor auto-routing swaps bd list/ready to the planning store (#4866) 2026-07-25 22:31:03 -07:00
storage fix(schema): drain proc result sets and heal partial 0040/0041 via repairs (#4504) 2026-07-28 13:29:08 -07:00
telemetry feat(storage): keyset reads — comments page, issue-list cursor, IsBlocked filter (#4922) 2026-07-21 22:29:09 -07:00
templates fix(cli): printed doc paths point at the new docs locations 2026-07-12 10:37:17 -07:00
testutil /{.github,docs,engdocs,go,internal,scripts}: bump dolt 2026-07-15 13:14:34 -07:00
timeparsing chore: unexport 14 dead symbols across 8 packages 2026-02-07 09:24:22 -08:00
tracker feat(storage): refuse test DBs on production dolt servers (AD-01) (#3632) 2026-07-24 01:28:51 -07:00
types feat(cli): bd schema — JSON Schema for --json/export output (carries #4413, griels) (#5098) 2026-07-27 14:16:56 -07:00
ui test(cli): cover --no-color flag handling and ui.DisableColors 2026-07-07 23:32:22 -04:00
uimd fix(cli): render markdown with terminal capability gates (#3881) 2026-06-25 13:55:28 -07:00
utils fix(mol): address maphew PR review feedback on #5027 2026-07-27 13:07:50 -07:00
validation fix(update): refuse overwriting a live foreign claim without --force (bd-98s5c) (#5066) 2026-07-27 15:10:12 -07:00
workspacegate feat(workspacegate): two-level cooperative workspace/physical-root gate (part 1: primitive) (#5046) 2026-07-26 12:05:10 -07:00