Commit graph

4 commits

Author SHA1 Message Date
Bee
91aeeb0548
perf(storage/db): batch fetchIssuesByIDs, GetBlockingInfo and loadStatusByID at 200 ids; delete tolerates only a missing wisps table (#6444)
* perf(storage/db): batch the bulk IN-list readers at 200 ids; export narrows the wisp-plane reads

bd export on a 19,413-issue rig in proxied-server (UOW) mode took 264-287s.
The cmd layer is bulk, but uowExportSource.LoadExportRelations handed ALL
ids to the domain use cases, and the domain/db readers built ONE IN (...)
list per statement: 19,409 placeholders each for labels, wisp_labels,
wisp_comments rows and counts, dependencies counts and wisp_dependencies
counts, at 24-60s a statement (~240s of the run). Dolt is super-linear in
IN-list size, prepared or literal alike: on the same server the labels rows
took 31s as one statement and 0.80s as 98 batches of 200; a 200-id batch
is 3-7ms. The wisp readers were also handed every durable id (rows=0 back).

- internal/storage/domain/db/batching.go: forEachIDBatch at queryBatchSize
  (200, mirroring issueops). commentSQLRepositoryImpl.CountsByIssueIDs /
  ListByIssueIDs, labelSQLRepositoryImpl.ListByIssueIDs,
  dependencySQLRepositoryImpl.ListByIssueIDs / CountsByIssueIDs and
  issueSQLRepositoryImpl.GetByIDs loop over batches and merge; an id lands in
  exactly one batch, so per-id groups and their ORDER BY survive. Error
  strings are byte-identical; empty input still issues no statement. The
  three hand-rolled loops (getLabelsFromTable, getDependencyRecordsFromTable,
  fetchCountsByIDs) collapse onto the helper.
- cmd/bd/export_source.go: LoadExportRelations reads wisps-table membership
  once (WispPlaneIDs, now batched through GetByIDs) and sends only wisp ids
  to GetLabelsForWisps / GetCommentsForWisps / GetWispCommentCounts;
  CountsByWispIDs keeps every id because its inbound leg counts wisp->durable
  edges under the durable target. Membership that could not be read fails
  OPEN to every id (wispReaderIDs), never to "no wisps".
- WispPlaneIDs now tolerates only a missing WISPS table
  (dberrors.IsMissingTable), not any table-not-exist: its read joins leases,
  and the blanket check reported a broken database as an empty wisp plane
  (the issue_search.go missingOptionalWispTable lesson). Behavior change on
  purpose: a rig missing leases now fails bd export with "failed to classify
  wisp-plane rows" instead of silently exporting without markers.

Measured in the same clone against the same shared server, 19,433 rows:
stock 264s / 266s / 287s -> batched 16.9s / 17.8s. Stock-vs-batched byte
oracle: 19,363 of 19,433 records byte-identical; every differing record is
a live lease/heartbeat field or an updated_at inside the seven minutes
between the two runs.

Tests: internal/storage/domain/db TestBulkReadersBatch (a recording Runner
over the real Dolt suite: ceil(N/200) statements per leg, <=200 id
placeholders each, merge across batch boundaries, the Types filter's extra
placeholder, every reader routed to the wisp tables, GetByIDs, empty input),
cmd/bd TestWispPlaneSubset + TestWispReaderIDs, and the proxied export
integration fixture's wisp now carries a label and a comment that the --all
export must emit.

Follow-up (not here): fetchIssuesByIDs (SearchIssues Limit=0),
GetBlockingInfo and loadStatusByID still build one unbounded IN list, and
issue_delete.go's two GetByIDs(wisps) calls keep the blanket
IsTableNotExist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDjKMk7ubzrgpEDj2JRAbu

* perf(storage/db): batch fetchIssuesByIDs, GetBlockingInfo and loadStatusByID at 200 ids; delete tolerates only a missing wisps table (wy-sm01o2)

Follow-up to #6267 (wy-237yfi), whose own body names these three readers as
the residual unbounded IN lists in internal/storage/domain/db. Dolt's cost per
IN-list placeholder is super-linear, so one statement over N ids is far slower
than ceil(N/200) statements over 200 each.

- issue_search.go fetchIssuesByIDs: one SELECT per <=200-id batch, merged into
  the same map. `ordered` is the scan order concatenated across batches and
  hydration runs ONCE over the merged slice (hydrateIssues batches its own
  relation reads since #6267), so callers get the same map and the same
  hydration for one round trip per 200 ids instead of one 19k-placeholder
  statement (export's SearchIssues with Limit=0 reaches it with ~19k ids).
  The signature and the missing-wisp-table classification are unchanged.
- dependency.go GetBlockingInfo: both legs batch through forEachIDBatch and
  their blockingRows are appended across batches; the merge that follows is
  untouched. Each leg keys its result by a column the IN list constrains
  (outbound issue_id, inbound the dependency target), and an id lands in
  exactly one batch, so every row sharing a key comes from the same batch.
  loadStatusByID batches its id list per table; the cross-plane duplicate-id
  error is unchanged, because an id is still recorded under `issues` before
  the `wisps` pass reaches it. GetBlockingInfoAcrossIssuesAndWisps needs no
  change: it only calls GetBlockingInfo twice and merges the maps.
- issue_delete.go: previewDelete's and collectConnectedIssues' two
  GetByIDs(UseWispsTable) calls read `FROM wisps LEFT JOIN leases`, so their
  blanket dberrors.IsTableNotExist reported a rig missing `leases` as a rig
  with no wisp plane (previewDelete listed live wisps under NotFound;
  hydrateNeighbors returned them unhydrated). Both now tolerate only
  dberrors.IsMissingTable(err, "wisps"), the fix #6267 applied to
  WispPlaneIDs. The other IsTableNotExist sites in that file are left alone:
  they guard reads whose FROM is the optional table itself
  (wisp_dependencies via ListByIssueIDs(UseWispsTable)).

Tests: internal/storage/domain/db TestBulkReadersBatch gains FetchIssuesByIDs
(ceil(N/200) statements, <=200 placeholders, rows on both sides of a boundary
merged, and exactly ONE label hydration over the merged slice),
GetBlockingInfoOutbound, GetBlockingInfoInbound, LoadStatusByID (both planes,
one statement per batch each) and LoadStatusByIDDuplicateAcrossTables; the
empty-input subtest now covers all three readers issuing no statement.
internal/storage/domain/issue_delete_missing_table_test.go proves a missing
`leases` (or `wisp_labels`) is no longer swallowed at either GetByIDs(wisps)
site while a missing `wisps` table still is, with an unrelated-error control.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(storage/db): consume the aborted duplicate-plane run's statements inside its own subtest (wy-sm01o2)

Salvaged WIP from the quota-killed executor (session-limit at 00:19Z): the
duplicate-plane loadStatusByID call aborts mid-run, so reset the recorder
before it and assert its statements under the cap in place, rather than
leaving them to pollute the next subtest.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rrz4umA2L5QU8s9EmCBPw8

* docs(cmd/bd): correct the LoadExportRelations plane-partition doc block

The header still documented pre-PR full-set wisp reads ("both planes are
queried with the FULL ID set and merged"), which this PR made false for the
wisp side. That block is the load-bearing record of the plane-partition
invariant, so a future editor trusting it could "restore" full-set wisp reads
and silently undo the wy-237yfi fix.

- The first bullet now describes the partitioned reads: membership is read
  from the wisps table itself (WispPlaneIDs) rather than inferred from row
  flags, the durable plane is queried with the full id set, the wisp plane
  only with that membership subset (wispReaderIDs), and unreadable membership
  fails OPEN to the full list. The "row flags are NOT a substitute" rationale
  is kept deliberately: it is *why* membership is read from the table at all.
- The IsTableNotExist sentence is extended, not replaced. It is still true of
  the three single-table wisp-relation readers, which correctly keep the
  blanket check; only the lease-joined membership probe (WispPlaneIDs) is
  pinned to a missing `wisps` table. Replacing the sentence outright would
  have introduced a new inaccuracy.

Maintainer review fixup for adopt-pr #6444. Documentation only: the tree is
AST-identical after stripping comments, so no behavior can change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
2026-09-12 02:19:39 -07:00
Julian Knutsen
f925f3f5b0
Retire the last three wisp-switch call sites, and bound the wisp-inclusive union reads (ga-2ltro.12, ga-2ltro.7) (#5587)
* Retire the last three wisp-switch call sites, and open the label guard on the whole package (ga-2ltro.12)

ga-26w10 deleted a four-way switch over (add|remove) x (wisp|durable) from
`bd label` and lint the door shut behind it. THE SWITCH SURVIVED AT THREE MORE
SITES, and the rule that was supposed to stop the next one could not see two of
them. This retires all three and makes the rule package-wide.

`bd tag`'s proxied route (cmd/bd/mutate_proxied_server.go) reached
uw.LabelUseCase() through proxiedMutateIssue, whose callback signature hands out
a unit of work AND an isWisp boolean — an invitation to pick a plane, which the
tag callback accepted. It now builds one IssuePatch and hands it to
issueops.Lifecycle.Update. IssuePlaneOnly stays false, so the role resolves the
plane inside its own transaction; TestProxiedServerTag/tag_wisp_routes_to_wisp_labels,
which asserts at the SQL table that a wisp's label lands in wisp_labels and the
durable table stays empty, passes through the role with no branch in the
command. It went red on the pre-fix tree under the boolean flipped backwards,
which is the mutation this exists to kill and which is no longer expressible.

THE MOLECULE PORT'S LABEL WRITE IS GONE RATHER THAN RE-ROUTED, and that is the
stronger fix. molWriter had an AddLabel verb with exactly one caller —
bondProtoProtoInto, labelling a compound it had just created — and two
implementations that each resolved the plane AGAIN by a DIFFERENT predicate:
storeMolWriter deferred to the storage layer's isActiveWisp, uowMolWriter
consulted a cache filled from IssueUseCase().GetWisp. Two predicates for one
fact is how a row and its labels come apart. The label now travels in
compound.Labels, so the create that decides the row's table places the label
too, and AddLabel is deleted from the interface and both implementations. The
disagreement is unrepresentable, not merely unlikely.

Worth stating plainly for the reviewer: uowMolWriter.AddLabel's WISP branch was
unreachable. Its only caller labels a durable epic it just created, so the cache
answers false every time. That is why this site had no end-to-end mutation to
run and why the new test is a unit test with a recording writer: it pins that
the create carries the label, which is the property that makes the branch
unnecessary rather than merely unused.

`bd set-state` had the switch FOUR times — the label read, the event create, the
old-label remove and the new-label add — and a second implementation of the whole
command in cmd/bd/state_proxied_server.go that no longer exists. All three verbs
now have ONE body each, on both routes, ending at issueops.Reader.Get and
issueops.Lifecycle:

  - the label READ is Reader.Get's already-hydrated, already-sorted Labels, so
    there is no label-shaped read here at all — the same move ga-26w10 made for
    `bd label list`;
  - the dimension SWAP is one IssuePatch carrying Add and Remove together;
  - the EVENT is one Lifecycle.Create with ParentID, which mints the same
    hierarchical child id and writes the parent edge atomically.

FOUR DELTAS, none of them absorbed.

THE DIRECT ROUTE'S DELTA IS THE EVENT'S PLANE, AND ONLY THAT. An earlier draft
of this message claimed the direct route "read no state and wrote its label into
the wrong table" for a wisp. That was reasoned, not measured, and it is FALSE:
store.GetLabels, AddLabel and RemoveLabel all reach issueops.*InTx with an EMPTY
table name, which routes by IsActiveWispInTx, and utils.ResolvePartialID
resolves through the wisp-inclusive search. Measured at the parent commit,
`bd set-state <wisp> patrol=muted` on the direct route resolves, swaps the
label, and `bd state <wisp> patrol` reads back "muted".

What the parent DID get wrong is the EVENT. It created the state-change bead
with store.CreateIssue and no wisp flag, so an ephemeral subject got a DURABLE
event child, and the parent edge was a second unguarded write across the two
planes. Measured on both sides of this commit, same wisp, same command:

  parent   event ms-wisp-865.1, no `ephemeral` field  -> durable
  this     event ms-wisp-859.1, `"ephemeral": true`   -> the subject's own plane

so a wisp's state history now dies with the wisp instead of outliving it in the
durable table, and its parent edge is written inside the guarded create rather
than beside it.

That correction is the finding, not the footnote: this commit's own brief is
MEASURED, NOT REASONED ABOUT, and the sentence that broke the rule was in the
commit message. The rule covers the artifact too.

A FAILED REMOVAL NO LONGER PASSES SILENTLY — with one half still open. Both
retired routes warned and carried on when the remove failed, leaving the issue
holding two values for one dimension, which findStateLabel then resolves by
whichever the label read returns first. A store-level failure now fails the
command, because the remove and the add are one guarded update.

WHAT THAT DOES NOT CLOSE, stated because the first draft of this line
overclaimed it as "the dimension has one value or nothing was written": the
CONCURRENT swap. Two racing `bd set-state <id> dim=...` calls both read the same
old label, each patch removes a label the other already removed — a no-op, not a
conflict — and both adds land, so the dimension ends with TWO values. A patch is
atomic for ONE writer; it is not a fence. ExpectedVersion cannot supply one
either, because label writes deliberately do not remint RowVersion. The race is
pre-existing, is not made worse here, and is filed as ga-dle5g.

The proxied route LOSES ONE TRANSACTION. Its event and its label swap used to
commit together; they are now two role calls. The direct route was never atomic
at all (four unguarded writes), so this is a net gain there and a stated cost
here, and it is the same trade ga-26w10 took for the same reason. The event's
plane is the subject's own retention mode copied onto the child rather than
chosen, and it is CHECKED: a guarded create refuses a parent edge it cannot
write, so an event that disagreed with its subject's plane fails loudly instead
of landing in the wrong table.

DOLT HISTORY ATTRIBUTION MOVES, which is cosmetic and is said anyway because an
unstated delta is how the two corrections above came to be needed. The role
composes its own version-commit message, so `bd tag` and `bd set-state` write
"bd: update <id>" where they used to write "bd: tag <id>" and
"bd: set-state <id>". `bd dolt log` still names the issue; it no longer names
the verb. Nothing greps for those strings in this tree.

THE FORBIDIGO RULE IS NOW PACKAGE-WIDE ACROSS cmd/bd. It was narrowed by a
path-except to cmd/bd/label.go on the argument that holding an unmigrated
command to a rule written for a migration it had not had would be a lint waived
on sight. What that missed is that the waiver was INVISIBLE — a file outside the
exemption got no marker, no reason and no diff — so the three commands still
hand-rolling the switch looked exactly like the files with nothing to declare.
Two of them were additionally exempted by `path:`-only holes written for the
FILTER rule, which silently open every forbidigo rule for their files:
mol_port.go by the mol_ hole and export_source.go by the bulk-movement one.

EVERY HOLE IN THAT FAMILY IS `text:`-SCOPED NOW, not just those two. Scoping
only the holes that covered a violation TODAY was the same reasoning error as
the paragraphs above — it sized the guard by where the calls are rather than by
where the next one may be WRITTEN — and a review probe proved it with two dodges
that both passed: a neutrally-named helper dropped into the ado/gitlab/jira/
linear hole, and that helper handing the RAW use case back to a NEW file, which
then needs no import and never spells the accessor. Both are caught now, the
second one twice (the return type and the call), and both were re-confirmed to
PASS with the hole unscoped, so the fix is the scope and not a coincidence. The
`_test\.go` exemption stays unscoped on purpose: `run: tests: false` means
forbidigo never analyzes a test file, so scoping it would assert nothing.

MEASURED, NOT REASONED ABOUT. Before: 27 LabelUseCase call sites in 8 non-test
files. After: 15 in 6, every one carrying //nolint:forbidigo and the reason that
site is not a role's question, and they add up: five in-transaction port reads
(mol_port.go), two alternate-view reads (show_proxied_server.go), two bulk
relation loads (export_source.go), two label-vocabulary reads and two
label-propagate WRITES (label_proxied_server.go), one in-transaction command
read (close_proxied_server.go) and one dry-run preview read
(create_proxied_server.go). 5+2+2+2+2+1+1 = 15. The propagate pair are the
acknowledged carve-out and the only survivors that write; each site names its
follow-up, and propagate's is issueops.BatchApplier.ApplyBatch, blocked on a
cmd/bd accessor.

Both halves verified by mutation, one bypass each. A scratch
uw.LabelUseCase().AddLabel() planted in cmd/bd/state.go — reached through
proxiedOpenReadUOW, so the file imports neither internal/storage/uow nor
internal/storage/domain and depguard is blind to it — is caught by forbidigo. A
scratch uow import in the same file is caught by depguard, which cmd/bd/state.go
now joins: it is the second entry on that list because its three verbs really
have migrated, so the rule pins something that happened.

The depguard prose said the forbidigo rule was "scoped to this same file" and
the exclusions block said its label narrowing cost "a `text:` on ONE of the
per-path holes". Both are now what they are: package-wide, and three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Bound the union legs, and stop letting SQL page a sort it cannot order (ga-2ltro.7)

The two residues the #5476 review deferred, and they turned out to be one rule
observed from both ends: A WINDOW IS ONLY EVER PUSHED UNDER AN ORDER THE QUERY
CAN EXPRESS.

F4 — THE LEGS CARRY THE WINDOW NOW. buildUnionSubquery emitted no bound on
either side of the UNION ALL; only the outer query had one, so both planes were
read in full to deliver a page. Measured on a seeded two-plane store, a
four-row page with twenty matching rows per plane: each leg returned 20 rows,
40 scanned to deliver 4. Each leg now carries the outer clause VERBATIM plus the
deepest row index the outer query can reach, and returns 5 — the bound plus its
probe row — for 10 scanned instead of 40.

EACH LEG IS PARENTHESIZED NOW, and that is load-bearing rather than cosmetic: a
leg carrying its own ORDER BY and LIMIT inside a bare `leg UNION ALL leg` is a
SYNTAX ERROR, the engine reading the trailing clause as the union's own. The
first cut of this change shipped without the parentheses and `bd search` failed
outright — caught by CI, not by the leg probe below, because that probe executes
each subquery STANDALONE where the same string is perfectly legal. A test that
measures the part is not evidence about the whole, so there is now a case that
runs the composed query on both twins across six shapes (ordered, offset, text
term, default order, unbounded, Go-side), and it goes red on the un-parenthesized
form with the exact engine error.

Reusing the outer ORDER BY string rather than rebuilding it is what makes the
bound safe: unionOrderBySQL names `id` and the `sort_*` aliases, both of which
every leg already projects, so one string is valid in both positions and a leg
cannot come to rest in a different order than the one the outer query is about
to cut against. The pushdown is sound because the order is TOTAL — every clause
sqlbuild renders ends in `id ASC` — so a row in the union's top-N is in its own
leg's top-N, there being nothing else it could be behind. legLimit is the bound
PLUS any offset the engine carries, because those are rows the outer query reads
and discards; an offset this package keeps for itself is already inside the
bound.

F5 — AND THE OTHER HALF IS WHY THE LEGS GET NOTHING UNDER `--sort id`. That key
is the one sqlbuild renders no ORDER BY for, so the query ran
`SELECT id, src FROM (leg UNION ALL leg) merged  LIMIT n` — a LIMIT with no
ordering anywhere in the statement. A LIMIT without an ORDER BY does not return
the first n rows, it returns n rows.

MEASURED, and it is worse than "arbitrary". On a store with 12 durable and 12
ephemeral beads whose ids interleave, `--sort id --limit 5` answered
[000 002 004 006 008]: five durable rows, and not one wisp. The Go-side sort
then put them in perfect order, which is why this reads as a correct page — the
ORDER was right and the MEMBERSHIP silently excluded an entire plane.

Two changes close it, and both are needed:

  - searchWindowFor pushes NO page bound for a Go-side sort, which is the same
    answer internal/workapi.SQLLimit gives the reader role for the same reason.
    The CAP is not waived with it: maxRows still sizes a bound, so an unordered
    scan too large to hold answers ErrTooManyRows rather than reading the
    workspace.
  - idSrcPage.sortGoSide establishes the order before finishWindow trims. This
    is the union seam's copy of issueops.sortMergedResults and exists for the
    identical reason that one gives: a concatenation trimmed without a re-sort
    keeps an arbitrary prefix. The id is already on the ref, so no row has to be
    hydrated to be placed.

THE LIVE CALLER IS `bd query`, NOT `bd list`, and the review's note undersized
it. `bd list` is protected by workapi.SQLLimit and the wire's list endpoint pins
"created" — but workapi.BuildQueryPlan pushes SortBy and Limit straight onto the
filter for any filter-expressible query with no such guard, DefaultQueryLimit
means a limit is ALWAYS set, and "id" is in both the CLI's and the HTTP
endpoint's accepted sort keys. So `bd query '<expr>' --sort id` and
GET /v0/beads/issues/query?sort=id reached this on the DEFAULT path, on a wire
the review recorded as safe. BuildQueryPlan's own comment justifies the pushdown
by promising "the page is the first N in the requested order on every backend";
that was true of every key except the one IsGoSideSort names. Both twins are
covered — the counts twin is the one those callers reach.

TWO THINGS THIS DOES NOT FIX, named rather than left for the next reader.
The per-table seam (internal/storage/issueops) bounds each leg with
EffectiveSearchLimit under the same absent ORDER BY, so it picks an arbitrary
subset per leg before sortMergedResults runs; and sqlbuild.Less ignores sortDesc
for "id", so a reversed id page there keeps the byte-FIRST rows. sortGoSide
honors sortDesc rather than mirroring that.

Costs, stated. A caller combining `--sort id` with a small limit and no cap now
reads every matching row where it used to read limit+1. It read the wrong ones,
and the reader role has always paid this price on the same query.

Also folded in, both comment-only and both named by the review: count.go cited
`bd doctor --check=cross-table`, a selector that does not exist (the flag takes
artifacts, conventions, pollution and validate; the cross-table check runs in
the default sweep), and scanIDSrcPage claimed the shadowed durable row is
dropped "wherever it sits" when the drop only ever reached the scanned window —
a distinction this change sharpens, since the legs are now bounded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Review Sim <ci@beads.test>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 17:07:59 -07:00
Steve Yegge
80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00
Steve Yegge
9291864c8c
feat(export): port bd export to proxied-server mode (bd-o58fz) (#5351)
* feat(export): port bd export to proxied-server mode (bd-o58fz)

Remove the proxied-server gate from bd export and route the command's
storage reads through an exportSource seam:

- storeExportSource preserves the classic (embedded/direct/server) call
  pattern over the store global, verbatim.
- uowExportSource reads through the proxied unit of work's domain use
  cases, with the whole read set inside ONE uow.RunTxRead transaction
  (read-only, never committed) so the export is a consistent snapshot.
  Plane-pinned use cases are re-partitioned to match the classic bulk
  loaders: labels/comments partition by the row's own Ephemeral/NoHistory
  flags; dependency records already partition internally; dependency
  counts sum both planes for every id, as classic does.

Everything downstream of the reads (filtering, sanitizeZeroTime, record
shaping, marshal order, atomicfile, stderr summary) is shared code, which
is what holds the byte-identical cross-mode output bar. Ordering parity
holds because both stacks render ORDER BY from the same sqlbuild.SortDefs
(priority ASC, created_at DESC, id ASC; labels by issue_id,label;
comments by created_at,id; memories sort.Strings).

TestProxiedServerExport builds a rich fixture through the proxied CLI and
asserts flag semantics plus the acceptance oracle: proxied export,
imported into a fresh classic embedded workspace and re-exported, is
byte-identical (default shape and --include-memories shape).

buildOwnerExcludeSet's export_auto.go call site is adapted to the new
seam signature (classic source over the same store global); auto-export
behavior is unchanged and its proxied port stays a separate bead.
export_source.go joins the bulk-movement forbidigo family in
.golangci.yml (it carries export.go's filter, it does not build one).

The import half is NOT in this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018YmpYVCa5eN6JtG26dJLMs

* fix(export): classify relation planes by table membership, not row flags (bd-o58fz)

Review items on #5351:

1. LoadExportRelations queried the wisp plane for any row carrying
   Ephemeral or NoHistory — but a promoted no-history wisp is a durable
   issues-table row still flagged NoHistory (promote clears only
   Ephemeral), so its durable-plane labels/comments were silently
   dropped. Both planes are now queried with the full ID set and merged
   (ID-disjointness per the GH#4455 cross-table guard), matching what
   the dep-counts leg already did; wisp-plane reads tolerate
   IsTableNotExist like that leg too.

2. The byte-identity oracle is blind to symmetric omissions, so the
   suite now asserts seeded content directly (labels, comment texts,
   comment_count, dependencies, blocks-only dep counts) and adds a
   promoted no-history fixture row in its own project. That shape stays
   out of the round-trip fixture deliberately: classic IMPORT routes by
   the same flag heuristic (issueops.IsWisp) and re-planes the row into
   the wisps table, a pre-existing export/import infidelity filed as
   bd-r9uce.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NvU6bUda8oQjvG3zoa4uyX

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 07:37:59 -07:00