beads/cmd/bd/import_promoted_wisp_embedded_test.go
Steve Yegge 80af94c0a7
fix(export): promoted no-history wisps survive import round-trip (bd-r9uce) (#5368)
* fix(export): promoted no-history wisps survive import round-trip (bd-r9uce)

A promoted no-history wisp is a durable issues-table row that still
carried no_history=true (PromoteFromEphemeralInTx cleared only
Ephemeral), and both import routes inferred the storage plane from row
flags (issueops.IsWisp: Ephemeral || NoHistory). Importing an export
stream containing such a row silently re-planed it into the wisps table,
dropping its cross-plane relations as cross-bucket dependencies on the
way — classic-mode data loss across export->import->export.

Belt and braces, per triage + the #5351 review endorsement:

1. Promote clears NoHistory too (and normalizes an explicit ephemeral
   StorageClass to empty, the NormalizePersistenceMode promotion rule):
   a promoted wisp is fully durable and the flag has no post-promotion
   meaning.

2. Import routes the plane by an explicit marker, never by no_history.
   Export stamps "wisp": true on rows that live in the WISPS table and
   whose flags alone cannot prove it (the no_history shape; ephemeral
   rows self-describe), classified by table membership via a new
   PartitionWispIDs store capability (classic: batched wisps-membership
   query behind an Unwrap walk through the telemetry/hook decorators;
   proxied: the plane-pinned GetWispsByIDs read). Both parse loops
   (bd import in both storage modes, bootstrap/init/auto-import) resolve
   the plane through one shared helper, applyImportWispPlane, which pins
   marker-less no_history records to the durable plane via a new
   non-serialized Issue.WispPlaneOverride consulted by IsWisp. The
   no_history flag itself is preserved on the row, keeping the
   export->import->export byte-identity for wild data; the legacy
   v0.35-v0.37 "wisp"-as-ephemeral alias behavior is unchanged.

Tests: the promoted shape now lives IN the proxied round-trip fixture
(cross_mode_byte_identical covers it, with relations); a classic-mode
round-trip test (fails on base: cross-bucket dependency drop + missing
plane marker); a proxied import plane-routing subtest; embeddeddolt pins
for promote clearing NoHistory and PartitionWispIDs membership
classification.

* docs(schema): document the export stream's explicit wisp plane marker (bd-r9uce)

* fix(export): fresh wisp_plane marker key — legacy-alias reuse was lossy under rollout skew (bd-r9uce)

Adopts lion's #5368 review recommendation: pre-fix v0.38+ binaries' alias
branch (hasWisp && !Ephemeral => Ephemeral=true) would import a marked
genuine no-history wisp as ephemeral — purge-eligible and excluded from
that rig's next export — making the common skew case lossy. A fresh key
degrades to flag routing on older readers, data-safe in both directions.
Legacy 'wisp' read alias preserved verbatim.

* fix(storage): clone WispPlaneOverride in cloneIssueForHook (bd-r9uce)

The hook snapshot guard (TestCloneIssueForHookCoversReferenceFields)
correctly flagged the new *bool field as uncloned: a hook mutating the
snapshot could alias the live issue's plane pin. Deep-copy it and add
it to the guard's covered-fields manifest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZXjihTayFcngYzo6xPd3p

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 12:48:44 -07:00

172 lines
7.2 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

//go:build cgo
package main
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// TestEmbeddedImportPromotedWispRoundtrip is the classic-mode round-trip
// oracle for bd-r9uce: a promoted no-history wisp — a durable issues-table
// row that (in wild data) still carries no_history=true — must survive
// export→import→export with its relations intact.
//
// Pre-fix, classic import routed the record by row flags (issueops.IsWisp:
// Ephemeral || NoHistory), silently re-planing the durable row into the
// wisps table; its dependency edge to a durable friend in the same batch was
// then dropped as a "cross-bucket dependency", and the row itself left the
// durable plane. Import now routes by the export stream's explicit "wisp_plane"
// plane marker: marker absent ⇒ durable plane, whatever no_history says.
func TestEmbeddedImportPromotedWispRoundtrip(t *testing.T) {
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt import tests")
}
t.Parallel()
bd := buildEmbeddedBD(t)
// recordByID returns the parsed export record for id, or nil.
recordByID := func(t *testing.T, stream, id string) map[string]any {
t.Helper()
for _, rec := range exportLines(t, stream) {
if rec["_type"] == "issue" && rec["id"] == id {
return rec
}
}
return nil
}
t.Run("promoted_shape_routes_durable", func(t *testing.T) {
dir, _, _ := bdInit(t, bd, "--prefix", "rtp")
// The wild promoted shape, as an export stream produced before the
// promote fix: a durable record still carrying no_history=true, NO
// "wisp_plane" marker, plus a durable friend whose dependency edge
// points at it.
wild := `{"_type":"issue","id":"rtp-wisp-promo","title":"Promoted no-history wisp","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-01T10:00:00Z","updated_at":"2026-08-01T10:00:00Z","labels":["keepme"],"comments":[{"id":"11111111-2222-3333-4444-555555555555","issue_id":"rtp-wisp-promo","author":"tester","text":"survive me","created_at":"2026-08-01T10:00:00Z"}],"no_history":true}
{"_type":"issue","id":"rtp-frend","title":"Durable friend","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-01T10:00:01Z","updated_at":"2026-08-01T10:00:01Z","dependencies":[{"issue_id":"rtp-frend","depends_on_id":"rtp-wisp-promo","type":"blocks","created_at":"2026-08-01T10:00:01Z","created_by":"tester"}]}
`
wildPath := filepath.Join(dir, "wild.jsonl")
if err := os.WriteFile(wildPath, []byte(wild), 0o644); err != nil {
t.Fatal(err)
}
importOut := bdImport(t, bd, dir, "-i", wildPath)
if strings.Contains(importOut, "Skipped dependency") {
t.Errorf("import dropped a relation (promoted row re-planed into the wisps bucket?):\n%s", importOut)
}
exportA := bdExport(t, bd, dir)
promo := recordByID(t, exportA, "rtp-wisp-promo")
if promo == nil {
t.Fatal("export missing rtp-wisp-promo")
}
// Durable plane: no "wisp_plane" marker; the stray flag itself is
// preserved (clearing it would change the content hash and break
// export→import→export byte identity for wild rows).
if _, marked := promo["wisp_plane"]; marked {
t.Errorf("promoted row re-exported with the wisps-plane marker; it must stay durable: %v", promo)
}
if noHist, _ := promo["no_history"].(bool); !noHist {
t.Errorf("promoted row lost no_history on the round trip: %v", promo)
}
if dc, _ := promo["dependent_count"].(float64); int(dc) != 1 {
t.Errorf("promoted row dependent_count = %v, want 1 (relation dropped?)", promo["dependent_count"])
}
friend := recordByID(t, exportA, "rtp-frend")
if friend == nil {
t.Fatal("export missing rtp-frend")
}
deps, _ := friend["dependencies"].([]any)
if len(deps) != 1 {
t.Fatalf("friend dependencies = %v, want the one edge onto the promoted row", friend["dependencies"])
}
if edge, ok := deps[0].(map[string]any); !ok || edge["depends_on_id"] != "rtp-wisp-promo" {
t.Errorf("friend dependency edge = %v, want depends_on_id=rtp-wisp-promo", deps[0])
}
// Second leg: the first export imported into a fresh rig re-exports
// byte-identically — the round trip is a fixed point.
dir2, _, _ := bdInit(t, bd, "--prefix", "rtp")
exportAPath := filepath.Join(dir2, "incoming.jsonl")
if err := os.WriteFile(exportAPath, []byte(exportA), 0o644); err != nil {
t.Fatal(err)
}
if out := bdImport(t, bd, dir2, "-i", exportAPath); strings.Contains(out, "Skipped dependency") {
t.Errorf("second-leg import dropped a relation:\n%s", out)
}
if exportB := bdExport(t, bd, dir2); exportB != exportA {
t.Errorf("round trip is not a fixed point:\n%s", firstStreamDiff(exportA, exportB))
}
})
t.Run("no_history_wisp_keeps_plane", func(t *testing.T) {
dir, _, _ := bdInit(t, bd, "--prefix", "rtw")
wispID := bdCreateSilent(t, bd, dir, "Genuine no-history wisp", "--no-history")
exportA := bdExport(t, bd, dir)
rec := recordByID(t, exportA, wispID)
if rec == nil {
t.Fatalf("export missing no-history wisp %s", wispID)
}
// An unpromoted no-history wisp lives in the wisps table, so export
// must stamp the explicit plane marker — flags alone cannot tell it
// apart from the promoted shape above.
if marked, _ := rec["wisp_plane"].(bool); !marked {
t.Fatalf("no-history wisp exported without the wisps-plane marker: %v", rec)
}
dir2, _, _ := bdInit(t, bd, "--prefix", "rtw")
path := filepath.Join(dir2, "incoming.jsonl")
if err := os.WriteFile(path, []byte(exportA), 0o644); err != nil {
t.Fatal(err)
}
bdImport(t, bd, dir2, "-i", path)
exportB := bdExport(t, bd, dir2)
if exportB != exportA {
t.Errorf("no-history wisp round trip is not a fixed point:\n%s", firstStreamDiff(exportA, exportB))
}
rec2 := recordByID(t, exportB, wispID)
if rec2 == nil {
t.Fatalf("re-export missing no-history wisp %s", wispID)
}
if marked, _ := rec2["wisp_plane"].(bool); !marked {
t.Errorf("re-imported no-history wisp lost the wisps-plane marker (re-planed durable?): %v", rec2)
}
})
t.Run("legacy_wisp_alias_still_means_ephemeral", func(t *testing.T) {
dir, _, _ := bdInit(t, bd, "--prefix", "rtl")
// v0.35–v0.37 exports spelled "ephemeral" as "wisp" and predate
// no_history; the marker parse must keep restoring Ephemeral for them.
legacy := `{"_type":"issue","id":"rtl-wisp-leg","title":"Legacy ephemeral","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-01T10:00:00Z","updated_at":"2026-08-01T10:00:00Z","wisp":true}
`
path := filepath.Join(dir, "legacy.jsonl")
if err := os.WriteFile(path, []byte(legacy), 0o644); err != nil {
t.Fatal(err)
}
bdImport(t, bd, dir, "-i", path)
out, err := bdRunWithFlockRetry(t, bd, dir, "show", "rtl-wisp-leg", "--json")
if err != nil {
t.Fatalf("bd show: %v\n%s", err, out)
}
var shown any
if err := json.Unmarshal(out, &shown); err != nil {
t.Fatalf("parse show --json: %v\n%s", err, out)
}
obj, _ := shown.(map[string]any)
if list, ok := shown.([]any); ok && len(list) > 0 {
obj, _ = list[0].(map[string]any)
}
if obj == nil {
t.Fatalf("unexpected show --json shape: %s", out)
}
if eph, _ := obj["ephemeral"].(bool); !eph {
t.Errorf("legacy wisp:true record imported with ephemeral=%v, want true", obj["ephemeral"])
}
})
}