* perf(storage/db): batch the bulk IN-list readers at 200 ids; export narrows the wisp-plane reads bd export on a 19,413-issue rig in proxied-server (UOW) mode took 264-287s. The cmd layer is bulk, but uowExportSource.LoadExportRelations handed ALL ids to the domain use cases, and the domain/db readers built ONE IN (...) list per statement: 19,409 placeholders each for labels, wisp_labels, wisp_comments rows and counts, dependencies counts and wisp_dependencies counts, at 24-60s a statement (~240s of the run). Dolt is super-linear in IN-list size, prepared or literal alike: on the same server the labels rows took 31s as one statement and 0.80s as 98 batches of 200; a 200-id batch is 3-7ms. The wisp readers were also handed every durable id (rows=0 back). - internal/storage/domain/db/batching.go: forEachIDBatch at queryBatchSize (200, mirroring issueops). commentSQLRepositoryImpl.CountsByIssueIDs / ListByIssueIDs, labelSQLRepositoryImpl.ListByIssueIDs, dependencySQLRepositoryImpl.ListByIssueIDs / CountsByIssueIDs and issueSQLRepositoryImpl.GetByIDs loop over batches and merge; an id lands in exactly one batch, so per-id groups and their ORDER BY survive. Error strings are byte-identical; empty input still issues no statement. The three hand-rolled loops (getLabelsFromTable, getDependencyRecordsFromTable, fetchCountsByIDs) collapse onto the helper. - cmd/bd/export_source.go: LoadExportRelations reads wisps-table membership once (WispPlaneIDs, now batched through GetByIDs) and sends only wisp ids to GetLabelsForWisps / GetCommentsForWisps / GetWispCommentCounts; CountsByWispIDs keeps every id because its inbound leg counts wisp->durable edges under the durable target. Membership that could not be read fails OPEN to every id (wispReaderIDs), never to "no wisps". - WispPlaneIDs now tolerates only a missing WISPS table (dberrors.IsMissingTable), not any table-not-exist: its read joins leases, and the blanket check reported a broken database as an empty wisp plane (the issue_search.go missingOptionalWispTable lesson). Behavior change on purpose: a rig missing leases now fails bd export with "failed to classify wisp-plane rows" instead of silently exporting without markers. Measured in the same clone against the same shared server, 19,433 rows: stock 264s / 266s / 287s -> batched 16.9s / 17.8s. Stock-vs-batched byte oracle: 19,363 of 19,433 records byte-identical; every differing record is a live lease/heartbeat field or an updated_at inside the seven minutes between the two runs. Tests: internal/storage/domain/db TestBulkReadersBatch (a recording Runner over the real Dolt suite: ceil(N/200) statements per leg, <=200 id placeholders each, merge across batch boundaries, the Types filter's extra placeholder, every reader routed to the wisp tables, GetByIDs, empty input), cmd/bd TestWispPlaneSubset + TestWispReaderIDs, and the proxied export integration fixture's wisp now carries a label and a comment that the --all export must emit. Follow-up (not here): fetchIssuesByIDs (SearchIssues Limit=0), GetBlockingInfo and loadStatusByID still build one unbounded IN list, and issue_delete.go's two GetByIDs(wisps) calls keep the blanket IsTableNotExist. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RDjKMk7ubzrgpEDj2JRAbu * perf(storage/db): batch fetchIssuesByIDs, GetBlockingInfo and loadStatusByID at 200 ids; delete tolerates only a missing wisps table (wy-sm01o2) Follow-up to #6267 (wy-237yfi), whose own body names these three readers as the residual unbounded IN lists in internal/storage/domain/db. Dolt's cost per IN-list placeholder is super-linear, so one statement over N ids is far slower than ceil(N/200) statements over 200 each. - issue_search.go fetchIssuesByIDs: one SELECT per <=200-id batch, merged into the same map. `ordered` is the scan order concatenated across batches and hydration runs ONCE over the merged slice (hydrateIssues batches its own relation reads since #6267), so callers get the same map and the same hydration for one round trip per 200 ids instead of one 19k-placeholder statement (export's SearchIssues with Limit=0 reaches it with ~19k ids). The signature and the missing-wisp-table classification are unchanged. - dependency.go GetBlockingInfo: both legs batch through forEachIDBatch and their blockingRows are appended across batches; the merge that follows is untouched. Each leg keys its result by a column the IN list constrains (outbound issue_id, inbound the dependency target), and an id lands in exactly one batch, so every row sharing a key comes from the same batch. loadStatusByID batches its id list per table; the cross-plane duplicate-id error is unchanged, because an id is still recorded under `issues` before the `wisps` pass reaches it. GetBlockingInfoAcrossIssuesAndWisps needs no change: it only calls GetBlockingInfo twice and merges the maps. - issue_delete.go: previewDelete's and collectConnectedIssues' two GetByIDs(UseWispsTable) calls read `FROM wisps LEFT JOIN leases`, so their blanket dberrors.IsTableNotExist reported a rig missing `leases` as a rig with no wisp plane (previewDelete listed live wisps under NotFound; hydrateNeighbors returned them unhydrated). Both now tolerate only dberrors.IsMissingTable(err, "wisps"), the fix #6267 applied to WispPlaneIDs. The other IsTableNotExist sites in that file are left alone: they guard reads whose FROM is the optional table itself (wisp_dependencies via ListByIssueIDs(UseWispsTable)). Tests: internal/storage/domain/db TestBulkReadersBatch gains FetchIssuesByIDs (ceil(N/200) statements, <=200 placeholders, rows on both sides of a boundary merged, and exactly ONE label hydration over the merged slice), GetBlockingInfoOutbound, GetBlockingInfoInbound, LoadStatusByID (both planes, one statement per batch each) and LoadStatusByIDDuplicateAcrossTables; the empty-input subtest now covers all three readers issuing no statement. internal/storage/domain/issue_delete_missing_table_test.go proves a missing `leases` (or `wisp_labels`) is no longer swallowed at either GetByIDs(wisps) site while a missing `wisps` table still is, with an unrelated-error control. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(storage/db): consume the aborted duplicate-plane run's statements inside its own subtest (wy-sm01o2) Salvaged WIP from the quota-killed executor (session-limit at 00:19Z): the duplicate-plane loadStatusByID call aborts mid-run, so reset the recorder before it and assert its statements under the cap in place, rather than leaving them to pollute the next subtest. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rrz4umA2L5QU8s9EmCBPw8 * docs(cmd/bd): correct the LoadExportRelations plane-partition doc block The header still documented pre-PR full-set wisp reads ("both planes are queried with the FULL ID set and merged"), which this PR made false for the wisp side. That block is the load-bearing record of the plane-partition invariant, so a future editor trusting it could "restore" full-set wisp reads and silently undo the wy-237yfi fix. - The first bullet now describes the partitioned reads: membership is read from the wisps table itself (WispPlaneIDs) rather than inferred from row flags, the durable plane is queried with the full id set, the wisp plane only with that membership subset (wispReaderIDs), and unreadable membership fails OPEN to the full list. The "row flags are NOT a substitute" rationale is kept deliberately: it is *why* membership is read from the table at all. - The IsTableNotExist sentence is extended, not replaced. It is still true of the three single-table wisp-relation readers, which correctly keep the blanket check; only the lease-joined membership probe (WispPlaneIDs) is pinned to a missing `wisps` table. Replacing the sentence outright would have introduced a new inaccuracy. Maintainer review fixup for adopt-pr #6444. Documentation only: the tree is AST-identical after stripping comments, so no behavior can change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
533 lines
21 KiB
Go
533 lines
21 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"os"
|
|
"path/filepath"
|
|
"strconv"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
// exportProxiedFixture is the seeded corpus for TestProxiedServerExport. It
|
|
// deliberately spans what the export must carry (mirroring the protocol
|
|
// interchange fixture): priorities incl. P0, several types, a closed issue
|
|
// with a reason, labels, comments, both dependency orientations
|
|
// (parent-child and blocks), metadata, an assignee, an ephemeral wisp that
|
|
// default export must EXCLUDE, an infra-type (message) bead, a promoted
|
|
// no-history wisp (durable issues-table row still carrying no_history=true —
|
|
// the bd-r9uce plane-classification trap, with relations), and memories.
|
|
type exportProxiedFixture struct {
|
|
critical string
|
|
epic string
|
|
child string
|
|
blocker string
|
|
done string
|
|
wisp string
|
|
infraMsg string
|
|
// promoted is the promoted no-history wisp: a DURABLE issues-table row
|
|
// whose no_history flag is still set (wild pre-fix promote shape).
|
|
// promotedFriend is durable and blocked by it, so the round-trip oracle
|
|
// also pins that the relation survives — pre-fix, classic import
|
|
// re-planed the row into the wisps table by its flags and dropped this
|
|
// edge as a cross-bucket dependency.
|
|
promoted string
|
|
promotedFriend string
|
|
}
|
|
|
|
func seedProxiedExportFixture(t *testing.T, bd string, p proxiedProject) exportProxiedFixture {
|
|
t.Helper()
|
|
var fx exportProxiedFixture
|
|
|
|
fx.critical = bdProxiedCreateSilent(t, bd, p.dir, "P0 critical bug",
|
|
"--type", "bug", "--priority", "0",
|
|
"--description", "Line one\nline two — unicode: ✓",
|
|
"--assignee", "alice",
|
|
"--design", "Design body",
|
|
"--acceptance", "All green",
|
|
"--notes", "Notes body")
|
|
// Metadata keys are deliberately the SAME length. JSON object key order
|
|
// is canonicalized differently by the two stacks — the proxied update
|
|
// path re-marshals metadata through a Go map (alphabetical key order),
|
|
// while the embedded engine normalizes its JSON column to MySQL
|
|
// binary-JSON order (key length first, then bytes) on read-back — and the
|
|
// two orders coincide exactly when all keys share one length. This is a
|
|
// pre-existing write-path/engine divergence visible to every read command
|
|
// (`bd show --json` shows it too), not an export behavior; with
|
|
// mixed-length keys the cross-mode byte-compare below would fail on it.
|
|
bdProxiedUpdateOne(t, bd, p.dir, fx.critical, "--metadata", `{"risk":"high","comp":"auth"}`)
|
|
|
|
fx.epic = bdProxiedCreateSilent(t, bd, p.dir, "Epic parent", "--type", "epic", "--priority", "1")
|
|
fx.child = bdProxiedCreateSilent(t, bd, p.dir, "Child task", "--type", "task", "--priority", "2", "--parent", fx.epic)
|
|
fx.blocker = bdProxiedCreateSilent(t, bd, p.dir, "Blocker", "--type", "chore", "--priority", "3")
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "dep", "add", fx.child, fx.blocker); err != nil {
|
|
t.Fatalf("dep add: %v\n%s", err, out)
|
|
}
|
|
|
|
for _, label := range []string{"zeta", "alpha"} {
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "label", "add", fx.child, label); err != nil {
|
|
t.Fatalf("label add %s: %v\n%s", label, err, out)
|
|
}
|
|
}
|
|
for _, text := range []string{"first comment", "second comment"} {
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "comment", fx.child, text); err != nil {
|
|
t.Fatalf("comment: %v\n%s", err, out)
|
|
}
|
|
}
|
|
|
|
fx.done = bdProxiedCreateSilent(t, bd, p.dir, "Finished feature", "--type", "feature", "--priority", "4")
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "close", fx.done, "--reason", "shipped in abc123"); err != nil {
|
|
t.Fatalf("close: %v\n%s", err, out)
|
|
}
|
|
|
|
// Promoted no-history wisp (bd-r9uce): a durable issues-table row still
|
|
// carrying no_history=true, with durable-plane relations. `bd promote` is
|
|
// not yet ported to proxied mode (bd-27bfd) — and the fixed promote now
|
|
// clears the flag — so produce the wild pre-fix shape directly: durable
|
|
// row, no_history flipped on. In the round trip below, classic import
|
|
// must route it by the (absent) explicit plane marker and keep it — and
|
|
// its inbound edge — in the durable plane.
|
|
fx.promoted = bdProxiedCreateSilent(t, bd, p.dir, "Promoted no-history wisp", "--type", "task", "--priority", "3")
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "label", "add", fx.promoted, "keepme"); err != nil {
|
|
t.Fatalf("label add: %v\n%s", err, out)
|
|
}
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "comment", fx.promoted, "survived promotion"); err != nil {
|
|
t.Fatalf("comment: %v\n%s", err, out)
|
|
}
|
|
fx.promotedFriend = bdProxiedCreateSilent(t, bd, p.dir, "Promoted wisp friend", "--type", "task", "--priority", "3")
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "dep", "add", fx.promotedFriend, fx.promoted); err != nil {
|
|
t.Fatalf("dep add: %v\n%s", err, out)
|
|
}
|
|
db := openProxiedDB(t, p)
|
|
if _, err := db.Exec("UPDATE issues SET no_history = 1 WHERE id = ?", fx.promoted); err != nil {
|
|
t.Fatalf("flip no_history: %v", err)
|
|
}
|
|
|
|
// Ephemeral wisp: excluded from default export, included with --all.
|
|
fx.wisp = bdProxiedCreateSilent(t, bd, p.dir, "Ephemeral heartbeat", "--ephemeral", "--wisp-type", "heartbeat")
|
|
// Relations on the wisp: the --all export must carry them, which is what
|
|
// pins the UOW plane partition (only wisp ids reach the wisp readers,
|
|
// wy-237yfi) against the classic-mode oracle.
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "label", "add", fx.wisp, "wisp-label"); err != nil {
|
|
t.Fatalf("label add on wisp: %v\n%s", err, out)
|
|
}
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "comment", fx.wisp, "wisp comment"); err != nil {
|
|
t.Fatalf("comment on wisp: %v\n%s", err, out)
|
|
}
|
|
|
|
// Infra-typed bead (message): auto-routed to the ephemeral plane and an
|
|
// infra type, so default export must exclude it and --all include it.
|
|
fx.infraMsg = bdProxiedCreateSilent(t, bd, p.dir, "Infra message", "--type", "message")
|
|
|
|
// Memories: `bd remember` is not yet ported to proxied-server mode, so
|
|
// seed the kv.memory.* config rows directly (same table, same rows).
|
|
for key, value := range map[string]string{
|
|
"kv.memory.zebra": "last memory by key",
|
|
"kv.memory.aardvark": "first memory by key",
|
|
} {
|
|
if out, err := bdProxiedRun(t, bd, p.dir, "config", "set", key, value); err != nil {
|
|
t.Fatalf("config set %s: %v\n%s", key, err, out)
|
|
}
|
|
}
|
|
|
|
return fx
|
|
}
|
|
|
|
// exportLines parses a JSONL stream, failing the test on any line that is not
|
|
// a JSON object — which is also the stdout-purity assertion when the stream
|
|
// came from stdout.
|
|
func exportLines(t *testing.T, stream string) []map[string]any {
|
|
t.Helper()
|
|
var out []map[string]any
|
|
for i, line := range strings.Split(stream, "\n") {
|
|
if strings.TrimSpace(line) == "" {
|
|
continue
|
|
}
|
|
var m map[string]any
|
|
if err := json.Unmarshal([]byte(line), &m); err != nil {
|
|
t.Fatalf("line %d is not JSON (%v): %q", i+1, err, line)
|
|
}
|
|
out = append(out, m)
|
|
}
|
|
return out
|
|
}
|
|
|
|
func exportIssueIDs(records []map[string]any) map[string]bool {
|
|
ids := make(map[string]bool)
|
|
for _, r := range records {
|
|
if r["_type"] == "issue" {
|
|
if id, ok := r["id"].(string); ok {
|
|
ids[id] = true
|
|
}
|
|
}
|
|
}
|
|
return ids
|
|
}
|
|
|
|
// exportIssueRecordByID returns the issue record with the given id, or nil.
|
|
func exportIssueRecordByID(records []map[string]any, id string) map[string]any {
|
|
for _, r := range records {
|
|
if r["_type"] == "issue" && r["id"] == id {
|
|
return r
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// exportStrings coerces a JSON array field to []string.
|
|
func exportStrings(v any) []string {
|
|
arr, _ := v.([]any)
|
|
out := make([]string, 0, len(arr))
|
|
for _, e := range arr {
|
|
if s, ok := e.(string); ok {
|
|
out = append(out, s)
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
// exportCommentTexts extracts the text field of each comment object in order.
|
|
func exportCommentTexts(v any) []string {
|
|
arr, _ := v.([]any)
|
|
out := make([]string, 0, len(arr))
|
|
for _, e := range arr {
|
|
if m, ok := e.(map[string]any); ok {
|
|
if s, ok := m["text"].(string); ok {
|
|
out = append(out, s)
|
|
}
|
|
}
|
|
}
|
|
return out
|
|
}
|
|
|
|
func exportMemoryKeys(records []map[string]any) []string {
|
|
var keys []string
|
|
for _, r := range records {
|
|
if r["_type"] == "memory" {
|
|
if k, ok := r["key"].(string); ok {
|
|
keys = append(keys, k)
|
|
}
|
|
}
|
|
}
|
|
return keys
|
|
}
|
|
|
|
// firstStreamDiff renders the first differing line of two JSONL streams
|
|
// (cmd/bd-package analog of protocol's diffStreams, which lives in another
|
|
// test package and cannot be imported).
|
|
func firstStreamDiff(want, got string) string {
|
|
wl := strings.Split(strings.TrimRight(want, "\n"), "\n")
|
|
gl := strings.Split(strings.TrimRight(got, "\n"), "\n")
|
|
var b strings.Builder
|
|
if len(wl) != len(gl) {
|
|
b.WriteString("line count differs: proxied=" + strconv.Itoa(len(wl)) +
|
|
" classic=" + strconv.Itoa(len(gl)) + "\n")
|
|
}
|
|
n := min(len(wl), len(gl))
|
|
for i := range n {
|
|
if wl[i] != gl[i] {
|
|
b.WriteString("first differing line " + strconv.Itoa(i+1) +
|
|
":\n proxied: " + wl[i] + "\n classic: " + gl[i] + "\n")
|
|
return b.String()
|
|
}
|
|
}
|
|
if len(wl) > n {
|
|
b.WriteString("only in proxied: " + wl[n] + "\n")
|
|
}
|
|
if len(gl) > n {
|
|
b.WriteString("only in classic: " + gl[n] + "\n")
|
|
}
|
|
return b.String()
|
|
}
|
|
|
|
// runClassic runs the (embedded-mode) bd binary in dir with the classic env.
|
|
func runClassic(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
out, err := bdRunWithFlockRetry(t, bd, dir, args...)
|
|
if err != nil {
|
|
t.Fatalf("classic bd %s failed: %v\n%s", strings.Join(args, " "), err, out)
|
|
}
|
|
return string(out)
|
|
}
|
|
|
|
func TestProxiedServerExport(t *testing.T) {
|
|
requireSharedProxiedServer(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
p := newSharedProxiedProject(t, bd, "pxe")
|
|
fx := seedProxiedExportFixture(t, bd, p)
|
|
|
|
durable := []string{fx.critical, fx.epic, fx.child, fx.blocker, fx.done}
|
|
|
|
t.Run("default_excludes_wisps_infra_memories", func(t *testing.T) {
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export")
|
|
if err != nil {
|
|
t.Fatalf("bd export: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
records := exportLines(t, stdout) // fails on any non-JSON stdout line
|
|
ids := exportIssueIDs(records)
|
|
for _, id := range durable {
|
|
if !ids[id] {
|
|
t.Errorf("default export missing durable issue %s", id)
|
|
}
|
|
}
|
|
if ids[fx.wisp] {
|
|
t.Errorf("default export must exclude ephemeral wisp %s", fx.wisp)
|
|
}
|
|
if ids[fx.infraMsg] {
|
|
t.Errorf("default export must exclude infra-typed bead %s", fx.infraMsg)
|
|
}
|
|
if keys := exportMemoryKeys(records); len(keys) != 0 {
|
|
t.Errorf("default export must not carry memories, got %v", keys)
|
|
}
|
|
if strings.Contains(stderr, "Exported") {
|
|
t.Errorf("summary must not print without -o; stderr:\n%s", stderr)
|
|
}
|
|
})
|
|
|
|
// Direct content oracle: the relational data must actually APPEAR in the
|
|
// export. The byte-identity round trip below is blind to symmetric
|
|
// omissions — a proxied read bug that drops a whole relation produces an
|
|
// impoverished stream that classic faithfully imports and re-exports, so
|
|
// bytes still match. These assertions pin the seeded content itself.
|
|
t.Run("relations_content_present", func(t *testing.T) {
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export")
|
|
if err != nil {
|
|
t.Fatalf("bd export: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
records := exportLines(t, stdout)
|
|
child := exportIssueRecordByID(records, fx.child)
|
|
if child == nil {
|
|
t.Fatalf("export missing %s", fx.child)
|
|
}
|
|
if labels := exportStrings(child["labels"]); len(labels) != 2 || labels[0] != "alpha" || labels[1] != "zeta" {
|
|
t.Errorf("child labels = %v, want [alpha zeta]", labels)
|
|
}
|
|
if texts := exportCommentTexts(child["comments"]); len(texts) != 2 || texts[0] != "first comment" || texts[1] != "second comment" {
|
|
t.Errorf("child comments = %v, want [first comment, second comment]", texts)
|
|
}
|
|
if cc, _ := child["comment_count"].(float64); int(cc) != 2 {
|
|
t.Errorf("child comment_count = %v, want 2", child["comment_count"])
|
|
}
|
|
deps, _ := child["dependencies"].([]any)
|
|
dependsOn := map[string]bool{}
|
|
for _, d := range deps {
|
|
if m, ok := d.(map[string]any); ok {
|
|
if id, ok := m["depends_on_id"].(string); ok {
|
|
dependsOn[id] = true
|
|
}
|
|
}
|
|
}
|
|
if len(deps) != 2 || !dependsOn[fx.epic] || !dependsOn[fx.blocker] {
|
|
t.Errorf("child dependencies depend on %v, want both %s and %s", dependsOn, fx.epic, fx.blocker)
|
|
}
|
|
// Dependency COUNTS tally only 'blocks' edges (both stacks:
|
|
// domain/db/dependency.go:396, classic GetDependencyCountsInTx), so
|
|
// the parent-child edge appears in `dependencies` but not the count.
|
|
if dc, _ := child["dependency_count"].(float64); int(dc) != 1 {
|
|
t.Errorf("child dependency_count = %v, want 1 (blocks edge only)", child["dependency_count"])
|
|
}
|
|
blocker := exportIssueRecordByID(records, fx.blocker)
|
|
if blocker == nil {
|
|
t.Fatalf("export missing %s", fx.blocker)
|
|
}
|
|
if dc, _ := blocker["dependent_count"].(float64); int(dc) != 1 {
|
|
t.Errorf("blocker dependent_count = %v, want 1", blocker["dependent_count"])
|
|
}
|
|
})
|
|
|
|
// The plane-classification trap from review: a promoted no-history wisp
|
|
// is a durable issues-table row that still carries no_history=true — the
|
|
// wild pre-fix promote shape — and its labels/comments live in the
|
|
// DURABLE plane. Classifying by row flags looks them up in the wisp plane
|
|
// and silently drops them; classification must follow table membership.
|
|
// Since bd-r9uce the shape lives IN the round-trip fixture (fx.promoted):
|
|
// import now routes by the explicit "wisp_plane" marker instead of the
|
|
// flags, so the cross_mode_byte_identical oracle below covers it too.
|
|
// The export half pinned here: relations present, and NO plane marker on
|
|
// the record (it is durable — stamping it would re-plane it on import).
|
|
t.Run("promoted_no_history_relations", func(t *testing.T) {
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export")
|
|
if err != nil {
|
|
t.Fatalf("bd export: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
records := exportLines(t, stdout)
|
|
rec := exportIssueRecordByID(records, fx.promoted)
|
|
if rec == nil {
|
|
t.Fatalf("export missing promoted no-history row %s (durable table membership must include it)", fx.promoted)
|
|
}
|
|
if noHist, _ := rec["no_history"].(bool); !noHist {
|
|
t.Fatalf("promoted row lost no_history in export: %v", rec)
|
|
}
|
|
if _, marked := rec["wisp_plane"]; marked {
|
|
t.Errorf("promoted row carries the wisps-plane marker; it is durable and must not be stamped: %v", rec)
|
|
}
|
|
if labels := exportStrings(rec["labels"]); len(labels) != 1 || labels[0] != "keepme" {
|
|
t.Errorf("promoted labels = %v, want [keepme] (durable-plane labels dropped?)", labels)
|
|
}
|
|
if texts := exportCommentTexts(rec["comments"]); len(texts) != 1 || texts[0] != "survived promotion" {
|
|
t.Errorf("promoted comments = %v, want [survived promotion] (durable-plane comments dropped?)", texts)
|
|
}
|
|
if cc, _ := rec["comment_count"].(float64); int(cc) != 1 {
|
|
t.Errorf("promoted comment_count = %v, want 1", rec["comment_count"])
|
|
}
|
|
if dc, _ := rec["dependent_count"].(float64); int(dc) != 1 {
|
|
t.Errorf("promoted dependent_count = %v, want 1 (inbound edge from friend)", rec["dependent_count"])
|
|
}
|
|
friend := exportIssueRecordByID(records, fx.promotedFriend)
|
|
if friend == nil {
|
|
t.Fatalf("export missing %s", fx.promotedFriend)
|
|
}
|
|
deps, _ := friend["dependencies"].([]any)
|
|
if len(deps) != 1 {
|
|
t.Fatalf("friend dependencies = %v, want the one edge onto the promoted row", friend["dependencies"])
|
|
}
|
|
if edge, ok := deps[0].(map[string]any); !ok || edge["depends_on_id"] != fx.promoted {
|
|
t.Errorf("friend dependency edge = %v, want depends_on_id=%s", deps[0], fx.promoted)
|
|
}
|
|
})
|
|
|
|
t.Run("dash_arg_streams_to_stdout", func(t *testing.T) {
|
|
// `bd export --all -` streams to stdout: the literal `-` is a
|
|
// positional no-op, NOT an output path (and `-o -` would mean a file
|
|
// named "-"). Pinned so the port preserves the quirk.
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export", "--all", "-")
|
|
if err != nil {
|
|
t.Fatalf("bd export --all -: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
if len(exportLines(t, stdout)) == 0 {
|
|
t.Fatal("expected JSONL on stdout")
|
|
}
|
|
})
|
|
|
|
t.Run("all_includes_wisps_infra_memories", func(t *testing.T) {
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export", "--all")
|
|
if err != nil {
|
|
t.Fatalf("bd export --all: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
records := exportLines(t, stdout)
|
|
ids := exportIssueIDs(records)
|
|
for _, id := range append(append([]string{}, durable...), fx.wisp, fx.infraMsg) {
|
|
if !ids[id] {
|
|
t.Errorf("--all export missing %s", id)
|
|
}
|
|
}
|
|
keys := exportMemoryKeys(records)
|
|
if len(keys) != 2 || keys[0] != "aardvark" || keys[1] != "zebra" {
|
|
t.Errorf("--all export memories = %v, want [aardvark zebra] (sorted)", keys)
|
|
}
|
|
// The wisp's own relations ride along: a misclassified plane would
|
|
// drop them silently while the id above still passes.
|
|
for _, rec := range records {
|
|
if rec["id"] != fx.wisp {
|
|
continue
|
|
}
|
|
labels, _ := rec["labels"].([]any)
|
|
if len(labels) != 1 || labels[0] != "wisp-label" {
|
|
t.Errorf("--all export wisp %s labels = %v, want [wisp-label]", fx.wisp, rec["labels"])
|
|
}
|
|
comments, _ := rec["comments"].([]any)
|
|
if len(comments) != 1 {
|
|
t.Errorf("--all export wisp %s comments = %v, want one", fx.wisp, rec["comments"])
|
|
}
|
|
}
|
|
})
|
|
|
|
t.Run("include_memories_after_issues_sorted", func(t *testing.T) {
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export", "--include-memories")
|
|
if err != nil {
|
|
t.Fatalf("bd export --include-memories: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
records := exportLines(t, stdout)
|
|
ids := exportIssueIDs(records)
|
|
if ids[fx.wisp] {
|
|
t.Errorf("--include-memories must not include the ephemeral wisp")
|
|
}
|
|
keys := exportMemoryKeys(records)
|
|
if len(keys) != 2 || keys[0] != "aardvark" || keys[1] != "zebra" {
|
|
t.Errorf("memories = %v, want [aardvark zebra] (sorted by key)", keys)
|
|
}
|
|
// All memory records come AFTER all issue records.
|
|
seenMemory := false
|
|
for _, r := range records {
|
|
switch r["_type"] {
|
|
case "memory":
|
|
seenMemory = true
|
|
case "issue":
|
|
if seenMemory {
|
|
t.Fatalf("issue record after memory record; memories must trail all issues")
|
|
}
|
|
}
|
|
}
|
|
})
|
|
|
|
t.Run("output_file_atomic_summary_on_stderr", func(t *testing.T) {
|
|
path := filepath.Join(t.TempDir(), "export.jsonl")
|
|
stdout, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export", "-o", path, "--include-memories")
|
|
if err != nil {
|
|
t.Fatalf("bd export -o: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
if strings.TrimSpace(stdout) != "" {
|
|
t.Errorf("with -o, stdout must be empty, got:\n%s", stdout)
|
|
}
|
|
if !strings.Contains(stderr, "Exported") || !strings.Contains(stderr, "2 memories") {
|
|
t.Errorf("expected 'Exported N issues and 2 memories' summary on stderr, got:\n%s", stderr)
|
|
}
|
|
if _, err := os.Stat(path); err != nil {
|
|
t.Fatalf("export file not written: %v", err)
|
|
}
|
|
})
|
|
|
|
// The acceptance oracle: the proxied export, imported into a fresh
|
|
// CLASSIC (embedded) workspace and re-exported there, must reproduce the
|
|
// proxied stream byte for byte — same records, same field encodings, same
|
|
// ordering. Mirrors protocol's §J6.6 round-trip, across storage modes.
|
|
t.Run("cross_mode_byte_identical", func(t *testing.T) {
|
|
proxiedPath := filepath.Join(t.TempDir(), "proxied.jsonl")
|
|
if _, stderr, err := bdProxiedRunBuffers(t, bd, p.dir, "export", "-o", proxiedPath, "--include-memories"); err != nil {
|
|
t.Fatalf("proxied export: %v\n%s", err, stderr)
|
|
}
|
|
proxiedBytes, err := os.ReadFile(proxiedPath)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
classicDir, _, _ := bdInit(t, bd, "--prefix", "cls")
|
|
incoming := filepath.Join(classicDir, "incoming.jsonl")
|
|
if err := os.WriteFile(incoming, proxiedBytes, 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
runClassic(t, bd, classicDir, "import", "-i", incoming)
|
|
|
|
classicPath := filepath.Join(classicDir, "classic.jsonl")
|
|
runClassic(t, bd, classicDir, "export", "-o", classicPath, "--include-memories")
|
|
classicBytes, err := os.ReadFile(classicPath)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
if string(proxiedBytes) != string(classicBytes) {
|
|
t.Errorf("proxied and classic exports differ:\n%s",
|
|
firstStreamDiff(string(proxiedBytes), string(classicBytes)))
|
|
}
|
|
|
|
// Same oracle for the default (no-memories) shape both from the
|
|
// proxied store and from the classic store now holding the same data.
|
|
proxiedDefault, _, err := bdProxiedRunBuffers(t, bd, p.dir, "export")
|
|
if err != nil {
|
|
t.Fatalf("proxied default export: %v", err)
|
|
}
|
|
classicDefaultPath := filepath.Join(classicDir, "classic-default.jsonl")
|
|
runClassic(t, bd, classicDir, "export", "-o", classicDefaultPath)
|
|
classicDefault, err := os.ReadFile(classicDefaultPath)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if proxiedDefault != string(classicDefault) {
|
|
t.Errorf("default-shape exports differ across modes:\n%s",
|
|
firstStreamDiff(proxiedDefault, string(classicDefault)))
|
|
}
|
|
})
|
|
}
|