beads/issueops/export_test.go
Julian Knutsen e7ad85bcff
fix(http): bd export over http through issueops.ReadExport (S21) (#7551)
* fix(http): bd export over http through issueops.ReadExport (S21)

bd export read through raw SearchIssues and the bulk label, dependency and
comment reads, which the HTTP backend refuses (B3 #16), so gc's JSONL export
order (gc bd export --all) failed on every remote scope.

issueops.ReadExport composes role reads every deployed server publishes: an
unlimited Reader.List for rows, labels and counts, EdgeReader.ReadEdges for
dependency records, and Reader.Get with comments for rows that have any. The
remote route feeds it into the same runExportFromSource the direct and
proxied routes use. A failed read fails the export. gc-export-all passes
against the reference profile; mc-promote stays pending (S21b) with its exact
reason: it needs a new server capability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(export): read remote memories from the memory plane, pin unmarked no-history rows (bd-o4qlui)

Review fixes for #7551, attempts 1 to 3.

Attempt 3. No test reached roleExportSource.ExportMemories's error
path. A unit test now drives it with a fake store: a memory role that
cannot be reached, and a listMemories that fails, each end the export
with that error and no map. Rewriting either branch to degrade to an
empty map would bring back a remote `bd export --all` that exits 0 with
no memory in it.

Declined in attempt 3, for finalize to file: an unlimited HTTP listing
that stops on a has_more page with no next_cursor returns the rows it
has with HasMore false, so ReadExport's refusal cannot fire over HTTP.
Only a server that breaks the spec's has_more/next_cursor pairing gets
there; this repo's server always sends a cursor with a non-empty
has_more page. The walker (fetchIssuePages) is unchanged since #7247
and shared with bd list, so the fix needs its own PR that checks bd
list's callers first. The remote route also has no single snapshot: a
comment changed between List and Get can make the written count
disagree with the bodies kept. That belongs in one follow-up with the
bounded retry declined in attempt 1, as a count/body consistency check.
The --include-infra e2e leg stays declined. The wire does carry the
knob: the client sends include_infra and include_templates on
listIssues, and the server decodes both. The GetInfraTypes nil-store
guard stays declined, and auto-export's own GetAllConfig memory read
predates this PR.

Attempt 2. A remote export dropped every memory. Over HTTP, `bd export
--all` and `--include-memories` exited 0 without a memory record, where
the base refused with exit 1. The memory lines came from GetAllConfig,
which over HTTP is listSettings, and the server's settings enumeration
withholds every kv.* row, memories included. gc's JSONL export runs
`gc bd export --all`, so its archive lost them without a word.

The exportSource seam now asks each source for its memories
(ExportMemories), not for the whole config table. The local and proxied
sources still take them from their config table. The remote source reads
the served memory plane, store.Memories().List (listMemories), which the
deployed bd 1.3.0 server advertises as memories.list. httpMemories.List
refuses a page that reports has_more: ListResult has no cursor, so a
truncated plane would pass for the whole one. The memory error now says
"failed to read memories"; nothing in beads or gascity matches the old
text.

TestE2E_ExportOverHTTP seeds two memories with `bd remember` over HTTP.
It requires exactly those two in --all, in --include-memories and in the
local re-export of the imported file, and none in the default export. A
unit case pins the has_more refusal. The CHANGELOG entry says where
remote memories come from, so its "Nothing is dropped" holds. The
WispPlaneIDs doc no longer says the listing refuses an id set, which
stops being true once a server advertises issues.list.ids (#7550).

Attempt 1. The remote route's WispPlaneIDs doc and the CHANGELOG had the
effect of the missing wisp_plane marker backwards. They said import then
routes a no-history row by its flags. A current import does the
opposite: it pins an unmarked no-history row to the durable plane
(applyImportWispPlane). So a genuine no-history wisp in a remote export,
which gc's session, wait and order beads are by default, re-imports onto
the Dolt-versioned issues plane, and gc's JSONL archive keeps it.

The route still stamps nothing. No served read publishes plane
membership: listIssues rows carry no plane field, however the listing is
scoped. Inferring the marker from the flags would re-plane a durable row
that still carries the flag and drop its relations (bd-r9uce), and
refusing the export would re-break gc's remote JSONL export. The doc and
the CHANGELOG now state the real consequence, and serving the membership
is bd-1lbvas. TestE2E_ExportOverHTTP pins it. A no-history row is
exported without the marker by --all and by the default export. A fresh
local workspace imports it onto the durable plane, against a locally
created control row that the local route does mark.

Attempt 1 minors and nits: ReadExport refuses a listing page marked
HasMore, as the uow source does, and unit cases pin that and a failed
comment read. The exportSource interface doc names ExportIssues and its
contract. The export error now says "failed to read the export set"
rather than "failed to search issues"; nothing in beads or gascity
matches the old text. roleExportSource.GetInfraTypes says it is unused
on this route.

Declined, for finalize to file: a bounded retry of ReadExport when a row
vanishes between the reads (a design change; today's fatal behavior is
pinned), and e2e rows for pinned, gate, template and infra-typed issues,
including an --include-infra leg over HTTP. Also declined: a nil-store
guard on roleExportSource.GetInfraTypes. The method is dead on this
route, and the source's other store readers (ExportIssues, GetConfig,
ExportMemories) dereference the store unguarded too.

Validation, attempt 3: git diff --check, the go.mod-pinned gofmt, go
vet, gazelle (no change) and the go_srcs check. The new test passed with
no skips. Three mutants were killed. An empty map with no error on a
failed listing, and the same on an unreachable role, each fail their own
subtest. An empty map returned beside the listing error fails the no-map
assertion; export.go discards the map on error today, so that case pins
the interface's "fail rather than return a partial set" contract, not
visible output.

Validation, attempt 2: git diff --check, gofmt and go vet. The
httpclient memory tests passed, 23 of them: 4 units and 19 served. So
did TestE2E_ExportOverHTTP on embedded Dolt with
BEADS_HTTP_TEST_REQUIRED=1 and no skips. Two mutants were killed: a
remote source that reads the settings listing fails 3 e2e assertions,
and dropping the has_more refusal fails the unit case. Export parity
passed with no skips on the classic (memory determinism, byte
stability), embedded and proxied tiers, including the proxied-versus-
classic byte identity with memories. The PR is now behind main, which
added #7549 and #7550, and conflicts with it textually in CHANGELOG.md,
backend/http/BUILD.bazel and issueops/BUILD.bazel. With those resolved
by union, the merge with main e6a891b900 builds and vets. On that merge,
in CI's pinned-shard layout, the served http tier passed 49 of 49,
including #7550's list-scope e2e. The command corpus ran 128 shapes: 113
pass, 3 allowed, 12 pending and 0 fail. gc-export-all passes without its
pending mark, the one shape that moved. The issueops ReadExport and
ListGates tests passed, as did the embedded and proxied export tiers.

Validation, attempt 1: the issueops ReadExport tests, with two mutants
killed; TestE2E_ExportOverHTTP with a flag-stamping WispPlaneIDs mutant
killed on three assertions; the cmd/bd export tests in a scrubbed env.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 03:59:35 +00:00

160 lines
6.5 KiB
Go

package issueops_test
import (
"context"
"errors"
"reflect"
"testing"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/issueops"
)
type exportReader struct {
page []*types.IssueWithCounts
hasMore bool
lists []issueops.ListRequest
gets []issueops.GetRequest
err error
getErr error
}
func (r *exportReader) Ready(context.Context, issueops.ReadyRequest) (issueops.IssuePage, error) {
return issueops.IssuePage{}, errors.New("ready is not export's read")
}
func (r *exportReader) List(_ context.Context, req issueops.ListRequest) (issueops.IssuePage, error) {
r.lists = append(r.lists, req)
if r.err != nil {
return issueops.IssuePage{}, r.err
}
return issueops.IssuePage{Items: r.page, HasMore: r.hasMore}, nil
}
func (r *exportReader) Get(_ context.Context, req issueops.GetRequest) (*issueops.IssueDetails, error) {
r.gets = append(r.gets, req)
if r.getErr != nil {
return nil, r.getErr
}
return &issueops.IssueDetails{Comments: []*types.Comment{{IssueID: req.ID, Text: "c of " + req.ID}}}, nil
}
type exportEdges struct {
calls []issueops.EdgeReadRequest
missing string
}
func (e *exportEdges) ReadEdges(_ context.Context, req issueops.EdgeReadRequest) (issueops.EdgeReadResult, error) {
e.calls = append(e.calls, req)
out := issueops.EdgeReadResult{Anchors: []issueops.AnchorEdges{}}
for _, id := range req.IDs {
a := issueops.AnchorEdges{ID: id, Edges: []*types.Dependency{}, Missing: id == e.missing}
if id == "x-1" {
a.Edges = []*types.Dependency{{IssueID: "x-1", DependsOnID: "x-2", Type: types.DepBlocks}}
}
out.Anchors = append(out.Anchors, a)
}
return out, nil
}
func exportPage() []*types.IssueWithCounts {
return []*types.IssueWithCounts{
{Issue: &types.Issue{ID: "x-1", Labels: []string{"l"}}, DependencyCount: 1, CommentCount: 2},
{Issue: &types.Issue{ID: "x-2"}, DependentCount: 1},
{Issue: &types.Issue{ID: "x-wisp", Ephemeral: true}},
{Issue: &types.Issue{ID: "x-nh", NoHistory: true}},
nil,
}
}
// TestReadExportAllExportsEveryRowThroughRoles is S21: `bd export --all`
// composes served role reads, so a remote backend exports without the raw
// search and bulk relation reads it does not serve.
func TestReadExportAllExportsEveryRowThroughRoles(t *testing.T) {
rd := &exportReader{page: exportPage()}
edges := &exportEdges{}
rows, err := issueops.ReadExport(context.Background(), rd, edges, issueops.ExportRequest{All: true})
if err != nil {
t.Fatalf("ReadExport: %v", err)
}
zero := 0
wantList := issueops.ListRequest{AllFlag: true, IncludeGates: true, IncludeEphemeral: true, IncludeInfra: true, IncludeTemplates: true, Limit: &zero}
if len(rd.lists) != 1 || !reflect.DeepEqual(rd.lists[0], wantList) {
t.Fatalf("List calls = %#v, want one %#v", rd.lists, wantList)
}
var ids []string
for _, r := range rows {
ids = append(ids, r.Issue.ID)
}
if want := []string{"x-1", "x-2", "x-wisp", "x-nh"}; !reflect.DeepEqual(ids, want) {
t.Fatalf("rows = %v, want %v (every row, in list order)", ids, want)
}
if len(edges.calls) != 1 || !reflect.DeepEqual(edges.calls[0].IDs, ids) {
t.Fatalf("ReadEdges calls = %#v, want one over %v", edges.calls, ids)
}
if got := rows[0].Issue.Dependencies; len(got) != 1 || got[0].DependsOnID != "x-2" {
t.Fatalf("x-1 dependencies = %#v, want the stored edge onto x-2", got)
}
if rows[1].Issue.Dependencies != nil {
t.Fatalf("x-2 dependencies = %#v, want none", rows[1].Issue.Dependencies)
}
// Only the row with comments is read for them.
if want := []issueops.GetRequest{{ID: "x-1", IncludeComments: true}}; !reflect.DeepEqual(rd.gets, want) {
t.Fatalf("Get calls = %#v, want %#v", rd.gets, want)
}
if c := rows[0].Issue.Comments; len(c) != 1 || c[0].Text != "c of x-1" {
t.Fatalf("x-1 comments = %#v", c)
}
if rows[0].CommentCount != 2 || rows[0].DependencyCount != 1 || rows[1].DependentCount != 1 {
t.Fatalf("counts lost: %+v %+v", rows[0], rows[1])
}
if !reflect.DeepEqual(rows[0].Issue.Labels, []string{"l"}) {
t.Fatalf("labels lost: %v", rows[0].Issue.Labels)
}
}
// TestReadExportDefaultDropsTrueWispsKeepsNoHistory pins the default scope:
// gates and the wisps plane are read, templates and infra are the listing's to
// exclude, and the true wisps are dropped while no-history rows stay (GH#3649).
func TestReadExportDefaultDropsTrueWispsKeepsNoHistory(t *testing.T) {
for _, infra := range []bool{false, true} {
rd := &exportReader{page: exportPage()}
rows, err := issueops.ReadExport(context.Background(), rd, &exportEdges{}, issueops.ExportRequest{IncludeInfra: infra})
if err != nil {
t.Fatalf("ReadExport: %v", err)
}
zero := 0
wantList := issueops.ListRequest{AllFlag: true, IncludeGates: true, IncludeEphemeral: true, IncludeInfra: infra, Limit: &zero}
if !reflect.DeepEqual(rd.lists[0], wantList) {
t.Fatalf("List = %#v, want %#v", rd.lists[0], wantList)
}
var ids []string
for _, r := range rows {
ids = append(ids, r.Issue.ID)
}
if want := []string{"x-1", "x-2", "x-nh"}; !reflect.DeepEqual(ids, want) {
t.Fatalf("rows = %v, want %v", ids, want)
}
}
}
func TestReadExportNeverShortensOnAFailure(t *testing.T) {
refusal := &issueops.ErrUnsupported{Op: "Reader.List", Backend: "http"}
if rows, err := issueops.ReadExport(context.Background(), &exportReader{err: refusal}, &exportEdges{}, issueops.ExportRequest{All: true}); !errors.Is(err, refusal) || rows != nil {
t.Fatalf("ReadExport = (%v, %v), want (nil, the refusal)", rows, err)
}
if _, err := issueops.ReadExport(context.Background(), &exportReader{page: exportPage()}, &exportEdges{missing: "x-2"}, issueops.ExportRequest{All: true}); err == nil {
t.Fatal("ReadExport with a vanished anchor succeeded, want an error")
}
if rows, err := issueops.ReadExport(context.Background(), &exportReader{page: exportPage(), hasMore: true}, &exportEdges{}, issueops.ExportRequest{All: true}); err == nil || rows != nil {
t.Fatalf("ReadExport over a truncated listing = (%v, %v), want (nil, an error)", rows, err)
}
notFound := errors.New("x-1 not found")
if rows, err := issueops.ReadExport(context.Background(), &exportReader{page: exportPage(), getErr: notFound}, &exportEdges{}, issueops.ExportRequest{All: true}); !errors.Is(err, notFound) || rows != nil {
t.Fatalf("ReadExport with a failed comment read = (%v, %v), want (nil, the failure)", rows, err)
}
var unsupported *issueops.ErrUnsupported
if _, err := issueops.ReadExport(context.Background(), nil, nil, issueops.ExportRequest{}); !errors.As(err, &unsupported) {
t.Fatalf("ReadExport(nil) = %v, want ErrUnsupported", err)
}
}