* fix(list): expose comment bodies and mark the omission in bd list --json (be-73x)
`bd list --json` reported comment_count and carried no comment text and no sign
that any was missing. The consequence is not a missing field: a content search
over a listing answers with a well-formed, NON-EMPTY result set whose matching
rows are absent, and nothing about that answer invites a second look. An empty
result invites suspicion; eight rows do not.
Two halves, and the second is the one that fixes what was actually reported:
1. --include-comments on `bd list` populates each row's comments with the
full bodies, the list twin of `bd show --include-comments`. Opt-in: it
costs one query per row that has comments.
2. Every row with a nonzero comment_count whose bodies were NOT hydrated now
carries comments_omitted=true, by default, with no flag. A flag nobody
knows to pass does not fix a silent hole; a marker on the row does.
The bodies ride on types.Issue.Comments, which already carried the `comments`
key for export/import and was simply never populated on this path, so the only
new field is the marker.
Hydration is one shared body (workapi.HydrateListComments) called from BOTH
issueops.Reader.List implementations, the store-backed and the unit-of-work
one. Those are separate bodies that have answered one contract method
differently before; it runs in the page epilogue beside FinishPageAt, after the
trim, so a page never pays for rows it is about to drop.
The source parameter is a CONSTRUCTOR rather than a source, and that is
load-bearing. Building the unit-of-work comment source eagerly made a
capability only the opt-in path uses into a requirement every caller had to
satisfy, and turned `GET /v0/beads/issues` into a 500 on a provider without a
comment use case — on requests that had asked for no comments. A count-only
listing now constructs nothing, pinned by a test that fails if the constructor
is called.
Tests pin each implementation separately rather than duplicating: unit tests
over a fake source, an embedded-dolt test over the real CLI on the direct
route, and a proxied-server test over the real CLI on the unit-of-work route.
Sabotage-controlled three ways — no-op the shared body and all three suites go
red; unwire either reader alone and only that reader's CLI test fails, which is
what proves neither integration test is redundant.
Not addressed, deliberately: gt-mfp5 (bd comment does not advance updated_at,
the orthogonal recency axis) and be-ozp (bd ready --json carries no marker; it
does not route through Reader.Ready).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XK4B7N2CrgfKGEQvDTPM4D
Executed-By: beadsrig/polecats/dag
* fix(list): do not read a skipped comment count as proof of no comments (be-73x)
Codex review of the parent commit, correctly.
`IncludeComments: true` combined with `SkipCounts: true` returned a page with
no comment bodies at all — no error, no marker, nothing — while
`IncludeComments` documents a guarantee to populate them. The hydration loop
skipped any row whose CommentCount was zero as an optimisation, and SkipCounts
zeroes every count: `issueops.ListRequest.SkipCounts` says in its own doc that
a zero then means UNKNOWN rather than none.
That is this bead's own failure mode reintroduced one layer down by the fix for
it: a silent, well-formed, empty answer to a request that asked for content.
I had considered the interaction and resolved it wrongly, in a comment reading
"the count is authoritative here because the JSON route never sets SkipCounts".
True of today's CLI callers, and not a property of the contract — issueops
.ListRequest is a PUBLIC request type and permits both fields at once. A
comment saying "X cannot happen because of how callers behave today" on a
public type is a bug with a justification attached, and worse than no comment,
because it tells the next reader the case was checked.
countsKnown is now a parameter rather than an assumption; call sites pass
!req.SkipCounts.
countsKnown true a zero count proves a row has nothing; skip its query.
countsKnown false query every row, because nothing proves otherwise, and
mark no row omitted, because comments_omitted asserts a
nonzero count the page cannot support.
The second half is the mirror defect on the marker path — the same conflation,
the other direction — which the review did not name.
No behaviour change on any current CLI path: the JSON route never sets
SkipCounts, so it passes countsKnown true, and the text route asks for no
comments and marked nothing before either. Verified: `bd list --json` output
identical, and text output identical with and without the flag.
Regression tests HonorsIncludeWhenCountsAreSkipped and
MarksNothingWhenCountsAreSkipped. Sabotage-controlled: restoring the
count-authoritative skip fails the first with "issued 0 comment reads, want 2".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XK4B7N2CrgfKGEQvDTPM4D
Executed-By: beadsrig/polecats/dag
* fix(list): resolve the comment plane by residence, not by the row's flags (be-73x)
Maintainer review fixup for #6473, iteration 1. Both review lanes reached the
same major independently and the synthesizer re-verified it against the tree.
The new list hydration derived the comment table from the row's own
Ephemeral/NoHistory flags - issueops.IsWisp's rule, which is right for routing a
record being WRITTEN and wrong on a read. `bd import` pins a `no_history` record
to the DURABLE table while preserving the flag on the row, because clearing it
would change the content hash and break export->import->export byte-identity,
and every clone materializes its database by importing JSONL. So a promoted
no-history bead that has been through a round-trip is a durable row whose flags
say wisp:
bd list --json --include-comments read wisp_comments, found nothing, and
returned comment_count=N beside an empty
`comments`, with no marker and no error
bd comment, bd show write and read that same row's comments on
the issues plane, by found-table
That is be-73x's own failure mode reintroduced under the flag whose contract is
"gets them or gets an error", and a cross-role divergence: `bd list` and
`bd show` disagreed about one row. The mirror case is real too - an import
record whose `wisp_plane` key pins it to the wisps table with neither flag set
reads back as durable - so no flag-based rule gets both directions right.
The plane now comes from the seam that knows it. HydrateListComments takes a
CommentPlanes lookup instead of inferring; the unit-of-work reader answers it
from the wisps TABLE (IssueUseCase.GetWispsByIDs, the call edgeReader.ReadEdges
already makes for the same question), once per page, inside the page's own read
transaction. The store-backed seam passes SourceRoutesCommentPlanes because its
source routes ids itself (DoltStore.GetIssueComments picks the table from
isActiveWisp) and would buy a lookup for an argument it ignores. isWispPlane is
deleted: nothing on this path infers a plane any more. A nil lookup is an error
rather than a silent "durable", so a third seam has to answer the question
rather than inherit the default this bug was.
A MISSING WISPS TABLE STAYS "NO WISPS" rather than a failed listing, classified
through the same two helpers the counts query that produced the page uses
(dberrors.MissingTableName + sqlbuild.OptionalWispTable). Any other failure of
the lookup fails the call: a guessed plane answers with the wrong table's
contents and nothing in the response shows it.
Tests. The unit table now asserts residence AGAINST the flags, with each row's
comments present only on the plane it lives on, so a wrong-table read shows up
as missing TEXT and not merely as a wrong flag in a recorded call. Restoring the
flag inference reds exactly the two rows where flags and table disagree
(durable-carrying-no_history, wisp-table-carrying-no-flags) and leaves the two
agreeing rows green. The old `override_wins_over_flags` case is gone:
WispPlaneOverride is `json:"-"` and set only by import, in memory, so no scanned
row can carry it and that branch was unreachable on this path - which is what
made the suite look sufficient (Codex's separate minor, folded in here as the
same root cause). New seam-level tests drive the real uow Reader.List over both
shapes, and pin that a listing without the flag resolves no planes at all, that
a missing wisps table is tolerated, and that any other lookup failure
propagates; emptying the residence answer reds the wisp-row leg.
Also from the review, non-gating:
- types.IssueWithCounts.CommentsOmitted and the OpenAPI property stated the
marker rule unconditionally on a type Reader.Ready and the claim response also
return, neither of which calls the epilogue. Both now scope the rule to the
List surfaces and cite be-ozp, which is where the parent commit deliberately
left the ready surfaces. (`bd ready --json` does not route through
Reader.Ready; GET /v0/beads/ready and ClaimNextResponse.claimed do.)
- GET /v0/beads/issues gained no `include_comments` parameter, so its
description now says so and points at the detail read, in the idiom the
`revision` paragraph above it already uses.
- One internal/httpapi list test for the new wire field: the marker present for
a commented row, absent for an uncommented one. It was asserted only by the
spec file; the detail twin has get_issue_test.go. Neutering
markCommentsOmitted reds it.
Validation: go build ./...; go vet over the changed packages and again with
CGO_ENABLED=0; make api-check clean, so the spec edits oblige no regeneration;
go test ./internal/workapi/... ./issueops/... ./internal/types/;
TestBothReaderImplementationsAgreeOnTheListEpilogue plus the new plane tests in
./internal/storage/uow/; BEADS_TEST_PROXIED_SERVER=1
TestProxiedServerListComments 3 PASS and BEADS_TEST_EMBEDDED_DOLT=1
TestEmbeddedListComments 4 PASS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
158 lines
6.2 KiB
Go
158 lines
6.2 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"os"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
// TestEmbeddedListComments drives `bd list --json` end to end, which is the
|
|
// entry point be-73x is about: the town-wide duplicate/prior-art search this
|
|
// repo's consumers run is a `bd list --json` piped to jq, and until this
|
|
// change it could not see a comment body at all.
|
|
//
|
|
// IT RUNS THE REAL COMMAND ON PURPOSE. internal/workapi's unit tests cover the
|
|
// same hydration against a fake source with rows the test builds itself, and a
|
|
// fixture a test constructs proves what the test constructed. The rows here
|
|
// are produced by `bd create` and `bd comment` and read back by `bd list`, so
|
|
// a hydration wired to the wrong reader, a flag that never reaches the
|
|
// request, or a marshaling tag that drops the field all fail here and none of
|
|
// them fail upstairs.
|
|
func TestEmbeddedListComments(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "lc")
|
|
|
|
// The phrase lives ONLY in a comment. That is the whole point: title,
|
|
// description and notes are the fields a listing already carried, so a
|
|
// term found in any of them would pass with or without this change.
|
|
const commentOnlyPhrase = "zzzcommentonlyphrasezzz"
|
|
|
|
commented := bdCreate(t, bd, dir, "Row with comments", "--type", "task")
|
|
bare := bdCreate(t, bd, dir, "Row with no comments", "--type", "task")
|
|
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "comment", commented.ID, "ROOT CAUSE: "+commentOnlyPhrase); err != nil {
|
|
t.Fatalf("bd comment: %v\n%s", err, out)
|
|
}
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "comment", commented.ID, "second comment"); err != nil {
|
|
t.Fatalf("bd comment: %v\n%s", err, out)
|
|
}
|
|
|
|
rowsByID := func(t *testing.T, args ...string) map[string]map[string]any {
|
|
t.Helper()
|
|
out, err := bdRunWithFlockRetry(t, bd, dir, append([]string{"list", "--json", "--status", "all"}, args...)...)
|
|
if err != nil {
|
|
t.Fatalf("bd list --json %s: %v\n%s", strings.Join(args, " "), err, out)
|
|
}
|
|
s := string(out)
|
|
start := strings.Index(s, "[")
|
|
if start < 0 {
|
|
t.Fatalf("no JSON array in list output:\n%s", s)
|
|
}
|
|
var rows []map[string]any
|
|
if err := json.Unmarshal([]byte(s[start:]), &rows); err != nil {
|
|
t.Fatalf("parse list JSON: %v\n%s", err, s[start:])
|
|
}
|
|
byID := make(map[string]map[string]any, len(rows))
|
|
for _, r := range rows {
|
|
id, _ := r["id"].(string)
|
|
byID[id] = r
|
|
}
|
|
// A denominator, so a vacuous pass is visible: every assertion below
|
|
// reads a specific row, and a page that lost the rows would otherwise
|
|
// look like a page that merely lacks the field.
|
|
if len(byID) < 2 {
|
|
t.Fatalf("list returned %d rows, want at least the 2 seeded", len(byID))
|
|
}
|
|
return byID
|
|
}
|
|
|
|
t.Run("default_marks_the_hole_instead_of_hiding_it", func(t *testing.T) {
|
|
byID := rowsByID(t)
|
|
|
|
row := byID[commented.ID]
|
|
if _, present := row["comments"]; present {
|
|
t.Error("default listing carried a comments field; comment bodies are opt-in")
|
|
}
|
|
if omitted, _ := row["comments_omitted"].(bool); !omitted {
|
|
t.Errorf("comments_omitted = %v on a row with comments, want true: an absent comments field must not read as none", row["comments_omitted"])
|
|
}
|
|
if count, _ := row["comment_count"].(float64); count != 2 {
|
|
t.Errorf("comment_count = %v, want 2", row["comment_count"])
|
|
}
|
|
|
|
// The control. A marker on every row would carry no information.
|
|
if _, present := byID[bare.ID]["comments_omitted"]; present {
|
|
t.Errorf("comments_omitted present on a row with no comments: a true empty stays plain omission")
|
|
}
|
|
})
|
|
|
|
t.Run("include_comments_makes_the_bodies_searchable", func(t *testing.T) {
|
|
byID := rowsByID(t, "--include-comments")
|
|
|
|
row := byID[commented.ID]
|
|
comments, _ := row["comments"].([]any)
|
|
if len(comments) != 2 {
|
|
t.Fatalf("hydrated %d comments, want 2: %v", len(comments), row["comments"])
|
|
}
|
|
// Assert on the TEXT, not the length. The defect was that the bodies
|
|
// were unreachable, and a count cannot tell a populated slice from a
|
|
// slice of empty ones.
|
|
var joined strings.Builder
|
|
for _, c := range comments {
|
|
m, _ := c.(map[string]any)
|
|
text, _ := m["text"].(string)
|
|
joined.WriteString(text)
|
|
}
|
|
if !strings.Contains(joined.String(), commentOnlyPhrase) {
|
|
t.Errorf("hydrated comment text %q does not contain %q", joined.String(), commentOnlyPhrase)
|
|
}
|
|
if _, present := row["comments_omitted"]; present {
|
|
t.Error("comments_omitted present beside populated comments: the two fields must read as one answer")
|
|
}
|
|
if _, present := byID[bare.ID]["comments"]; present {
|
|
t.Error("a row with no comments carried a comments field")
|
|
}
|
|
})
|
|
|
|
// The acceptance test as an agent actually runs it: grep the serialized
|
|
// page for a phrase that exists only in a comment. This is the assertion
|
|
// that fails on today's binary, and it fails with a plausible non-zero
|
|
// hit count elsewhere in the page rather than with an error, which is why
|
|
// it is worth pinning rather than trusting the field checks above.
|
|
t.Run("a_content_search_over_the_page_finds_a_comment_only_phrase", func(t *testing.T) {
|
|
withoutFlag, err := bdRunWithFlockRetry(t, bd, dir, "list", "--json", "--status", "all")
|
|
if err != nil {
|
|
t.Fatalf("bd list --json: %v\n%s", err, withoutFlag)
|
|
}
|
|
if strings.Contains(string(withoutFlag), commentOnlyPhrase) {
|
|
t.Error("default listing leaked comment text; bodies are supposed to be opt-in")
|
|
}
|
|
|
|
withFlag, err := bdRunWithFlockRetry(t, bd, dir, "list", "--json", "--status", "all", "--include-comments")
|
|
if err != nil {
|
|
t.Fatalf("bd list --json --include-comments: %v\n%s", err, withFlag)
|
|
}
|
|
if !strings.Contains(string(withFlag), commentOnlyPhrase) {
|
|
t.Errorf("a search over the whole page missed a phrase that exists only in a comment; this is be-73x")
|
|
}
|
|
})
|
|
|
|
t.Run("text_output_is_untouched", func(t *testing.T) {
|
|
plain := bdList(t, bd, dir, "--status", "all")
|
|
withFlag := bdList(t, bd, dir, "--status", "all", "--include-comments")
|
|
if plain != withFlag {
|
|
t.Errorf("--include-comments changed text output.\nwithout:\n%s\nwith:\n%s", plain, withFlag)
|
|
}
|
|
if strings.Contains(withFlag, commentOnlyPhrase) {
|
|
t.Error("text listing printed a comment body")
|
|
}
|
|
})
|
|
}
|