beads/internal/workapi/list_comments.go
mtccressall d14ed67f71
fix(list): expose comment bodies in bd list/show --json (be-73x) (#6473)
* fix(list): expose comment bodies and mark the omission in bd list --json (be-73x)

`bd list --json` reported comment_count and carried no comment text and no sign
that any was missing. The consequence is not a missing field: a content search
over a listing answers with a well-formed, NON-EMPTY result set whose matching
rows are absent, and nothing about that answer invites a second look. An empty
result invites suspicion; eight rows do not.

Two halves, and the second is the one that fixes what was actually reported:

  1. --include-comments on `bd list` populates each row's comments with the
     full bodies, the list twin of `bd show --include-comments`. Opt-in: it
     costs one query per row that has comments.
  2. Every row with a nonzero comment_count whose bodies were NOT hydrated now
     carries comments_omitted=true, by default, with no flag. A flag nobody
     knows to pass does not fix a silent hole; a marker on the row does.

The bodies ride on types.Issue.Comments, which already carried the `comments`
key for export/import and was simply never populated on this path, so the only
new field is the marker.

Hydration is one shared body (workapi.HydrateListComments) called from BOTH
issueops.Reader.List implementations, the store-backed and the unit-of-work
one. Those are separate bodies that have answered one contract method
differently before; it runs in the page epilogue beside FinishPageAt, after the
trim, so a page never pays for rows it is about to drop.

The source parameter is a CONSTRUCTOR rather than a source, and that is
load-bearing. Building the unit-of-work comment source eagerly made a
capability only the opt-in path uses into a requirement every caller had to
satisfy, and turned `GET /v0/beads/issues` into a 500 on a provider without a
comment use case — on requests that had asked for no comments. A count-only
listing now constructs nothing, pinned by a test that fails if the constructor
is called.

Tests pin each implementation separately rather than duplicating: unit tests
over a fake source, an embedded-dolt test over the real CLI on the direct
route, and a proxied-server test over the real CLI on the unit-of-work route.
Sabotage-controlled three ways — no-op the shared body and all three suites go
red; unwire either reader alone and only that reader's CLI test fails, which is
what proves neither integration test is redundant.

Not addressed, deliberately: gt-mfp5 (bd comment does not advance updated_at,
the orthogonal recency axis) and be-ozp (bd ready --json carries no marker; it
does not route through Reader.Ready).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XK4B7N2CrgfKGEQvDTPM4D

Executed-By: beadsrig/polecats/dag

* fix(list): do not read a skipped comment count as proof of no comments (be-73x)

Codex review of the parent commit, correctly.

`IncludeComments: true` combined with `SkipCounts: true` returned a page with
no comment bodies at all — no error, no marker, nothing — while
`IncludeComments` documents a guarantee to populate them. The hydration loop
skipped any row whose CommentCount was zero as an optimisation, and SkipCounts
zeroes every count: `issueops.ListRequest.SkipCounts` says in its own doc that
a zero then means UNKNOWN rather than none.

That is this bead's own failure mode reintroduced one layer down by the fix for
it: a silent, well-formed, empty answer to a request that asked for content.

I had considered the interaction and resolved it wrongly, in a comment reading
"the count is authoritative here because the JSON route never sets SkipCounts".
True of today's CLI callers, and not a property of the contract — issueops
.ListRequest is a PUBLIC request type and permits both fields at once. A
comment saying "X cannot happen because of how callers behave today" on a
public type is a bug with a justification attached, and worse than no comment,
because it tells the next reader the case was checked.

countsKnown is now a parameter rather than an assumption; call sites pass
!req.SkipCounts.

  countsKnown true   a zero count proves a row has nothing; skip its query.
  countsKnown false  query every row, because nothing proves otherwise, and
                     mark no row omitted, because comments_omitted asserts a
                     nonzero count the page cannot support.

The second half is the mirror defect on the marker path — the same conflation,
the other direction — which the review did not name.

No behaviour change on any current CLI path: the JSON route never sets
SkipCounts, so it passes countsKnown true, and the text route asks for no
comments and marked nothing before either. Verified: `bd list --json` output
identical, and text output identical with and without the flag.

Regression tests HonorsIncludeWhenCountsAreSkipped and
MarksNothingWhenCountsAreSkipped. Sabotage-controlled: restoring the
count-authoritative skip fails the first with "issued 0 comment reads, want 2".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XK4B7N2CrgfKGEQvDTPM4D

Executed-By: beadsrig/polecats/dag

* fix(list): resolve the comment plane by residence, not by the row's flags (be-73x)

Maintainer review fixup for #6473, iteration 1. Both review lanes reached the
same major independently and the synthesizer re-verified it against the tree.

The new list hydration derived the comment table from the row's own
Ephemeral/NoHistory flags - issueops.IsWisp's rule, which is right for routing a
record being WRITTEN and wrong on a read. `bd import` pins a `no_history` record
to the DURABLE table while preserving the flag on the row, because clearing it
would change the content hash and break export->import->export byte-identity,
and every clone materializes its database by importing JSONL. So a promoted
no-history bead that has been through a round-trip is a durable row whose flags
say wisp:

  bd list --json --include-comments   read wisp_comments, found nothing, and
                                      returned comment_count=N beside an empty
                                      `comments`, with no marker and no error
  bd comment, bd show                 write and read that same row's comments on
                                      the issues plane, by found-table

That is be-73x's own failure mode reintroduced under the flag whose contract is
"gets them or gets an error", and a cross-role divergence: `bd list` and
`bd show` disagreed about one row. The mirror case is real too - an import
record whose `wisp_plane` key pins it to the wisps table with neither flag set
reads back as durable - so no flag-based rule gets both directions right.

The plane now comes from the seam that knows it. HydrateListComments takes a
CommentPlanes lookup instead of inferring; the unit-of-work reader answers it
from the wisps TABLE (IssueUseCase.GetWispsByIDs, the call edgeReader.ReadEdges
already makes for the same question), once per page, inside the page's own read
transaction. The store-backed seam passes SourceRoutesCommentPlanes because its
source routes ids itself (DoltStore.GetIssueComments picks the table from
isActiveWisp) and would buy a lookup for an argument it ignores. isWispPlane is
deleted: nothing on this path infers a plane any more. A nil lookup is an error
rather than a silent "durable", so a third seam has to answer the question
rather than inherit the default this bug was.

A MISSING WISPS TABLE STAYS "NO WISPS" rather than a failed listing, classified
through the same two helpers the counts query that produced the page uses
(dberrors.MissingTableName + sqlbuild.OptionalWispTable). Any other failure of
the lookup fails the call: a guessed plane answers with the wrong table's
contents and nothing in the response shows it.

Tests. The unit table now asserts residence AGAINST the flags, with each row's
comments present only on the plane it lives on, so a wrong-table read shows up
as missing TEXT and not merely as a wrong flag in a recorded call. Restoring the
flag inference reds exactly the two rows where flags and table disagree
(durable-carrying-no_history, wisp-table-carrying-no-flags) and leaves the two
agreeing rows green. The old `override_wins_over_flags` case is gone:
WispPlaneOverride is `json:"-"` and set only by import, in memory, so no scanned
row can carry it and that branch was unreachable on this path - which is what
made the suite look sufficient (Codex's separate minor, folded in here as the
same root cause). New seam-level tests drive the real uow Reader.List over both
shapes, and pin that a listing without the flag resolves no planes at all, that
a missing wisps table is tolerated, and that any other lookup failure
propagates; emptying the residence answer reds the wisp-row leg.

Also from the review, non-gating:

- types.IssueWithCounts.CommentsOmitted and the OpenAPI property stated the
  marker rule unconditionally on a type Reader.Ready and the claim response also
  return, neither of which calls the epilogue. Both now scope the rule to the
  List surfaces and cite be-ozp, which is where the parent commit deliberately
  left the ready surfaces. (`bd ready --json` does not route through
  Reader.Ready; GET /v0/beads/ready and ClaimNextResponse.claimed do.)
- GET /v0/beads/issues gained no `include_comments` parameter, so its
  description now says so and points at the detail read, in the idiom the
  `revision` paragraph above it already uses.
- One internal/httpapi list test for the new wire field: the marker present for
  a commented row, absent for an uncommented one. It was asserted only by the
  spec file; the detail twin has get_issue_test.go. Neutering
  markCommentsOmitted reds it.

Validation: go build ./...; go vet over the changed packages and again with
CGO_ENABLED=0; make api-check clean, so the spec edits oblige no regeneration;
go test ./internal/workapi/... ./issueops/... ./internal/types/;
TestBothReaderImplementationsAgreeOnTheListEpilogue plus the new plane tests in
./internal/storage/uow/; BEADS_TEST_PROXIED_SERVER=1
TestProxiedServerListComments 3 PASS and BEADS_TEST_EMBEDDED_DOLT=1
TestEmbeddedListComments 4 PASS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-27 02:12:13 -07:00

210 lines
9.8 KiB
Go

package workapi
import (
"context"
"fmt"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// CommentStreamer is the one method list-comment hydration needs off a detail
// source. It is named separately, and taken as the narrower interface, so the
// hydration body cannot quietly grow a second read: DetailSource satisfies it,
// and both of the sources that exist (NewStoreDetailSource, NewUOWDetailSource)
// are therefore usable here without a new adapter.
type CommentStreamer interface {
IterComments(ctx context.Context, id string, isWisp bool) (storage.Iter[types.Comment], error)
}
// CommentPlanes answers, for one page of ids, which of them keep their comments
// on the WISP plane: an id the map reports true for is read from wisp_comments,
// and an id it reports false for or omits from comments.
//
// IT IS A QUERY, NOT A PREDICATE ON THE ROW, because residence is the answer
// and only a query has it. An earlier version of this body derived the plane
// from the row's own Ephemeral/NoHistory flags — issueops.IsWisp's rule, which
// is the right rule for routing a record being WRITTEN — and that inference is
// wrong on a read for a row class the tree deliberately maintains. `bd import`
// pins a `no_history` record to the durable table while PRESERVING the flag on
// the row (cmd/bd/import_shared.go, applyImportWispPlane: clearing it would
// change the content hash and break export→import→export byte-identity, so
// "only the routing is pinned"), and every clone materializes its database by
// importing JSONL. A promoted no-history bead that has been through
// export→import is therefore a durable row whose flags say wisp, and reading
// its comments by those flags queries wisp_comments, finds nothing, and answers
// with a nonzero comment_count, an empty comments array, no marker and no error
// — be-73x's own failure mode, under the flag whose contract is "gets them or
// gets an error".
//
// Every other comment-touching role already resolves residence instead of
// inferring it: GetIssueOrWisp behind both Reader.Get implementations and behind
// uow.commenter.AddComment, DoltStore.GetIssueComments through isActiveWisp,
// issueops.GetCommentCountsInTx through PartitionWispIDsInTx. So `bd comment`
// writes such a row's comments where `bd show` reads them, and a listing that
// went by the flags was the one role that disagreed with the other three about
// the same row. A page is that same question asked about many ids, so it is
// answered the same way, once per page rather than once per row.
type CommentPlanes func(ctx context.Context, ids []string) (map[string]bool, error)
// SourceRoutesCommentPlanes is the CommentPlanes to pass when the comment
// source needs no plane told to it because it resolves each id itself: the
// store-backed source is the one that does (DoltStore.GetIssueComments picks
// the table from isActiveWisp), and it ignores the argument entirely.
//
// It is a NAMED no-op rather than a nil default so that the seam with a routing
// question and the seam without one are told apart at the call site, and so a
// third seam added later has to answer the question rather than inherit
// "durable" by omission — which is exactly how the wrong-table read described
// on CommentPlanes got in.
func SourceRoutesCommentPlanes(context.Context, []string) (map[string]bool, error) {
return nil, nil
}
// HydrateListComments fills in the comment half of one list page.
//
// It is the shared epilogue step FinishPageAt is, and for the identical
// reason: `bd list --json` reaches this through two Reader.List
// implementations — the store-backed one and the unit-of-work one — and a
// hydration written out longhand in each is how the two came apart before.
// One body, called from both, is what makes a CLI listing and an HTTP one
// answer the same request the same way.
//
// TWO OUTCOMES, AND NEITHER IS SILENT.
//
// include = true every row's Issue.Comments is populated, and any row that
// cannot be read fails the whole call. A caller that asked
// for the bodies gets them or gets an error; it never gets a
// short list it has no way to detect.
// include = false every row with a nonzero CommentCount is marked
// CommentsOmitted, so an absent `comments` key means "none"
// on one row and "not asked for" on the other, and the row
// says which.
//
// THE SOURCE IS A CONSTRUCTOR, NOT A SOURCE, AND THAT IS LOAD-BEARING RATHER
// THAN STYLE. Comment hydration is opt-in, so the DEPENDENCY on a comment
// reader must be opt-in with it: building one eagerly on every listing makes a
// capability that only the opt-in path uses into a requirement every caller
// has to satisfy. It is not hypothetical — the unit-of-work source is built
// from four use-case accessors, and constructing it unconditionally turned
// every `GET /v0/beads/issues` on a provider without a comment use case into a
// panic, on a request that had asked for no comments at all. A count-only
// listing now calls nothing.
//
// THE MARKER IS THE POINT, not the hydration (be-73x). A page that silently
// carries no comment text answers a content search with a plausible non-zero
// result whose matching rows are missing, and nothing in that answer invites a
// second look. The count alone does not fix it: comment_count sits on the row
// accurately reporting how much is not there, which reads as reassurance
// rather than as a warning.
//
// A ZERO COUNT IS NOT ALWAYS ZERO COMMENTS, AND countsKnown IS WHAT SEPARATES
// THE TWO. Under issueops.ListRequest.SkipCounts the cardinalities come back
// zero meaning UNKNOWN — the request type says so in as many words — so a body
// that reads CommentCount as authoritative would answer IncludeComments with
// an empty page and no error. countsKnown false therefore means: query every
// row, because there is no count to prove a row has nothing to fetch, and mark
// no row omitted, because the marker asserts a nonzero count this page cannot
// support.
//
// It is a PARAMETER rather than an assumption because issueops.ListRequest is
// PUBLIC and permits both fields at once. An earlier version of this body
// skipped every row on that combination and justified it in a comment reading
// "the JSON route never sets SkipCounts" — true of today's CLI callers, and
// not a property of the contract they were reading. The HTTP surface and any
// future caller may set both.
//
// THE PLANE COMES FROM THE PAGE'S SEAM, NOT FROM THE ROW. Which comment table
// an id's comments live in is a fact about where the row was found, so the
// caller resolves it (CommentPlanes) rather than this body inferring it from
// the row's flags — see CommentPlanes for the row class where the flags and the
// table disagree, and why that made a listing answer with content-free rows.
func HydrateListComments(ctx context.Context, newSrc func() CommentStreamer, planes CommentPlanes, items []*types.IssueWithCounts, include, countsKnown bool) error {
if len(items) == 0 {
return nil
}
if !include {
if countsKnown {
markCommentsOmitted(items)
}
return nil
}
if newSrc == nil {
return fmt.Errorf("hydrate list comments: comment source must not be nil")
}
if planes == nil {
return fmt.Errorf("hydrate list comments: comment plane resolution must not be nil (pass SourceRoutesCommentPlanes when the source routes ids itself)")
}
rows := rowsNeedingComments(items, countsKnown)
if len(rows) == 0 {
return nil
}
src := newSrc()
if src == nil {
return fmt.Errorf("hydrate list comments: comment source must not be nil")
}
// One plane query for the whole page, and only for the rows that are about
// to be read: the ids are the same set the loop below walks, so a page that
// needs no comment read pays for no plane lookup either.
wisp, err := planes(ctx, rowIDs(rows))
if err != nil {
return fmt.Errorf("hydrate list comments: resolve comment planes: %w", err)
}
for _, item := range rows {
comments, err := collectComments(ctx, src, item.ID, wisp[item.ID])
if err != nil {
return fmt.Errorf("hydrate list comments: %w", err)
}
item.Issue.Comments = comments
}
return nil
}
// rowsNeedingComments is the page narrowed to the rows a comment read has to be
// issued for, and it is what bounds both that read and the plane lookup.
//
// A row the page can PROVE has no comments needs no query. Only a hydrated
// count proves it: under SkipCounts a zero means unknown, and treating it as
// none is how this body once dropped every comment a caller had asked for.
func rowsNeedingComments(items []*types.IssueWithCounts, countsKnown bool) []*types.IssueWithCounts {
out := make([]*types.IssueWithCounts, 0, len(items))
for _, item := range items {
if item == nil || item.Issue == nil {
continue
}
if countsKnown && item.CommentCount == 0 {
continue
}
out = append(out, item)
}
return out
}
func rowIDs(items []*types.IssueWithCounts) []string {
out := make([]string, 0, len(items))
for _, item := range items {
out = append(out, item.ID)
}
return out
}
// markCommentsOmitted flags the rows whose comment text was never fetched.
// Rows the caller already hydrated are left alone: CommentsOmitted never
// appears beside a populated slice, which is what lets a consumer read the two
// fields as one unambiguous answer.
//
// It is called only when the counts are real. The marker's meaning is "this
// row HAS comments and they are not here", so a page whose counts were skipped
// cannot honestly set it on any row — it does not know. That page is less
// informative, which is what its caller asked for by skipping the counts.
func markCommentsOmitted(items []*types.IssueWithCounts) {
for _, item := range items {
if item == nil || item.Issue == nil {
continue
}
if item.CommentCount > 0 && item.Issue.Comments == nil {
omitted := true
item.CommentsOmitted = &omitted
}
}
}