beads/internal/storage/issueops/comments.go
CI Bot 5b3aa25dd5 feat(events): journal read/prune surface with typed truncation (bd-opisf)
The engine could write the journal but nothing could read it. This adds the
consumer-facing half: a store capability both plumbings implement, retention
floors, and a typed answer to the one question a sequential reader cannot
otherwise ask.

`WHERE seq > since` cannot distinguish "nothing new" from "your prefix was
pruned": both come back empty. A consumer resuming past a prune therefore
either stalls forever on an empty success or silently skips to the current
floor and drops every record in between, and neither leaves a trace. Reads now
fail with *storage.EventsJournalTruncatedError{Since, Floor, Head}, handing
back the window the engine can still serve so the caller decides explicitly:
resume from Floor-1 and accept the gap, or rebuild. The decision lives in one
place, ComputeEventsTruncation, so every plumbing answers a pruned-past
checkpoint identically. Seqs are gapless and prune only ever removes a prefix,
so the contiguous case is decided from the rows alone — `tail --follow` polls
once a second and must not pay for a counter read to be told it is healthy.
The counter is the head even when every row is gone, because prune never
touches it; that is what makes a fully-pruned journal distinguishable from a
caught-up one at all.

Pruning is a strict PREFIX delete: --before and both floors resolve to ONE
exclusive seq bound. That is a deliberate departure from the reference, which
expressed the age floor as a per-row `AND ts < cutoff` — and carries a latent
defect because of it. ts is CLIENT-stamped at insert and is NOT monotone in
seq: two writers against one SQL server, or one writer whose clock is stepped
back by NTP between commits, produce an OLDER timestamp at a HIGHER seq. The
per-row predicate then deletes that higher seq while keeping its lower-seq
neighbour, leaving a hole in the MIDDLE of the retained window — and the
truncation check can only see a missing prefix, so the hole is silent record
loss in the one feature whose whole promise is that a cursor can be trusted.
Resolving the age floor to a seq (MIN(seq) WHERE ts >= cutoff) costs one
scalar read and makes an interleaved timestamp keep more rather than split a
pair. Enterprise carries the per-row form and therefore the latent bug; flag it
at the next sync.

The read then checks the other half of the same property. Rows arrive in
memory already, so ComputeEventsTruncation sweeps the batch for an interior
seq gap and refuses on any hole regardless of cause — which covers what the
prefix argument cannot: a restored, hand-edited or partially-copied journal
table, and any future prune path that forgets the rule. It costs no query on
the healthy path.

The two checks are ordered: missing prefix first, interior sweep second. A
batch can have BOTH shapes at once and only one window can be reported;
naming the interior hole would hand back a Floor beyond a gap the caller was
never told about, stepping over it silently. Reporting the nearest hole means
a consumer that resumes from Floor-1 meets the next one on its next read, so
every gap is surfaced one resume at a time. An interior hole reports the last
seq servable from the caller's checkpoint as Since, so the value never moves
BACKWARD from what the caller presented and echoing it can never re-deliver
records a consumer already holds.

Both retention floors default ON (7 days / 100k rows). With both at 0 a single
`prune --before <large>` deletes the whole journal and strands the consumer it
was enabled for, with no local copy to recover from. Floors can only narrow the
bound; 0 still disables one explicitly. They are NOT a consumer watermark and
must not be sold as one — they protect a recent window, and a consumer further
behind than both allow can still be pruned past.

EventsJournalAccessor is the read/prune capability, implemented by BOTH direct
stores: the embedded store through its per-operation withConn (it owns its
connections and exposes no stable *sql.DB, so RawDBAccessor is not an option
there), the server-mode store through withReadTx/withRetryTx. The query bodies
stay at the issueops seam, so the two differ only in transaction machinery.

The unit-of-work plumbing gets the same reach through a repository and use case
rather than a raw connection — and, more importantly, it gets an activation
path it never had. Emission at the issueops seam is only half of coverage: the
plumbing must also turn journaling ON for the transaction the mutation runs in,
and doltSQLProvider had no way to. Left that way the second plumbing would have
shipped a journal that is permanently empty while every command succeeds and
every write lands, which is invisible from the outside. The provider now
implements storage.EventsJournalConfigurer and binds activation to the pinned
connection in BeginTx, after START TRANSACTION so the seq UPDATE and the SELECT
that must observe it stay on one session inside one transaction. The scope is
released wherever the connection is — releaseConn and poisonConn — so an
activation entry cannot outlive the transaction it describes and hand the next
borrower a journaling session.

Two vocabularies are frozen here rather than left implicit. The op set: six
wire ops plus engine-only `comment`, declared independently of the EventOp
constants so a new op that nobody classifies fails the test. And
EventComment.Source, which was an untyped string repeated at seven emit sites —
now CommentSourceStructured / CommentSourceAudit, which is what stops a
consumer-facing fixture from pinning a third value no emitter produces.
Consumers switch on these strings, so a rename is a breaking change even when
it compiles.

Also fixes a wire-contract gap the engine commit left: types.Issue gained
IsBlocked, which put is_blocked on the /v0 HTTP surface without a spec entry,
and the bijection guard fails on exactly that. The property is documented in
the five schemas pinned to the issue types; no generated output changes,
because those schemas are pinned to types.Issue rather than mirrored. Its
description says what is true rather than what would be tidy: no /v0 read
endpoint populates the field today, so its absence from a response says
nothing about whether an issue is blocked — it is the journal snapshots that
carry it, and they serialize the same type these schemas pin.

Ports 0a3fe0f9b (API-3.1 truncation + floors) and the accessor/domain shapes of
3bfed2431 onto current main. Deviations: the prune predicate above (an upstream
defect, fixed rather than replayed); and the reference activates the proxied
path by context (issueops.WithEventsJournal), which the engine commit
documented as test-only and in-transaction-only — provider-scoped activation is
the same guarantee bound to a real transaction.

Co-authored-by: CI Bot <ci@beads.test>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agent-Signature: claude-code-claude-opus-5[1m]-unknown-reasoning on behalf of julianknutsen
2026-08-09 01:58:17 +00:00

322 lines
12 KiB
Go

package issueops
import (
"context"
"database/sql"
"fmt"
"strings"
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// GetIssueCommentsInTx retrieves comments for an issue within an existing
// transaction. Automatically routes to wisp_comments if the ID is an active wisp.
//
//nolint:gosec // G201: table names come from WispTableRouting (hardcoded constants)
func GetIssueCommentsInTx(ctx context.Context, tx DBTX, issueID string) ([]*types.Comment, error) {
table := "comments"
if IsActiveWispInTx(ctx, tx, issueID) {
table = "wisp_comments"
}
rows, err := tx.QueryContext(ctx, fmt.Sprintf(`
SELECT id, issue_id, author, text, created_at
FROM %s
WHERE issue_id = ?
ORDER BY created_at ASC, id ASC
`, table), issueID)
if err != nil {
return nil, fmt.Errorf("get issue comments from %s: %w", table, err)
}
defer rows.Close()
var comments []*types.Comment
for rows.Next() {
var c types.Comment
if err := rows.Scan(&c.ID, &c.IssueID, &c.Author, &c.Text, &c.CreatedAt); err != nil {
return nil, fmt.Errorf("get issue comments: scan: %w", err)
}
comments = append(comments, &c)
}
return comments, rows.Err()
}
// Comment page-read tuning. Mirrors the EventsSince keyset clamp: an unbounded
// page defeats the purpose of paging a long thread, so a non-positive limit
// falls back to the default and any larger request is capped.
const (
defaultCommentsPageLimit = 100
maxCommentsPageLimit = 500
)
// CommentsKeysetPredicate is the SARGABLE (created_at ASC, id ASC) keyset resume
// predicate GetIssueCommentsPageInTx ANDs in once a page cursor is set. Its three
// ? placeholders bind, in order: created_at (the sargable lower bound),
// created_at (strict), and id (the same-second tie-break).
//
// It is logically "(created_at, id) is strictly after the cursor" under
// created_at ASC, id ASC — i.e. (created_at > ?) OR (created_at = ? AND id > ?)
// — but rewritten with a redundant `created_at >= ?` leading bound so the
// planner seeks the (issue_id, created_at, id) index (an IndexedTableAccess
// range on Dolt) instead of scanning the issue's comments and filtering. The two
// forms select the same rows: created_at >= C is true whenever the OR is, and
// prunes only created_at < C, which the OR already excludes. It is exported so
// the backend sargability guard EXPLAINs this exact string rather than a
// hand-copied literal — a change here then breaks the guard.
const CommentsKeysetPredicate = `created_at >= ? AND ((created_at > ?) OR (id > ?))`
// CommentsPageQuery returns the exact SQL GetIssueCommentsPageInTx executes for
// the given comment table (comments or wisp_comments), cursor presence, and
// already-clamped limit. The ? placeholders bind in order: issue_id, then — when
// hasCursor — the three CommentsKeysetPredicate binds (created_at, created_at,
// id). It is exported so the sargability guard EXPLAINs this production string
// (with CommentsKeysetPredicate embedded) rather than a copy — a drift in either
// the surrounding query or the predicate then breaks the guard.
//
//nolint:gosec // G201: table is a hardcoded routing constant and limit is an int the caller clamps; every runtime value is a bound parameter.
func CommentsPageQuery(table string, hasCursor bool, limit int) string {
query := fmt.Sprintf(`
SELECT id, issue_id, author, text, created_at
FROM %s
WHERE issue_id = ?`, table)
if hasCursor {
query += " AND " + CommentsKeysetPredicate
}
query += fmt.Sprintf(" ORDER BY created_at ASC, id ASC LIMIT %d", limit)
return query
}
// GetIssueCommentsPageInTx returns one keyset page of an issue's comments within
// an existing transaction, ordered by (created_at ASC, id ASC). It routes to
// wisp_comments when issueID is an active wisp, exactly like GetIssueCommentsInTx.
//
// after is the resume position — the (created_at, id) of the last comment already
// seen. The zero cursor starts from the beginning of the thread. Walking the
// pages, feeding each page's last (created_at, id) back in as after, yields
// exactly the same comments in the same order as GetIssueCommentsInTx, with no
// dropped or duplicated comment even when several comments share a created_at
// second, because id (the primary key) is a total tie-break.
//
// The cursor must come from a previously returned comment: created_at is a
// DATETIME(0) column, so a sub-second cursor CreatedAt sorts after same-second
// rows stored at the truncated second and skips them on resume (AddIssueCommentInTx
// truncates its returned CreatedAt to avoid exactly this). And like an audit
// feed, a comment inserted with a backdated created_at behind an in-progress
// cursor is not seen by that forward-only walk.
//
// limit <= 0 falls back to defaultCommentsPageLimit (100); a larger limit is
// capped at maxCommentsPageLimit (500), so a caller that pages until
// len(page) < limit must keep limit <= 500 or terminate on an empty page
// instead. A missing issue routes to comments and returns an empty page with no
// error, matching GetIssueCommentsInTx.
func GetIssueCommentsPageInTx(ctx context.Context, tx *sql.Tx, issueID string, after storage.CommentPageCursor, limit int) ([]*types.Comment, error) {
if limit <= 0 {
limit = defaultCommentsPageLimit
}
if limit > maxCommentsPageLimit {
limit = maxCommentsPageLimit
}
table := "comments"
if IsActiveWispInTx(ctx, tx, issueID) {
table = "wisp_comments"
}
hasCursor := !after.CreatedAt.IsZero() || after.ID != ""
args := []any{issueID}
if hasCursor {
// Bind the cursor time as time.Time, not a formatted string: created_at
// is a DATETIME column, so a time.Time value compares correctly on every
// backend while an RFC3339 string can mis-compare. Bound twice (the
// sargable lower bound and the strict bound), then the id tie-break.
args = append(args, after.CreatedAt, after.CreatedAt, after.ID)
}
rows, err := tx.QueryContext(ctx, CommentsPageQuery(table, hasCursor, limit), args...)
if err != nil {
return nil, fmt.Errorf("get issue comments page from %s (after %v/%q): %w", table, after.CreatedAt, after.ID, err)
}
defer rows.Close()
var comments []*types.Comment
for rows.Next() {
var c types.Comment
if err := rows.Scan(&c.ID, &c.IssueID, &c.Author, &c.Text, &c.CreatedAt); err != nil {
return nil, fmt.Errorf("get issue comments page: scan: %w", err)
}
comments = append(comments, &c)
}
return comments, rows.Err()
}
// GetCommentCountsInTx returns comment counts per issue ID within a transaction.
// Routes each ID to comments or wisp_comments based on wisp status.
// Uses batched IN clauses (queryBatchSize) to avoid query-planner spikes.
func GetCommentCountsInTx(ctx context.Context, tx *sql.Tx, issueIDs []string) (map[string]int, error) {
if len(issueIDs) == 0 {
return make(map[string]int), nil
}
result := make(map[string]int)
wispIDs, permIDs, err := PartitionWispIDsInTx(ctx, tx, issueIDs)
if err != nil {
return nil, fmt.Errorf("partition comment issue IDs: %w", err)
}
for _, pair := range []struct {
table string
ids []string
}{
{"wisp_comments", wispIDs},
{"comments", permIDs},
} {
if len(pair.ids) == 0 {
continue
}
for start := 0; start < len(pair.ids); start += queryBatchSize {
end := start + queryBatchSize
if end > len(pair.ids) {
end = len(pair.ids)
}
batch := pair.ids[start:end]
placeholders := make([]string, len(batch))
args := make([]any, len(batch))
for i, id := range batch {
placeholders[i] = "?"
args[i] = id
}
//nolint:gosec // G201: pair.table is hardcoded
rows, err := tx.QueryContext(ctx, fmt.Sprintf(
`SELECT issue_id, COUNT(*) as cnt FROM %s WHERE issue_id IN (%s) GROUP BY issue_id`,
pair.table, strings.Join(placeholders, ",")), args...)
if err != nil {
return nil, fmt.Errorf("get comment counts from %s: %w", pair.table, err)
}
for rows.Next() {
var issueID string
var count int
if err := rows.Scan(&issueID, &count); err != nil {
_ = rows.Close()
return nil, fmt.Errorf("get comment counts: scan: %w", err)
}
result[issueID] = count
}
_ = rows.Close()
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("get comment counts: rows: %w", err)
}
}
}
return result, nil
}
// AddIssueCommentInTx adds a structured comment to an issue within a transaction.
// Routes to comments or wisp_comments based on wisp status.
//
// The insert timestamp is truncated to whole seconds to match the created_at
// column's DATETIME(0) precision, so the returned Comment.CreatedAt equals the
// stored value. That makes the returned comment safe to use directly as a
// GetIssueCommentsPage cursor: an un-truncated sub-second CreatedAt would sort
// after same-second rows stored at the truncated second and skip them on resume.
//
// It is also advanced past the issue's newest existing comment when that
// truncation would otherwise collide — see nextLiveCommentTime.
//
//nolint:gosec // G201: table names come from hardcoded constants
func AddIssueCommentInTx(ctx context.Context, tx *sql.Tx, issueID, author, text string) (*types.Comment, error) {
return addIssueCommentInTx(ctx, tx, issueID, author, text, time.Now().UTC().Truncate(time.Second), true)
}
// ImportIssueCommentInTx adds a comment preserving the original timestamp.
//
//nolint:gosec // G201: table names come from hardcoded constants
func ImportIssueCommentInTx(ctx context.Context, tx *sql.Tx, issueID, author, text string, createdAt time.Time) (*types.Comment, error) {
return addIssueCommentInTx(ctx, tx, issueID, author, text, createdAt, false)
}
//nolint:gosec // G201: table names come from hardcoded constants
func addIssueCommentInTx(ctx context.Context, tx *sql.Tx, issueID, author, text string, createdAt time.Time, live bool) (*types.Comment, error) {
isWisp := IsActiveWispInTx(ctx, tx, issueID)
issueTable, _, _, _ := WispTableRouting(isWisp)
commentTable := "comments"
if isWisp {
commentTable = "wisp_comments"
}
// Verify issue exists.
var exists bool
if err := tx.QueryRowContext(ctx, fmt.Sprintf(
`SELECT EXISTS(SELECT 1 FROM %s WHERE id = ?)`, issueTable), issueID).Scan(&exists); err != nil {
return nil, fmt.Errorf("check issue existence: %w", err)
}
if !exists {
return nil, fmt.Errorf("issue %s not found", issueID)
}
if live {
advanced, err := NextLiveCommentTime(ctx, tx, commentTable, issueID, createdAt)
if err != nil {
return nil, err
}
createdAt = advanced
}
createdAtText := FormatAuxTime(createdAt)
id, _, err := InsertDerivedComment(ctx, tx, commentTable, issueID, author, text, createdAtText)
if err != nil {
return nil, err
}
stored, err := ParseAuxTime(createdAtText)
if err != nil {
return nil, fmt.Errorf("add comment to %s: %w", commentTable, err)
}
comment := &types.Comment{
ID: id,
IssueID: issueID,
Author: author,
Text: text,
CreatedAt: stored,
}
if err := RecordCommentEventInTx(ctx, tx, issueID, &EventComment{
ID: id, Author: author, Text: text, CreatedAt: stored, Source: CommentSourceStructured,
}); err != nil {
return nil, err
}
return comment, nil
}
// AddCommentEventInTx adds a comment as an event to an issue within a transaction.
// Routes to events or wisp_events based on wisp status.
//
//nolint:gosec // G201: table names come from WispTableRouting (hardcoded constants)
func AddCommentEventInTx(ctx context.Context, tx DBTX, issueID, actor, comment string) error {
isWisp := IsActiveWispInTx(ctx, tx, issueID)
_, _, eventTable, _ := WispTableRouting(isWisp)
createdAt := NowAuxTime()
id, err := InsertDerivedEventReturningID(ctx, tx, eventTable, AuxEvent{
IssueID: issueID,
EventType: types.EventCommented,
Actor: actor,
Comment: str(comment),
CreatedAt: createdAt,
})
if err != nil {
return fmt.Errorf("add comment event to %s: %w", eventTable, err)
}
stored, err := ParseAuxTime(createdAt)
if err != nil {
return fmt.Errorf("add comment event to %s: %w", eventTable, err)
}
// An audit-trail comment is replayable text a consumer must be able to
// reproduce, so it carries the same payload as a structured comment,
// distinguished by Source.
return RecordCommentEventInTx(ctx, tx, issueID, &EventComment{
ID: id, Author: actor, Text: comment, CreatedAt: stored, Source: CommentSourceAudit,
})
}