feat(journalops): the mutation journal becomes the third facade namespace (#5552)

* feat(journalops): the mutation journal becomes the third facade namespace

The durable mutation journal's read side is now a role. `journalops` is a
leaf package beside `issueops` and `memoryops` holding Row, Page,
TruncatedError/TruncatedCode and `Journal`, whose doc comment is the
specification: since-exclusive, seq-ascending, limit 0 uncapped with the
wire's cap named as the handler's, a head that comes with its rows and never
decreases, a caught-up checkpoint, and a typed truncation naming a resumable
window.

NOTHING CHANGES BEHAVIOR. The four names in `internal/storage` become
ALIASES of the leaf — `type EventsJournalCursor = journalops.Journal` and its
three siblings — so every implementation, every errors.As site and every
caller compiles unchanged. The alias direction is what avoids the import
cycle: the leaf imports context and fmt and cannot name internal/storage, so
the canon goes down and the alias goes up. Outside storage.go and the
conformance tier, the only non-test edits in this change are COMMENTS.

The role brings the apparatus a role owes:

- backend/conformance/journal_contract.go, six cases over a fixture carrying
  the role, the activation switch, a prune, and a mutation kit that is
  exhaustive against the engine's closed op vocabulary.
- Wirings on all three legs (dolt, embeddeddolt, uow), so #5499's
  entrypoint-scoped lock passes with every leg running every case.
- journalops added to the conformance census's facadePackages. That is the
  #5459 demand side, and the role that needed it: `Journal` is handed out by
  NO accessor — a front door type-asserts for it — so only a source parse can
  see it. issueops.Importer is the precedent.

EventsJournalAccessor and EventsJournalConfigurer stay in internal/storage.
Retention and activation are operator surface, and a server documented to
publish the journal and never retain it must not hold a delete.

Net deletion on the bespoke side: TestEventsJournalPageReportsTheHead is
subsumed by the contract and is gone. TestEventsJournalRestartAndRetentionBoundary
stays, because it observes ReadEventsInTx — the accessor's second read body,
deliberately not on the role — and a mutation of one is invisible to the other.

Mutation-measured, per case, on the embedded leg: an inclusive checkpoint, a
head derived from the page, an untyped truncation, a head taken from MAX(seq)
of surviving rows, and a dropped engine-only op each redden exactly the arm
named for them, and a fixture whose activation hook is a no-op reddens all
six. Reading the head BEFORE the rows leaves all six green — that promise is
declared unpinnable in the contract header rather than faked.

Refs ga-fvya1, ga-2yaqp.4.

* docs(journalops): fix three comment-precision findings from review

All comments, plus one test-only compile-time pin block.

1. journal_facade_test.go told a misassembled causal story: it used the
   CURSOR de-aliasing as its example and then attributed the breakage to
   errors.As. Those are two different failures. Measured, both:
   de-aliasing the interface builds the WHOLE TREE clean and breaks nothing
   behavioral — what it costs is the claim that one role has one name — while
   the error's dangerous shape is not the bare defined type (which loses
   Error() and fails to build) but the COPY-PASTE TWIN, which compiles in
   silence and stops crossing errors.As for everything holding the leaf's
   spelling. Rewritten to say that straight.

2. The seq commit-order unpinnable declaration cited "the dolt package's own
   journal tests" for the server-mode half. Those are single-threaded. The
   actual pin is internal/storage/domain/db/journal_seq_test.go —
   TestEventsJournal_CommitOrderedGaplessSeq drives two OVERLAPPING
   transactions on independent connections, and
   TestEventsJournal_ConcurrentWritersGaplessNoDup a racing batch. Cited
   correctly, and the single-threaded ones named as NOT the pin.

3. EveryMutationKindLandsARow's runs-last ordering is load-bearing but held
   by convention: a reorder compiles, passes, and silently gives up the only
   coverage that a journal keeps recording after a full prune. It was stated
   in the dispatch table and the three leg wirings; it is now stated at the
   case itself, where somebody editing a leg file is actually looking.

And the compile-time half of the alias pin, which fires before any test runs:
a pointer conversion per name. The pointer form is not decoration — for the
three structs a plain assignment would do, but for the INTERFACE it would be
blind, since interface-to-interface assignment goes by method set and a
de-aliased cursor still accepts a journalops.Journal. Watched fail: the
de-aliased interface, which nothing else in the tree catches.

---------

Co-authored-by: Review Sim <ci@beads.test>
This commit is contained in:
Julian Knutsen 2026-08-10 10:58:05 -07:00 • committed by GitHub
commit 4e7cc88180
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
16 changed files with 1587 additions and 177 deletions

View file

@ -0,0 +1,676 @@
package conformance
import (
"context"
"errors"
"math"
"testing"
storeops "github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/journalops"
)
// This file holds the contract every implementation of journalops.Journal must
// satisfy. Each case asserts what journalops/journal.go PROMISES rather than
// what any one backend happens to do today.
//
// THE VOTE COUNT. Three wirings — the server-backed store, the embedded store
// and the unit-of-work provider — and ONE body between them: all three bottom
// out in issueops.ReadEventsPageInTx, the two stores through a five-line
// accessor around their own transaction helper and the unit of work through
// domain.EventsJournalUseCase. So this is one reading plus an engine check, the
// arrangement issueops.TreeWalker was the first to have, and the cases are
// written for it: they assert the typed error's FIELDS and the page's members
// rather than message text, because what a per-leg failure would actually BE
// here is a wrapper that loses a transaction, drops the head, or breaks
// errors.As across the alias.
//
// THAT LAST ONE IS WHY THE TRUNCATION CASE MATTERS MORE THAN IT LOOKS.
// storage.EventsJournalCursor, storage.EventsJournalRow, storage.EventsJournalPage
// and storage.EventsJournalTruncatedError are ALIASES of the journalops names,
// so every leg still returns the type it always returned. These cases hold the
// role's own spelling — journalops.TruncatedError — against three
// implementations that never name it, which is the alias identity checked at
// runtime rather than argued from the language spec.
//
// WHAT THIS CONTRACT DELIBERATELY DOES NOT PIN, and the reason, so a later
// reader does not mistake the gap for an oversight:
//
// - THE HEAD AND THE ROWS COMING FROM ONE INSTANT. It is the promise the Page
// type exists for, and it is not black-box observable here. That was
// MEASURED rather than assumed: reordering ReadEventsPageInTx to read the
// head BEFORE the rows — the exact regression the promise forbids — leaves
// all six cases green, because a single-threaded case has nothing to commit
// in the gap it opens. A concurrent case would be flaky at three engines and
// would buy a suite people re-run rather than a guarantee.
//
// What IS observable is the OTHER way a head goes wrong, and both are
// pinned: a head derived from the page (RunJournalLimitCapsRowsNotHead) and
// a head derived from the surviving rows instead of the counter
// (RunJournalHeadSurvivesAFullPrune). The one-instant promise itself is held
// by the SHAPE of the body — both reads are inside the caller's transaction
// and there is no two-call composition to regress into without deleting it —
// and by review. Do not fake it with sleeps.
// - THE COMMIT-ORDER PROPERTY OF SEQ. That seqs are gapless and ordered by
// COMMIT rather than by insert is a claim about concurrent writers
// (issueops.nextEventSeq), and the leg that can actually violate it is the
// shared SQL server. It is pinned where the concurrency actually lives:
// internal/storage/domain/db/journal_seq_test.go, whose
// TestEventsJournal_CommitOrderedGaplessSeq drives two OVERLAPPING
// transactions on independent connections and whose
// TestEventsJournal_ConcurrentWritersGaplessNoDup drives a racing batch,
// plus embeddeddolt/journal_concurrency_test.go for the embedded engine.
// Not the dolt package's own journal tests, which exercise the same
// server-mode plumbing but are single-threaded, and not here, where every
// case is one writer.
// - RETENTION POLICY. The retain-days and retain-rows floors are the
// operator's surface (storage.EventsJournalAccessor), deliberately not on
// this role. The fixture's Prune hook exists to CREATE the truncation the
// read contract has to answer, and the cases pass both floors disabled; how
// a floor resolves is pinned beside the prune bodies.
//
// SEEDING DISCIPLINE: EVERY CASE REBASELINES. The journal is append-only and
// workspace-global, the fixture is built once per role, and two of these cases
// PRUNE — one of them to nothing. So no case may assume an empty journal or a
// seq of its own choosing. Each takes the live head first (journalHead), drives
// its own mutations, and asserts about the window above that baseline. A case
// written against absolute seqs would pass in isolation and fail the moment it
// ran second, which is the shape of a contract nobody can reorder.
// JournalFixture supplies adapter-specific access for the journal assertions:
// the role under test, the activation switch its rows depend on, the prune that
// creates a truncation, and the mutation kit that makes records exist.
type JournalFixture struct {
// IssuePrefix namespaces the ids each case seeds. Ids are global to a
// workspace and every case here shares one, so a case that reused another's
// id would read another's journal rows.
IssuePrefix string
// Journal is the role under test, reached through whatever the backend
// hands it out by — a type assertion on the store, or the unit-of-work
// provider's EventsJournalCursor accessor. Never a constructor: the
// accessor, or the assertion, is what a front door actually holds.
Journal journalops.Journal
// SetJournalEnabled turns durable journaling on for the workspace under
// test (storage.EventsJournalConfigurer). It is OPERATOR surface and
// deliberately not on the role, which is exactly why the fixture has to
// carry it: the cases need records to exist, and a workspace that never
// opted in records nothing while every read still succeeds emptily.
//
// Every case calls it with true before it seeds, rather than trusting a
// wiring to have done so. It is idempotent, it costs nothing, and a leg
// that forgot would otherwise fail with "no rows" in six places instead of
// naming the one thing that was wrong.
SetJournalEnabled func(enabled bool)
// Prune deletes journal records below before, honoring the retain-days and
// retain-rows floors (0 disables a floor), and reports how many it removed
// (storage.EventsJournalAccessor.PruneEventsJournal, or the unit of work's
// EventsJournalUseCase.Prune).
//
// It is here to MANUFACTURE the condition the read contract has to answer,
// not to be tested. Both truncation cases pass both floors disabled, so the
// bound is exactly what they ask for.
Prune func(ctx context.Context, before int64, retainDays, retainRows int) (int64, error)
// Mutations is one hook per journaled op — see JournalMutations, which
// states why the kit is exhaustive rather than a sample.
Mutations JournalMutations
}
// JournalMutations is the mutation kit: one hook per op the engine journals,
// each driving the backend's own front-door verb for it.
//
// IT IS EXHAUSTIVE BY CONSTRUCTION, and that is the point of naming the hooks
// as fields rather than accepting a list. The journaled vocabulary is closed
// and declared once (issueops.WireEventOps and issueops.EngineOnlyEventOps),
// RunJournalEveryMutationKindLandsARow checks this kit against it, and a leg
// that supplied a sample would prove the journal records the ops it happens to
// have thought of. An op added to the engine vocabulary without a hook here
// fails that case rather than quietly joining the set nothing drives.
//
// EVERY HOOK DRIVES THE BACKEND'S OWN VERB, never the journal. Emission lives
// at a seam beneath all of these (issueops.RecordEventInTx and its siblings),
// so a hook that inserted a record directly would assert that this test can
// write a row.
type JournalMutations struct {
// Create creates a durable issue with the given id.
Create func(ctx context.Context, id string) error
// Update mutates a scalar field of an existing issue.
Update func(ctx context.Context, id string) error
// Close closes an existing open issue.
Close func(ctx context.Context, id string) error
// Delete removes an existing issue. Its record carries no issue snapshot —
// there is no surviving row to take one from.
Delete func(ctx context.Context, id string) error
// AddDependency adds one blocking edge from an existing issue to another.
// The record is the SOURCE's: from is the id the row names.
AddDependency func(ctx context.Context, from, to string) error
// RemoveDependency removes that edge again, and its record is the source's
// for the same reason.
RemoveDependency func(ctx context.Context, from, to string) error
// Comment writes one structured comment on an existing issue. Its op is the
// one the engine journals and the wire never carries; see
// RunJournalEveryMutationKindLandsARow.
Comment func(ctx context.Context, id, text string) error
}
// RunJournalPagesAreSeqAscendingAndSinceExclusive pins the two properties a
// resuming consumer's correctness rests on: the checkpoint it hands back is
// EXCLUSIVE, and what it gets back is in seq order.
//
// The exclusivity is asserted twice, and the second one is the load-bearing
// half. A read from the caller's own baseline could start at baseline+1 under
// either boundary if nothing sits exactly at baseline, so the case also reads
// from a seq it has just been SERVED — a row that provably exists — and
// requires that row to be absent from the answer. An inclusive boundary
// replays one record per poll forever, which is silent duplication rather than
// a visible failure.
func RunJournalPagesAreSeqAscendingAndSinceExclusive(t *testing.T, ctx context.Context, fixture JournalFixture) {
baseline := journalBaseline(t, ctx, fixture)
ids := journalSeedCreates(t, ctx, fixture, "asc", 3)
page := journalRead(t, ctx, fixture, baseline, 0)
if len(page.Rows) != len(ids) {
t.Fatalf("read %d records above the baseline, want the %d this case created: %+v",
len(page.Rows), len(ids), journalOpsOf(page.Rows))
}
if page.Rows[0].Seq != baseline+1 {
t.Errorf("first record seq = %d, want %d: the answer begins immediately after the checkpoint",
page.Rows[0].Seq, baseline+1)
}
for i, row := range page.Rows {
if row.Seq <= baseline {
t.Errorf("record %d has seq %d, at or below the checkpoint %d: since is EXCLUSIVE",
i, row.Seq, baseline)
}
if i > 0 && row.Seq <= page.Rows[i-1].Seq {
t.Errorf("record %d has seq %d, not above its predecessor's %d: records are seq-ASCENDING",
i, row.Seq, page.Rows[i-1].Seq)
}
if row.IssueID != ids[i] {
t.Errorf("record %d names %q, want %q: the order records arrive in is the order the "+
"mutations committed in", i, row.IssueID, ids[i])
}
}
// Resume from a record the caller has been served. Under an inclusive
// boundary this answer repeats it.
resumed := journalRead(t, ctx, fixture, page.Rows[0].Seq, 0)
if len(resumed.Rows) != len(ids)-1 {
t.Fatalf("resuming from seq %d returned %d records, want %d",
page.Rows[0].Seq, len(resumed.Rows), len(ids)-1)
}
if resumed.Rows[0].Seq == page.Rows[0].Seq {
t.Errorf("resuming from seq %d served that same record again: a consumer storing the seq it "+
"processed and handing it back would replay one record on every poll", page.Rows[0].Seq)
}
if resumed.Rows[0].Seq != page.Rows[1].Seq {
t.Errorf("resuming from seq %d began at %d, want %d: the next record and nothing before it",
page.Rows[0].Seq, resumed.Rows[0].Seq, page.Rows[1].Seq)
}
}
// RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp pins what a poller reads
// to decide between asking again and waiting.
//
// Three facts, and the third is what the whole role is for: the head describes
// the journal's history, it moves with the records, and a checkpoint that has
// reached it is CAUGHT UP — no records, no error, the same head. A checkpoint
// ABOVE the head is caught up too, which is not a curiosity: it is what makes
// the head safe to probe, and every other case in this file relies on it.
func RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp(t *testing.T, ctx context.Context, fixture JournalFixture) {
baseline := journalBaseline(t, ctx, fixture)
journalSeedCreates(t, ctx, fixture, "head", 2)
page := journalRead(t, ctx, fixture, baseline, 0)
if len(page.Rows) != 2 {
t.Fatalf("read %d records, want the 2 this case created: %+v", len(page.Rows), journalOpsOf(page.Rows))
}
last := page.Rows[len(page.Rows)-1].Seq
if page.Head < last {
t.Fatalf("head = %d, below the last record served (%d): a consumer reads that as being past the "+
"end of the journal and stops polling", page.Head, last)
}
if page.Head != last {
t.Errorf("head = %d, want %d: nothing else wrote, so the head is the last record", page.Head, last)
}
caughtUp := journalRead(t, ctx, fixture, page.Head, 0)
if len(caughtUp.Rows) != 0 {
t.Errorf("reading from the head returned %d records, want none", len(caughtUp.Rows))
}
if caughtUp.Head != page.Head {
t.Errorf("head moved from %d to %d with nothing written between the two reads", page.Head, caughtUp.Head)
}
// A checkpoint the journal has never reached is caught up, not an error.
// This is the property journalHead is built on.
beyond := journalRead(t, ctx, fixture, page.Head+1000, 0)
if len(beyond.Rows) != 0 || beyond.Head != page.Head {
t.Errorf("reading from a checkpoint above the head = %d records and head %d, want none and %d",
len(beyond.Rows), beyond.Head, page.Head)
}
// One more mutation, and the head follows it.
journalSeedCreates(t, ctx, fixture, "head-again", 1)
advanced := journalRead(t, ctx, fixture, page.Head, 0)
if len(advanced.Rows) != 1 {
t.Fatalf("read %d records after one more mutation, want 1", len(advanced.Rows))
}
if advanced.Head != advanced.Rows[0].Seq || advanced.Head <= page.Head {
t.Errorf("head = %d after a record at seq %d (was %d): the head has to move with what it describes",
advanced.Head, advanced.Rows[0].Seq, page.Head)
}
}
// RunJournalLimitCapsRowsNotHead pins the difference between the page and the
// journal, which is the whole reason Page carries two members.
//
// A head derived from the records in hand is the plausible implementation and
// the broken one: every bounded read would then report a head equal to its last
// record, a consumer would read "caught up", and it would stall however far
// behind it happened to be. So the case takes a page NARROWER than the backlog
// and requires the head to be ahead of it. It also pins limit 0 as uncapped,
// because a role that quietly imposed a ceiling of its own would make the same
// consumer stall at a boundary nothing documents.
func RunJournalLimitCapsRowsNotHead(t *testing.T, ctx context.Context, fixture JournalFixture) {
baseline := journalBaseline(t, ctx, fixture)
ids := journalSeedCreates(t, ctx, fixture, "limit", 4)
bounded := journalRead(t, ctx, fixture, baseline, 2)
if len(bounded.Rows) != 2 {
t.Fatalf("a limit of 2 returned %d records, want 2", len(bounded.Rows))
}
lastServed := bounded.Rows[len(bounded.Rows)-1].Seq
if bounded.Head <= lastServed {
t.Errorf("a bounded page reported head %d with its last record at %d: the head describes the "+
"JOURNAL, not the page, and a consumer reads this one as caught up while %d records wait",
bounded.Head, lastServed, len(ids)-2)
}
if bounded.Head != baseline+int64(len(ids)) {
t.Errorf("bounded head = %d, want %d: the head of everything this case wrote",
bounded.Head, baseline+int64(len(ids)))
}
uncapped := journalRead(t, ctx, fixture, baseline, 0)
if len(uncapped.Rows) != len(ids) {
t.Fatalf("a limit of 0 returned %d records, want all %d: 0 means uncapped, and any ceiling on "+
"this read belongs to a front door rather than to the role", len(uncapped.Rows), len(ids))
}
if uncapped.Head != bounded.Head {
t.Errorf("head = %d uncapped and %d bounded; the limit must not reach it", uncapped.Head, bounded.Head)
}
// The bounded page is a PREFIX of the uncapped answer, not a sample of it.
for i, row := range bounded.Rows {
if row.Seq != uncapped.Rows[i].Seq {
t.Errorf("bounded record %d is seq %d, want %d: a limit takes the first n, so paging with "+
"one loses nothing", i, row.Seq, uncapped.Rows[i].Seq)
}
}
}
// RunJournalTruncationIsTypedAndNamesTheWindow pins the failure the journal
// exists to make loud.
//
// A checkpoint below the retained window cannot be answered with records, and
// the two silent alternatives are both data loss: an empty success strands the
// consumer forever, and skipping to the current floor drops every record in
// between without saying so. So the read FAILS, with a typed error naming a
// window the caller can act on — and the case proves the window is actually
// resumable rather than merely printed, by resuming from it.
//
// The type is asserted through journalops.TruncatedError, which no leg names:
// they all return the storage alias. errors.As matching across that is the
// alias identity, checked rather than assumed.
func RunJournalTruncationIsTypedAndNamesTheWindow(t *testing.T, ctx context.Context, fixture JournalFixture) {
baseline := journalBaseline(t, ctx, fixture)
journalSeedCreates(t, ctx, fixture, "trunc", 4)
head := baseline + 4
// Prune the first two of the four, floors disabled so the bound is exactly
// what this case asks for. The delete is a prefix, so everything earlier
// cases wrote goes with them.
firstRetained := baseline + 3
journalPrune(t, ctx, fixture, firstRetained)
trunc := journalRequireTruncated(t, "reading from a pruned checkpoint",
journalReadErr(ctx, fixture, baseline, 0))
if trunc.Since != baseline {
t.Errorf("Since = %d, want the caller's own checkpoint %d: the nearest hole is the one a "+
"consumer has to decide about first", trunc.Since, baseline)
}
if trunc.Floor != firstRetained {
t.Errorf("Floor = %d, want %d: the lowest seq still retained", trunc.Floor, firstRetained)
}
if trunc.Head != head {
t.Errorf("Head = %d, want %d: the highest seq ever assigned, which a prune does not move",
trunc.Head, head)
}
// The window is RESUMABLE: Floor-1 is a checkpoint the implementation can
// serve, and it serves everything retained.
resumed := journalRead(t, ctx, fixture, trunc.Floor-1, 0)
if len(resumed.Rows) != 2 || resumed.Rows[0].Seq != trunc.Floor {
t.Fatalf("resuming from Floor-1 (%d) returned %d records starting at %d, want 2 starting at %d: "+
"a window a caller cannot resume from is a window that told it nothing",
trunc.Floor-1, len(resumed.Rows), journalFirstSeq(resumed), trunc.Floor)
}
// A full export from the beginning of history must not present the
// surviving suffix as a complete one.
exported := journalRequireTruncated(t, "exporting from the beginning of history",
journalReadErr(ctx, fixture, 0, 0))
if exported.Floor != firstRetained {
t.Errorf("export Floor = %d, want %d", exported.Floor, firstRetained)
}
// And a BOUNDED read from the same checkpoint takes the same decision, with
// the same window. A limit that returned the first retained records as a
// success would be the silent skip, reintroduced by the paging path.
bounded := journalRequireTruncated(t, "reading a bounded page from a pruned checkpoint",
journalReadErr(ctx, fixture, baseline, 2))
if bounded.Since != trunc.Since || bounded.Floor != trunc.Floor || bounded.Head != trunc.Head {
t.Errorf("a bounded read reported the window [%d..%d] after %d and an unbounded one [%d..%d] "+
"after %d; the limit bounds the ANSWER, never the verdict",
bounded.Floor, bounded.Head, bounded.Since, trunc.Floor, trunc.Head, trunc.Since)
}
}
// RunJournalHeadSurvivesAFullPrune pins the one state where records and history
// disagree, and it is the state that decides whether a consumer can ever stop
// polling.
//
// Prune deletes records; it never touches the counter. So a journal with
// nothing left in it still knows how far it got, which is what lets it answer
// "you are at the end of my history" instead of "I have nothing" — the two are
// the same empty result set at the SQL level, and a head derived from the
// surviving records cannot tell them apart. The case also pins the consequence
// on the write side: seq does not reset, so the next mutation continues the
// history rather than colliding with a consumer's dedupe window.
func RunJournalHeadSurvivesAFullPrune(t *testing.T, ctx context.Context, fixture JournalFixture) {
baseline := journalBaseline(t, ctx, fixture)
journalSeedCreates(t, ctx, fixture, "prune", 2)
head := baseline + 2
journalPrune(t, ctx, fixture, head+1)
emptied := journalRead(t, ctx, fixture, head, 0)
if len(emptied.Rows) != 0 {
t.Fatalf("a fully pruned journal returned %d records", len(emptied.Rows))
}
if emptied.Head != head {
t.Errorf("head = %d after every record was pruned, want %d: the head is the journal's HISTORY, "+
"and a head derived from the surviving records would drop to nothing here", emptied.Head, head)
}
// A consumer that had not reached the head is told, and the window it is
// told about is the empty one: Floor above Head means "fully pruned".
trunc := journalRequireTruncated(t, "reading a fully pruned journal from below its head",
journalReadErr(ctx, fixture, baseline, 0))
if trunc.Floor != head+1 || trunc.Head != head {
t.Errorf("fully pruned window = [%d..%d], want [%d..%d]: Floor above Head is how a caller reads "+
"'nothing retained, caught up to Head'", trunc.Floor, trunc.Head, head+1, head)
}
// Seq continues from the counter, not from the (now empty) table.
journalSeedCreates(t, ctx, fixture, "prune-after", 1)
next := journalRead(t, ctx, fixture, head, 0)
if len(next.Rows) != 1 {
t.Fatalf("read %d records after a post-prune mutation, want 1", len(next.Rows))
}
if next.Rows[0].Seq != head+1 {
t.Errorf("the first record after a full prune is seq %d, want %d: a seq that restarted would "+
"hand a consumer numbers it has already processed", next.Rows[0].Seq, head+1)
}
}
// RunJournalEveryMutationKindLandsARow pins the journal's completeness: every
// op the engine journals is one a reader actually receives.
//
// The vocabulary is CLOSED and declared once — issueops.WireEventOps plus
// issueops.EngineOnlyEventOps — and this case checks the fixture's kit against
// it, so an op added to the engine without a hook here fails rather than
// joining a set nothing drives. A backend that recorded five of the seven would
// otherwise pass every other case in this file: they all read whatever records
// exist, and a missing KIND is invisible to a reader that only counts.
//
// THE ENGINE-ONLY OPS ARE DEMANDED, NOT SKIPPED, and that distinction is the
// whole reason issueops.IsWireEventOp exists. The journal records seven ops and
// the public event vocabulary is six; a projector SKIPS the seventh rather than
// faulting on it, and that skip is only sound while the record is there to be
// skipped. A contract that dropped the engine-only ops because "no wire event
// carries them" would be asserting the projector's view of the journal instead
// of the journal's own, and the first backend to stop recording a comment would
// pass it.
//
// It runs LAST, after both pruning cases, which is deliberate: a journal that
// was emptied and then written to again is the state a long-lived workspace
// actually spends its life in, and the mutations here have to land in it.
//
// MOVING IT EARLIER IS A SILENT WEAKENING, so this is stated here as well as in
// the dispatch table and the three leg wirings — at the site somebody editing a
// leg file would actually be looking. Every case rebaselines off the live head,
// so a reorder still COMPILES AND STILL PASSES; what it quietly gives up is the
// only coverage anywhere that a journal keeps recording after a full prune,
// with nothing going red to say so. If you reorder these, say why.
func RunJournalEveryMutationKindLandsARow(t *testing.T, ctx context.Context, fixture JournalFixture) {
journalBaseline(t, ctx, fixture)
subject := fixture.IssuePrefix + "-kind-subject"
other := fixture.IssuePrefix + "-kind-other"
kit := journalMutationKit(fixture, subject, other)
journalRequireKitCoversTheVocabulary(t, kit)
sawEngineOnly := false
for _, mutation := range kit {
before := journalHead(t, ctx, fixture)
if err := mutation.run(ctx); err != nil {
t.Fatalf("%s: %v", mutation.what, err)
}
page := journalRead(t, ctx, fixture, before, 0)
if !journalRecords(page.Rows, string(mutation.op), mutation.subject) {
t.Errorf("%s recorded no %q for %s; the journal above seq %d holds %v. A mutation kind the "+
"journal drops is a replay that silently diverges from the workspace it claims to mirror",
mutation.what, mutation.op, mutation.subject, before, journalOpsOf(page.Rows))
continue
}
if !storeops.IsWireEventOp(mutation.op) {
sawEngineOnly = true
}
}
if !sawEngineOnly {
t.Errorf("no engine-only op was journaled; the journal is supposed to be WIDER than the public " +
"event vocabulary, and a projector's skip of one is only sound while the record exists")
}
}
// journalMutation is one row of the mutation kit under test: the op it must
// record, the id that record names, and how to drive it.
type journalMutation struct {
op storeops.EventOp
what string
subject string
run func(ctx context.Context) error
}
// journalMutationKit binds each hook the fixture supplies to the op it must
// produce, in an order that leaves each mutation something to act on: the two
// issues exist before anything edits them, the edge exists before it is
// removed, and the delete comes last.
func journalMutationKit(fixture JournalFixture, subject, other string) []journalMutation {
m := fixture.Mutations
return []journalMutation{
{op: storeops.EventCreate, what: "creating an issue", subject: subject,
run: func(ctx context.Context) error {
if err := m.Create(ctx, other); err != nil {
return err
}
return m.Create(ctx, subject)
}},
{op: storeops.EventUpdate, what: "updating an issue", subject: subject,
run: func(ctx context.Context) error { return m.Update(ctx, subject) }},
{op: storeops.EventCommentWrite, what: "commenting on an issue", subject: subject,
run: func(ctx context.Context) error {
return m.Comment(ctx, subject, "a comment the journal records and no wire event carries")
}},
{op: storeops.EventDepAdd, what: "adding a dependency", subject: subject,
run: func(ctx context.Context) error { return m.AddDependency(ctx, subject, other) }},
{op: storeops.EventDepRemove, what: "removing a dependency", subject: subject,
run: func(ctx context.Context) error { return m.RemoveDependency(ctx, subject, other) }},
{op: storeops.EventClose, what: "closing an issue", subject: subject,
run: func(ctx context.Context) error { return m.Close(ctx, subject) }},
{op: storeops.EventDelete, what: "deleting an issue", subject: subject,
run: func(ctx context.Context) error { return m.Delete(ctx, subject) }},
}
}
// journalRequireKitCoversTheVocabulary fails when the kit and the engine's
// closed op vocabulary disagree in either direction.
//
// Both directions matter. An op the engine journals and the kit does not drive
// is a kind nothing here proves is recorded — the gap this case was written
// for. An op the kit drives that the engine does not declare is a case
// asserting a vocabulary nobody publishes, which is how a contract outlives the
// thing it describes.
func journalRequireKitCoversTheVocabulary(t *testing.T, kit []journalMutation) {
t.Helper()
driven := map[storeops.EventOp]bool{}
for _, mutation := range kit {
driven[mutation.op] = true
}
declared := map[storeops.EventOp]bool{}
for _, op := range append(storeops.WireEventOps(), storeops.EngineOnlyEventOps()...) {
declared[op] = true
if !driven[op] {
t.Fatalf("the engine journals %q and this contract drives no mutation that produces it: add a "+
"hook to JournalMutations and a row to journalMutationKit, or the op joins the set nothing "+
"proves is recorded", op)
}
}
for op := range driven {
if !declared[op] {
t.Fatalf("journalMutationKit drives %q, which is in neither issueops.WireEventOps nor "+
"issueops.EngineOnlyEventOps: this contract is asserting a vocabulary nobody declares", op)
}
}
}
// journalBaseline activates the journal and returns the head every assertion in
// the calling case is written relative to. See the seeding-discipline note at
// the top of this file for why no case may assume an empty journal.
func journalBaseline(t *testing.T, ctx context.Context, fixture JournalFixture) int64 {
t.Helper()
fixture.SetJournalEnabled(true)
return journalHead(t, ctx, fixture)
}
// journalHead reads the journal's head through the role itself, from a
// checkpoint no journal will ever reach.
//
// A read at or above the head is CAUGHT UP — no records, no error, the head —
// which RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp pins directly. That
// makes this the one probe that works in every state these cases put the
// journal in, including after a prune has taken the floor above every seq an
// earlier case knew about.
func journalHead(t *testing.T, ctx context.Context, fixture JournalFixture) int64 {
t.Helper()
page, err := fixture.Journal.ReadEventsJournalPage(ctx, math.MaxInt64, 0)
if err != nil {
t.Fatalf("reading the journal head: %v", err)
}
if len(page.Rows) != 0 {
t.Fatalf("a checkpoint above every possible seq returned %d records", len(page.Rows))
}
return page.Head
}
// journalRead is one page, with the error failing the case. Cases that expect a
// failure use journalReadErr instead.
func journalRead(t *testing.T, ctx context.Context, fixture JournalFixture, since int64, limit int) journalops.Page {
t.Helper()
page, err := fixture.Journal.ReadEventsJournalPage(ctx, since, limit)
if err != nil {
t.Fatalf("reading the journal from %d (limit %d): %v", since, limit, err)
}
return page
}
// journalReadErr is one page read for its error alone.
func journalReadErr(ctx context.Context, fixture JournalFixture, since int64, limit int) error {
_, err := fixture.Journal.ReadEventsJournalPage(ctx, since, limit)
return err
}
// journalRequireTruncated asserts err is the role's typed truncation and
// returns it.
//
// It matches journalops.TruncatedError, which no in-tree leg names — they all
// return the storage alias — so the match is the alias identity holding at
// runtime. A nil error is reported as the silent case it actually is, because
// "no error" here means the read presented a gap as a success.
func journalRequireTruncated(t *testing.T, what string, err error) *journalops.TruncatedError {
t.Helper()
if err == nil {
t.Fatalf("%s succeeded; a checkpoint below the retained window has to fail loudly, or the "+
"consumer either stalls forever or skips the pruned records without knowing", what)
}
var trunc *journalops.TruncatedError
if !errors.As(err, &trunc) {
t.Fatalf("%s failed with %T (%v), want *journalops.TruncatedError: a caller dispatches on the "+
"TYPE and reads the window off its fields", what, err, err)
}
return trunc
}
// journalPrune removes every record below before, both retention floors
// disabled.
func journalPrune(t *testing.T, ctx context.Context, fixture JournalFixture, before int64) {
t.Helper()
if _, err := fixture.Prune(ctx, before, 0, 0); err != nil {
t.Fatalf("pruning the journal below %d: %v", before, err)
}
}
// journalSeedCreates creates n issues under the case's own marker and returns
// their ids in creation order.
func journalSeedCreates(t *testing.T, ctx context.Context, fixture JournalFixture, marker string, n int) []string {
t.Helper()
ids := make([]string, 0, n)
for i := range n {
id := fixture.IssuePrefix + "-" + marker + "-" + string(rune('a'+i))
if err := fixture.Mutations.Create(ctx, id); err != nil {
t.Fatalf("seeding %s: %v", id, err)
}
ids = append(ids, id)
}
return ids
}
// journalRecords reports whether any record names op for issue.
func journalRecords(rows []journalops.Row, op, issue string) bool {
for _, row := range rows {
if row.Op == op && row.IssueID == issue {
return true
}
}
return false
}
// journalOpsOf renders a window's ops for a failure message. The payloads are
// deliberately left out: what a reader needs from a failure here is which kinds
// arrived, and issue snapshots are kilobytes each.
func journalOpsOf(rows []journalops.Row) []string {
ops := make([]string, 0, len(rows))
for _, row := range rows {
ops = append(ops, row.Op+"("+row.IssueID+")")
}
return ops
}
// journalFirstSeq is the first record's seq, or 0 for an empty page, so a
// failure message can name what it got without an index panic.
func journalFirstSeq(page journalops.Page) int64 {
if len(page.Rows) == 0 {
return 0
}
return page.Rows[0].Seq
}

View file

@ -92,6 +92,7 @@ type RoleContractBundle struct {
EdgeReader func(t *testing.T) *EdgeReaderFixture
GraphCounter func(t *testing.T) *GraphCounterFixture
Importer func(t *testing.T) *ImporterFixture
Journal func(t *testing.T) *JournalFixture
LifecycleCloseReopen func(t *testing.T) *LifecycleCloseReopenFixture
LifecycleCreate func(t *testing.T) *LifecycleCreateFixture
LifecycleUpdate func(t *testing.T) *LifecycleUpdateFixture

View file

@ -362,6 +362,23 @@ var roleContractCases = []roleContract{
RunIssueOperationsCreateParentChildRecomputesWaitsForClosure,
),
// The accessor named here is not an accessor at all, alone among these
// rows. journalops.Journal is reached by TYPE ASSERTION on a store —
// `bd serve` does exactly that (cmd/bd/serve.go, serveJournalCursor) — or
// through uow.EventsJournalCursorSource on a unit-of-work provider, because
// the journal is engine state on a dolt_ignored table that
// storage.DoltStorage does not publish. A backend that cannot read it
// leaves this field nil; see journal_contract.go's header.
roleCases("Journal", "the storage.EventsJournalCursor type assertion (or uow.EventsJournalCursor())", oncePerRole,
func(b RoleContractBundle) func(t *testing.T) *JournalFixture { return b.Journal },
RunJournalPagesAreSeqAscendingAndSinceExclusive,
RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp,
RunJournalLimitCapsRowsNotHead,
RunJournalTruncationIsTypedAndNamesTheWindow,
RunJournalHeadSurvivesAFullPrune,
RunJournalEveryMutationKindLandsARow,
),
roleCases("LifecycleCloseReopen", "IssueLifecycle()", oncePerRole,
func(b RoleContractBundle) func(t *testing.T) *LifecycleCloseReopenFixture {
return b.LifecycleCloseReopen

View file

@ -17,6 +17,7 @@ import (
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/issueops"
"github.com/steveyegge/beads/journalops"
"github.com/steveyegge/beads/memoryops"
)
@ -39,8 +40,17 @@ const modulePath = "github.com/steveyegge/beads"
// gate names its interfaces by. The paths come from real types rather than
// string literals, so moving a package breaks the build here instead of
// quietly emptying the census.
//
// journalops is the third, and it is the entry that proves this census had to
// be a SOURCE parse. Its one role is handed out by no accessor at all — a
// backend publishes the journal by implementing an interface a caller
// type-asserts for — so reflectRoleAccessors below can never see it, exactly as
// it can never see issueops.Importer. A package listed here is censused from
// its declarations, which is what puts a role with no accessor under the
// exhaustiveness gate rather than outside it.
var facadePackages = map[string]string{
reflect.TypeOf((*issueops.Reader)(nil)).Elem().PkgPath(): "issueops",
reflect.TypeOf((*journalops.Journal)(nil)).Elem().PkgPath(): "journalops",
reflect.TypeOf((*memoryops.Memories)(nil)).Elem().PkgPath(): "memoryops",
}

View file

@ -735,6 +735,16 @@ func serveIssueRoles(src serveRoleSource, journalEnabled bool) (serveRoles, erro
// assertion has to reach the concrete store or it finds nothing at
// all. Nothing is skipped by going the whole way: there is no
// journal decorator to peel past.
//
// A ROLE REACHED BY ASSERTION IS NOT A ROLE OUTSIDE THE RULES.
// journalops.Journal is a facade role with a contract tier and a
// per-leg lock like every other; what it has no accessor for is
// the reason issueops.Importer has none either — the capability is
// not on storage.DoltStorage's published surface, so the census
// that would otherwise miss it reads SOURCE rather than reflecting
// over accessors (backend/conformance/role_coverage_scan_test.go).
// The assertion below is what a front door does with such a role,
// not a shortcut around one.
cursor, ok := serveJournalCursor(src)
if ok {
roles.eventsJournal = cursor
@ -763,6 +773,15 @@ func serveIssueRoles(src serveRoleSource, journalEnabled bool) (serveRoles, erro
// is narrower than the whole store UnwrapStore takes. A source that is not a
// store has no decorator to peel — the test stubs are exactly that — so it is
// asked for the seam as it stands.
//
// It survives the journal's promotion to a facade role (journalops.Journal,
// which storage.EventsJournalCursor aliases) unchanged, and deliberately: a
// role with no accessor is reached exactly this way. issueops.Importer set the
// precedent — one accessor, none on the store interface — and the apparatus
// that keeps such a role honest is the conformance census, which parses the
// facade packages for declarations instead of reflecting over the accessors a
// store hands out. There is no accessor for this function to have been
// replaced by.
func serveJournalCursor(src serveRoleSource) (storage.EventsJournalCursor, bool) {
raw := any(src)
if store, ok := src.(storage.DoltStorage); ok {
@ -835,10 +854,14 @@ type serveRoles struct {
memories memoryops.Memories
// eventsJournal is the only role here that comes from a TYPE ASSERTION
// rather than an accessor, because the journal is not part of DoltStorage's
// published surface: it is engine state on a dolt_ignored table that the two
// concrete stores implement and a backend may not. Missing it is a startup
// error rather than a route that 500s, which is the same deal every other
// role here takes.
// published surface: it is a replay feed over engine state on a
// dolt_ignored table that the two concrete stores implement and a backend
// may not. Missing it is a startup error rather than a route that 500s,
// which is the same deal every other role here takes.
//
// It IS a role — journalops.Journal, which storage.EventsJournalCursor
// aliases — with its own contract tier and per-leg lock. Accessorless is
// the issueops.Importer shape, not an exemption; see serveJournalCursor.
eventsJournal storage.EventsJournalCursor
}

View file

@ -560,6 +560,97 @@ now carry a per-row `inner` comparand naming the surface each one must not be
fixture a new namespace cannot satisfy is a signal: the row you add beside it
may be a row that cannot fail.**
## The third namespace
`journalops.Journal` is the durable mutation journal's read side: the
seq-ordered replay feed behind `bd events tail`, `bd events export` and
`GET /v0/beads/events`. It is a third leaf rather than a role in either
existing one because its rows are neither beads nor settings — they are
clone-local engine state on a `dolt_ignore`d table, written in the same
transaction as the mutation they describe, versioned by nothing and replicated
nowhere. A plane whose rows deliberately survive no merge has nothing in common
with the plane that holds the merged data, however similar an id column makes
them look.
Most of what the second namespace decided transferred without argument. Four
things did not, and the first two are the ones a fourth namespace inherits.
**A role can have NO ACCESSOR AT ALL, and that is what makes the census a
source parse rather than a convenience.** Every other role in this tree is
handed out by a method on a store or a provider; this one is reached by TYPE
ASSERTION, because the journal is not on `storage.DoltStorage`'s published
surface and a backend is free not to implement it (`cmd/bd/serve.go`,
`serveJournalCursor`). `issueops.Importer` is the precedent and the warning: it
had no contract case from the day it was written and nothing noticed, precisely
because a reflection-only census can only ask about types something already
names. So the demand side is where an accessorless role is added —
`facadePackages` in `backend/conformance/role_coverage_scan_test.go`, one line
— and from there `TestEveryRoleMethodHasAContractCase` treats it exactly like a
role with three accessors. The supply side needs nothing: `#5499`'s per-leg
lock is ENTRYPOINT-scoped, so the six new `Run…` functions were demanded of all
three legs the moment the contract file existed. Check which of the two gates
your role is invisible to before assuming both.
Nothing else in the accessor apparatus applies, and the absences should read as
decisions: no row in either `role_accessor_decorator_test.go`, because there is
no accessor to decorate; no `RoleFiresHooks` entry, because a read fires none;
no `.golangci.yml` deny entry, for `memoryops`' reason — the body is an `…InTx`
function (`issueops.ReadEventsPageInTx`) that no front door can hold, so there
is no constructor to deny.
**Conditional requiredness is a RESOLVED BOOLEAN the caller hands in, never
something the server works out.** `httpapi.Config.EventsJournal` is the one
role field required conditionally — on `Config.EventsJournalEnabled` — and the
flag is separate from the role because it CANNOT BE INFERRED FROM THE DATA: a
disabled journal presents as zero rows and a head of zero, byte-identical to an
enabled journal nothing has written to yet, so a server without the flag would
answer "you are caught up" to a consumer polling a workspace that will never
emit a record. Activation lives in the target workspace's own config and
environment, which `internal/httpapi` resolves none of. If your role is
optional, ask what tells the difference between "off" and "empty"; if the
answer is nothing, the flag is a field and not an inference.
**Alias FORWARD from the old home when a role is carved out of shipped code.**
The second namespace's section above argued that moving a vocabulary later
costs nothing because a Go alias preserves identity in both directions, and
called the opposite claim a false dichotomy. This is that claim measured. Four
names and a constant moved from `internal/storage` into the leaf and the old
spellings became aliases —
`type EventsJournalCursor = journalops.Journal` and its three siblings — and
the whole tree compiled with **no non-test change anywhere else**: not in
`internal/httpapi`, not in `cmd/bd`, not in any of the four implementations,
not in the enterprise sync. `errors.As` against `*journalops.TruncatedError`
matches an error every leg constructs as `*storage.EventsJournalTruncatedError`,
because they are one type. The direction is what has to be right — the leaf
imports `context` and `fmt` and cannot name `internal/storage`, so the canon
goes down and the alias goes up — and the contract asserts the identity at
runtime on three legs rather than leaving it to be argued from the spec.
**A role may arrive AFTER its front doors, and the leaf is where that gets
written down.** The checklist's usual worry is a role whose command or
operation lands later (`GraphCounter`, `BatchApplier`, `VersionReconciler`).
This is the inverse: the CLI, the HTTP operation and four implementations all
shipped first, against a seam in `internal/storage`, and the role was carved
out of them afterwards. What that buys is exactly what a facade-only slice
buys, one step later: ONE place where the promises are stated and three legs
held to them. What it costs is that the promises have to be reconstructed from
working code rather than written before it — so read the bodies for what they
actually do at every boundary (a checkpoint at or above the head; a limit of 0;
a head after a full prune) and put each answer in the leaf doc, because those
are the cases the shipped tests were least likely to have covered.
**And keep the operator's half OFF the role, deliberately and in writing.**
`storage.EventsJournalAccessor` (read plus prune) and
`storage.EventsJournalConfigurer` (per-instance activation) stayed in
`internal/storage` when the read moved out. The entitlement test from `bd init`
applies and comes back loudly yes: `bd serve` documents itself as publishing
the journal and never retaining it, so handing it a delete would make that
documentation the only thing between a consumer's checkpoint and a prune. The
conformance fixture still needs both — the cases have to create records and
manufacture a truncation — so they arrive as fixture hooks with a comment
saying they are the operator surface being borrowed, not part of what is under
test (`backend/conformance/journal_contract.go`, `JournalFixture.Prune`).
## When all three legs share one body
`issueops.TreeWalker` was the first, and `issueops.MetadataCAS` is the second:

View file

@ -278,8 +278,11 @@ type Config struct {
// EventsJournal is the durable mutation journal's READ side, and the ONE
// role here that is required CONDITIONALLY — which is why it is absent from
// sourceRoles and checked on its own. Like Memories it is not an issueops
// role: the journal is engine state on a dolt_ignored table, not a bead
// query, so its seam lives in internal/storage beside the rows it yields.
// role: the journal is a replay feed over engine state on a dolt_ignored
// table, not a bead query, so it has its own leaf package (journalops,
// whose doc states why). storage.EventsJournalCursor is an ALIAS of
// journalops.Journal, so this field names the role whichever spelling
// reaches it.
//
// Required exactly when EventsJournalEnabled. A workspace that records
// nothing needs no reader, and a storage backend that cannot read the
@ -288,7 +291,7 @@ type Config struct {
// activation to a store. Demanding it unconditionally would make a
// capability nothing uses a precondition for running the server.
//
// It is a storage.EventsJournalCursor and deliberately NOT the wider
// It is the READ role and deliberately NOT the wider
// storage.EventsJournalAccessor the CLI takes. That interface also prunes,
// and a server that is documented to publish the journal and never retain
// it should not be holding a delete it merely promises not to call.

View file

@ -0,0 +1,103 @@
package dolt
import (
"context"
"testing"
"github.com/steveyegge/beads/backend/conformance"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// TestJournalContract runs the Journal contract against the server-backed
// store, which reaches issueops.ReadEventsPageInTx through withReadTx. All
// three legs share that one body, so this is an ENGINE CHECK rather than an
// independent vote — but it is the leg whose engine can actually differ: the
// shared sql-server is where a read that escaped its transaction, or a head
// taken from a second one, would have somewhere to go wrong.
//
// The cases are subtests of one parent so the whole role suite shares one store
// and one copy-on-write branch, and they run in order with no t.Parallel: two
// of them prune the journal — one to nothing — and every case rebaselines off
// the live head (see journal_contract.go's seeding note). setupTestStore
// already marks the PARENT parallel.
func TestJournalContract(t *testing.T) {
fixture, ctx, cleanup := newDoltJournalFixture(t, "jc")
defer cleanup()
t.Run("PagesAreSeqAscendingAndSinceExclusive", func(t *testing.T) {
conformance.RunJournalPagesAreSeqAscendingAndSinceExclusive(t, ctx, fixture)
})
t.Run("HeadArrivesWithItsRowsAndDetectsCaughtUp", func(t *testing.T) {
conformance.RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp(t, ctx, fixture)
})
t.Run("LimitCapsRowsNotHead", func(t *testing.T) {
conformance.RunJournalLimitCapsRowsNotHead(t, ctx, fixture)
})
t.Run("TruncationIsTypedAndNamesTheWindow", func(t *testing.T) {
conformance.RunJournalTruncationIsTypedAndNamesTheWindow(t, ctx, fixture)
})
t.Run("HeadSurvivesAFullPrune", func(t *testing.T) {
conformance.RunJournalHeadSurvivesAFullPrune(t, ctx, fixture)
})
t.Run("EveryMutationKindLandsARow", func(t *testing.T) {
conformance.RunJournalEveryMutationKindLandsARow(t, ctx, fixture)
})
}
func newDoltJournalFixture(t *testing.T, prefix string) (conformance.JournalFixture, context.Context, func()) {
t.Helper()
store, storeCleanup := setupTestStore(t)
ctx, cancel := testContext(t)
// Through the type assertion `bd serve` makes, never the concrete method
// set: the journal is not on storage.DoltStorage, so publishing it IS
// implementing this interface, and a backend that stopped would fail here
// rather than keep compiling against a struct.
cursor, ok := any(store).(storage.EventsJournalCursor)
if !ok {
cancel()
storeCleanup()
t.Fatalf("%T does not implement storage.EventsJournalCursor", store)
}
fixture := conformance.JournalFixture{
IssuePrefix: prefix,
Journal: cursor,
SetJournalEnabled: store.SetEventsJournalEnabled,
Prune: store.PruneEventsJournal,
Mutations: conformance.JournalMutations{
Create: func(ctx context.Context, id string) error {
return store.CreateIssue(ctx, &types.Issue{
ID: id, Title: "t-" + id, IssueType: types.TypeTask, Status: types.StatusOpen,
}, "actor")
},
Update: func(ctx context.Context, id string) error {
return store.UpdateIssue(ctx, id, map[string]any{"title": "renamed " + id}, "actor")
},
Close: func(ctx context.Context, id string) error {
return store.CloseIssue(ctx, id, "done", "actor", "")
},
Delete: func(ctx context.Context, id string) error {
return store.DeleteIssue(ctx, id)
},
AddDependency: func(ctx context.Context, from, to string) error {
return store.AddDependency(ctx, &types.Dependency{
IssueID: from, DependsOnID: to, Type: types.DepBlocks,
}, "actor")
},
RemoveDependency: func(ctx context.Context, from, to string) error {
return store.RemoveDependency(ctx, from, to, "actor")
},
Comment: func(ctx context.Context, id, text string) error {
return store.AddComment(ctx, id, "actor", text)
},
},
}
return fixture, ctx, func() {
// Journaling is instance-scoped, and this store outlives the fixture in
// the shared-database harness. Leaving it on would journal every
// mutation a later test in this package makes.
store.SetEventsJournalEnabled(false)
cancel()
storeCleanup()
}
}

View file

@ -0,0 +1,95 @@
//go:build cgo
package embeddeddolt_test
import (
"context"
"testing"
"github.com/steveyegge/beads/backend/conformance"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/types"
)
// TestJournalContract runs the Journal contract against the embedded store,
// which reaches issueops.ReadEventsPageInTx through its own per-operation
// connection. All three legs share that one body, so this is an ENGINE CHECK
// rather than an independent vote — what it can actually catch here is a
// wrapper that loses the transaction, a withConn arm that commits a read, or a
// backend that stops implementing the seam at all.
//
// One environment for the whole suite, and NO t.Parallel: the journal is
// append-only and workspace-global, two of the cases prune it — one of them to
// nothing — and every case rebaselines off the live head, which only works
// while the subtests run in order (see journal_contract.go's seeding note).
func TestJournalContract(t *testing.T) {
skipUnlessEmbeddedDolt(t)
te := newTestEnv(t, "jc")
ctx := t.Context()
fixture := newEmbeddedJournalFixture(t, te, "jc")
t.Run("PagesAreSeqAscendingAndSinceExclusive", func(t *testing.T) {
conformance.RunJournalPagesAreSeqAscendingAndSinceExclusive(t, ctx, fixture)
})
t.Run("HeadArrivesWithItsRowsAndDetectsCaughtUp", func(t *testing.T) {
conformance.RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp(t, ctx, fixture)
})
t.Run("LimitCapsRowsNotHead", func(t *testing.T) {
conformance.RunJournalLimitCapsRowsNotHead(t, ctx, fixture)
})
t.Run("TruncationIsTypedAndNamesTheWindow", func(t *testing.T) {
conformance.RunJournalTruncationIsTypedAndNamesTheWindow(t, ctx, fixture)
})
t.Run("HeadSurvivesAFullPrune", func(t *testing.T) {
conformance.RunJournalHeadSurvivesAFullPrune(t, ctx, fixture)
})
t.Run("EveryMutationKindLandsARow", func(t *testing.T) {
conformance.RunJournalEveryMutationKindLandsARow(t, ctx, fixture)
})
}
func newEmbeddedJournalFixture(t *testing.T, te *testEnv, prefix string) conformance.JournalFixture {
t.Helper()
store := te.store
// Through the type assertion `bd serve` makes, never the concrete method
// set: the journal is not on storage.DoltStorage, so publishing it IS
// implementing this interface, and a backend that stopped would fail here
// rather than keep compiling against a struct.
cursor, ok := any(store).(storage.EventsJournalCursor)
if !ok {
t.Fatalf("%T does not implement storage.EventsJournalCursor", store)
}
return conformance.JournalFixture{
IssuePrefix: prefix,
Journal: cursor,
SetJournalEnabled: store.SetEventsJournalEnabled,
Prune: store.PruneEventsJournal,
Mutations: conformance.JournalMutations{
Create: func(ctx context.Context, id string) error {
return store.CreateIssue(ctx, &types.Issue{
ID: id, Title: "t-" + id, IssueType: types.TypeTask, Status: types.StatusOpen,
}, "actor")
},
Update: func(ctx context.Context, id string) error {
return store.UpdateIssue(ctx, id, map[string]any{"title": "renamed " + id}, "actor")
},
Close: func(ctx context.Context, id string) error {
return store.CloseIssue(ctx, id, "done", "actor", "")
},
Delete: func(ctx context.Context, id string) error {
return store.DeleteIssue(ctx, id)
},
AddDependency: func(ctx context.Context, from, to string) error {
return store.AddDependency(ctx, &types.Dependency{
IssueID: from, DependsOnID: to, Type: types.DepBlocks,
}, "actor")
},
RemoveDependency: func(ctx context.Context, from, to string) error {
return store.RemoveDependency(ctx, from, to, "actor")
},
Comment: func(ctx context.Context, id, text string) error {
return store.AddComment(ctx, id, "actor", text)
},
},
}
}

View file

@ -19,6 +19,18 @@ import (
// It runs against the real embedded store — the same read path `bd events
// tail --since` takes — so the counter self-heal, the prune floors, and the
// truncation check are all exercised together rather than mocked.
//
// IT SURVIVED THE JOURNAL CONTRACT, and which half survived is the point.
// backend/conformance/journal_contract.go now pins the same restart, retention
// and truncation promises on all three legs — but it pins them on
// journalops.Journal, whose body is issueops.ReadEventsPageInTx. Every read
// below is ReadEventsJournal, which is storage.EventsJournalAccessor's
// list-only read and a SECOND composition of the same parts: it pays for the
// head only where the verdict is ambiguous, because `bd events tail --follow`
// runs it every second. A mutation of one is invisible to the other, so the
// role contract cannot stand in for this file. Its page-path twin,
// TestEventsJournalPageReportsTheHead, could and was deleted with the contract
// that replaced it.
func TestEventsJournalRestartAndRetentionBoundary(t *testing.T) {
env := newTestEnv(t, "trn")
ctx := context.Background()
@ -129,75 +141,14 @@ func TestEventsJournalRestartAndRetentionBoundary(t *testing.T) {
}
}
// TestEventsJournalPageReportsTheHead is the guard for the read that
// GET /v0/beads/events answers from. The head is the whole reason that read
// exists beside ReadEventsJournal — it is how a polling consumer tells "this
// page is the end" from "there is more, keep asking" — so it is checked against
// the substrate rather than assumed.
//
// Three properties, and the last one is the one that would break silently:
// the head reflects every mutation ever journaled; it survives a prune, because
// prune deletes rows and never touches the counter; and a truncated read still
// fails through this path rather than answering with rows and a plausible head.
func TestEventsJournalPageReportsTheHead(t *testing.T) {
env := newTestEnv(t, "pge")
ctx := context.Background()
store := env.store
store.SetEventsJournalEnabled(true)
must := func(err error, what string) {
t.Helper()
if err != nil {
t.Fatalf("%s: %v", what, err)
}
}
for _, id := range []string{"pge-1", "pge-2", "pge-3", "pge-4", "pge-5"} {
must(store.CreateIssue(ctx, &types.Issue{
ID: id, Title: "t-" + id, IssueType: types.TypeTask, Status: types.StatusOpen,
}, "actor"), "create "+id)
}
// A BOUNDED page: the head must describe the journal, not the page. A head
// derived from the last row returned would read as "caught up" here and
// stall a consumer three records short.
page, err := store.ReadEventsJournalPage(ctx, 0, 2)
must(err, "read page")
if len(page.Rows) != 2 {
t.Fatalf("rows = %d, want 2", len(page.Rows))
}
if page.Head != 5 {
t.Fatalf("head = %d, want 5 — the head must be the journal's, not the page's", page.Head)
}
// Caught up: no rows, and the same head.
caughtUp, err := store.ReadEventsJournalPage(ctx, 5, 0)
must(err, "read caught up")
if len(caughtUp.Rows) != 0 || caughtUp.Head != 5 {
t.Fatalf("caught-up page = %+v, want no rows and head 5", caughtUp)
}
// Prune the whole journal. The counter is untouched, so the head stands
// while the rows are gone — which is what lets a fully pruned journal say
// "you are at the end of my history" instead of "I have nothing".
if _, err := store.PruneEventsJournal(ctx, 6, 0, 0); err != nil {
t.Fatalf("prune: %v", err)
}
afterPrune, err := store.ReadEventsJournalPage(ctx, 5, 0)
must(err, "read after prune")
if len(afterPrune.Rows) != 0 || afterPrune.Head != 5 {
t.Fatalf("post-prune page = %+v, want no rows and head 5", afterPrune)
}
// And a checkpoint below the pruned window still FAILS on this path, with
// the same window the CLI's read reports. A page read that answered an
// empty success here would be the silent-loss case the truncation contract
// exists to prevent, reintroduced by the second read plumbing.
_, err = store.ReadEventsJournalPage(ctx, 2, 0)
trunc := requireTruncated(t, err)
if trunc.Since != 2 || trunc.Floor != 6 || trunc.Head != 5 {
t.Fatalf("truncation = %+v, want since 2, floor 6, head 5", trunc)
}
}
// TestEventsJournalPageReportsTheHead was here, and the Journal contract
// replaced it: RunJournalLimitCapsRowsNotHead pins that a bounded page reports
// the JOURNAL's head, RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp pins
// the caught-up answer, RunJournalHeadSurvivesAFullPrune pins the head standing
// over an emptied table, and RunJournalTruncationIsTypedAndNamesTheWindow pins
// the typed failure on the same read. Every one of them now runs on three legs
// where this ran on one, against the same body
// (issueops.ReadEventsPageInTx). See journal_contract_test.go.
func requireTruncated(t *testing.T, err error) *storage.EventsJournalTruncatedError {
t.Helper()

View file

@ -0,0 +1,113 @@
package storage
import (
"errors"
"fmt"
"reflect"
"testing"
"github.com/steveyegge/beads/journalops"
)
// The COMPILE-TIME half, which fires before any test runs. A pointer type is
// identical to another only when the types it points at are identical, so each
// of these conversions is legal exactly while the name on the right is an alias
// of the one on the left.
//
// The pointer form is not decoration. For the three STRUCTS a plain assignment
// would do — two defined types with identical underlying types are not
// assignable to each other — but for the INTERFACE it would be blind, because
// interface-to-interface assignment goes by method set and a de-aliased
// EventsJournalCursor would still accept a journalops.Journal. One form that
// holds for all four is worth more than three that hold and a fourth that
// silently does not.
var (
_ = func(p *journalops.Row) *EventsJournalRow { return p }
_ = func(p *journalops.Page) *EventsJournalPage { return p }
_ = func(p *journalops.TruncatedError) *EventsJournalTruncatedError { return p }
_ = func(p *journalops.Journal) *EventsJournalCursor { return p }
)
// TestJournalNamesAreAliasesNotCopies pins the property the journal's move into
// a leaf package rests on: storage.EventsJournalRow, EventsJournalPage,
// EventsJournalTruncatedError and EventsJournalCursor are the journalops types,
// not lookalikes declared beside them.
//
// It exists because the two ways this breaks fail in OPPOSITE ways, and
// BUILDING THE TREE CATCHES NEITHER. Both readings below were measured, not
// reasoned about.
//
// THE INTERFACE BREAKS SILENTLY AND HARMLESSLY, which is what makes it easy to
// leave. Turning `type EventsJournalCursor = journalops.Journal` into
// `type EventsJournalCursor journalops.Journal` builds the WHOLE TREE clean —
// measured — because every implementation still satisfies both and interface
// satisfaction is structural. Nothing misbehaves at runtime; what is lost is the
// CLAIM, that one role has one name, and with it the guarantee that two callers
// holding the two spellings hold the same thing.
//
// THE ERROR IS WHERE IDENTITY IS BEHAVIORAL, and the dangerous shape is not the
// obvious one. `type EventsJournalTruncatedError journalops.TruncatedError`
// does not inherit Error(), so it stops being an error at every return site and
// the build fails loudly — that one takes care of itself. What compiles in
// silence is the COPY-PASTE TWIN: a struct declared here with the same fields
// and its own Error method. Every leg keeps returning it, every message reads
// identically, and errors.As against *journalops.TruncatedError stops matching
// for everything holding the leaf's spelling — the conformance tier, and any
// consumer downstream of it.
//
// TWO LAYERS CATCH THEM, and the order is worth knowing. The pins above fire
// FIRST, at compile time: with them in place neither break reaches a test run.
// The two tests below are the same properties stated as behavior, and they are
// what stands if the pins are ever removed — measured with them stripped, the
// twin reds BOTH (it is a different type, and it does not cross errors.As) and
// the de-aliased interface reds the first. Keeping both layers is deliberate:
// the pins say it soonest, and the errors.As test is the only place the
// property appears in the form a consumer actually depends on.
//
// The role contract asserts that errors.As half too
// (backend/conformance/journal_contract.go matches *journalops.TruncatedError
// against errors every leg constructs as *EventsJournalTruncatedError), but it
// only runs where a Dolt server or cgo does. These run everywhere, in
// milliseconds, and name the alias as the thing that broke.
func TestJournalNamesAreAliasesNotCopies(t *testing.T) {
for _, alias := range []struct {
name string
storage reflect.Type
leaf reflect.Type
}{
{"EventsJournalRow", reflect.TypeFor[EventsJournalRow](), reflect.TypeFor[journalops.Row]()},
{"EventsJournalPage", reflect.TypeFor[EventsJournalPage](), reflect.TypeFor[journalops.Page]()},
{"EventsJournalTruncatedError", reflect.TypeFor[EventsJournalTruncatedError](), reflect.TypeFor[journalops.TruncatedError]()},
{"EventsJournalCursor", reflect.TypeFor[EventsJournalCursor](), reflect.TypeFor[journalops.Journal]()},
} {
if alias.storage != alias.leaf {
t.Errorf("storage.%s is %s, not the journalops type %s: it is a redeclaration rather than an "+
"alias, so every caller now has two vocabularies for one thing and errors.As matches only one",
alias.name, alias.storage, alias.leaf)
}
}
if EventsJournalTruncatedCode != journalops.TruncatedCode {
t.Errorf("EventsJournalTruncatedCode = %q, want journalops.TruncatedCode (%q): the wire spelling of "+
"a truncation cannot differ by which package a handler imported",
EventsJournalTruncatedCode, journalops.TruncatedCode)
}
}
// TestJournalTruncationCrossesTheAliasUnderErrorsAs is the runtime half, and the
// one that matches how the failure would actually present: a caller holding the
// leaf's spelling classifies an error a storage-side body constructed with the
// alias, through a wrapper, and gets its fields.
func TestJournalTruncationCrossesTheAliasUnderErrorsAs(t *testing.T) {
wrapped := fmt.Errorf("reading the journal: %w",
&EventsJournalTruncatedError{Since: 7, Floor: 12, Head: 40})
var trunc *journalops.TruncatedError
if !errors.As(wrapped, &trunc) {
t.Fatalf("errors.As(%v, **journalops.TruncatedError) did not match an error built as "+
"*storage.EventsJournalTruncatedError; the two spellings have to be one type", wrapped)
}
if trunc.Since != 7 || trunc.Floor != 12 || trunc.Head != 40 {
t.Errorf("window = [%d..%d] after %d, want [12..40] after 7", trunc.Floor, trunc.Head, trunc.Since)
}
}

View file

@ -10,11 +10,11 @@ import (
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/steveyegge/beads/internal/types"
"github.com/steveyegge/beads/issueops"
"github.com/steveyegge/beads/journalops"
"github.com/steveyegge/beads/memoryops"
)
@ -743,117 +743,62 @@ type StateHasher interface {
GetStateHash(ctx context.Context) (string, error)
}
// EventsJournalRow is one raw bd_events_journal row surfaced to the
// `bd events` CLI. IssueJSON is empty when the op is a delete (no surviving
// row); DepJSON is empty for non-dependency ops; CommentJSON is empty for
// non-comment ops. TS is the insert-time timestamp (stamped inside the
// committing transaction) normalized to a string.
type EventsJournalRow struct {
Seq int64
TS string
Op string
IssueID string
IssueJSON string
DepJSON string
CommentJSON string
}
// EventsJournalTruncatedCode is the stable machine-readable code a consumer
// matches on when its checkpoint has fallen below the retained journal window.
const EventsJournalTruncatedCode = "events_journal_truncated"
// EventsJournalTruncatedError reports that a sequential read cannot resume from
// the caller's checkpoint because the rows it needs next were pruned.
// The durable mutation journal's vocabulary lives in the journalops leaf, and
// these four names are ALIASES of it — not copies, and not a compatibility
// shim to be deleted later. The canon is journalops: what a record carries,
// what a page promises and what a truncation means are stated there, once, and
// every citation in this tree points at those symbols rather than repeating
// them here.
//
// Without it, `WHERE seq > since` cannot distinguish "nothing new" from "your
// prefix is gone": a consumer resuming past a prune would either see an empty
// success and stall forever, or silently skip to the current floor and lose
// every record in between. Both are silent data loss, so the read fails loudly
// instead and hands back the window it can actually serve.
// THE ALIAS DIRECTION IS THE LOAD-BEARING PART. journalops imports context and
// fmt and nothing else, so it cannot name anything in this package; this
// package can name it. Declaring the canon down there and aliasing up here is
// what makes the leaf a leaf. And because a Go alias is the SAME TYPE, every
// existing implementation, every errors.As site and every caller compiles
// unchanged across the move — which is why the journal became a role without a
// line of behavior changing.
//
// Floor is the lowest seq still retained, or Head+1 when the journal holds no
// rows at all. Head is the highest seq the counter has ever assigned; it never
// decreases under a prune, so Floor > Head means "fully pruned, caught up to
// Head". A consumer that receives this must decide explicitly — resume from
// Floor-1 and accept the gap, or rebuild from scratch — and the engine does not
// decide for it.
type EventsJournalTruncatedError struct {
// Since is the checkpoint the reported window begins after, which is the
// caller's own checkpoint in every case except one.
//
// When the rows the read can serve start above the caller's checkpoint —
// the ordinary "your prefix was pruned" case — Since IS that checkpoint and
// Floor is the first row still retained.
//
// When the prefix is intact but the retained window has an interior hole
// (a restored or hand-edited table; bd's own prune cannot produce one),
// Since is instead the last seq the engine could serve contiguously from
// the caller's checkpoint, and Floor is where the next intact island
// starts. A batch with BOTH shapes reports the prefix one; the interior
// hole is reported on the next read, once the caller has resumed past the
// first. Every gap is surfaced, one resume at a time.
//
// Since therefore never reports a value BELOW what the caller presented, so
// echoing it back can never make a consumer re-read records it already has.
Since int64
// Floor is the lowest retained seq (Head+1 when nothing is retained).
Floor int64
// Head is the highest seq ever assigned.
Head int64
}
// The two interfaces BELOW these aliases stay declared here on purpose. They
// are the operator's half of the plane — retention and per-instance activation
// — and journalops states why they are deliberately not on the role.
type (
// EventsJournalRow is journalops.Row: one raw bd_events_journal record.
EventsJournalRow = journalops.Row
// EventsJournalPage is journalops.Page: rows plus the journal head.
EventsJournalPage = journalops.Page
// EventsJournalTruncatedError is journalops.TruncatedError: a checkpoint
// below the retained window, carrying the window that can still be served.
EventsJournalTruncatedError = journalops.TruncatedError
// EventsJournalCursor is journalops.Journal: the role a consumer holds to
// page through the journal from a checkpoint.
EventsJournalCursor = journalops.Journal
)
func (e *EventsJournalTruncatedError) Error() string {
return fmt.Sprintf(
"events journal truncated: checkpoint %d is below the retained window [%d..%d]; records %d..%d were pruned",
e.Since, e.Floor, e.Head, e.Since+1, e.Floor-1)
}
// EventsJournalPage is one journal read that also reports how far behind the
// caller is: the rows it asked for, and the head of the journal's history at
// the moment they were read.
//
// The two travel together because they are only meaningful together. Rows alone
// cannot answer "poll again now, or wait?" — a full page might be the end of
// the journal or the first thousand of a hundred thousand — and a head read
// separately could be taken from a different instant and land BELOW the last
// row served, which a consumer reads as "past the end" and stalls on.
//
// Head is the highest seq the counter has ever assigned. It never decreases
// under a prune (see EventsJournalTruncatedError), so it is the journal's
// history rather than its contents: a fully pruned journal reports its rows
// gone and its head unchanged.
type EventsJournalPage struct {
Rows []EventsJournalRow
Head int64
}
// EventsJournalCursor is the READ-ONLY half of the journal seam, for a consumer
// that pages through the journal with a checkpoint.
//
// It is separate from EventsJournalAccessor rather than a method on it because
// of who holds it. Retention is an operator decision made by the workspace
// (`bd events prune` and the automatic bounding); a surface that only PUBLISHES
// the journal — bd serve's GET /v0/beads/events — must not be handed a delete.
// Narrowing the interface is what makes "this surface cannot prune" a fact
// about the type rather than a promise about the handler.
type EventsJournalCursor interface {
// ReadEventsJournalPage returns rows with seq greater than since, ordered
// by seq ascending and capped by limit (0 = no cap), together with the
// journal head read in the same transaction. It returns
// *EventsJournalTruncatedError on the same terms ReadEventsJournal does.
ReadEventsJournalPage(ctx context.Context, since int64, limit int) (EventsJournalPage, error)
}
// EventsJournalTruncatedCode is journalops.TruncatedCode: the stable wire
// spelling of a truncation.
const EventsJournalTruncatedCode = journalops.TruncatedCode
// EventsJournalAccessor reads and prunes the durable events journal
// (bd_events_journal) through the store's own transaction machinery. Unlike
// RawDBAccessor — which only the server-mode store provides — this works on the
// embedded store too, which owns its connections and exposes no stable *sql.DB.
// Callers that need the journal should type-assert to this interface; a caller
// that only reads should ask for EventsJournalCursor instead.
//
// IT IS THE OPERATOR SURFACE, and it is deliberately not the role. Retention is
// a decision the workspace makes, so a caller that only READS the journal asks
// for EventsJournalCursor — journalops.Journal — instead, and a surface
// documented never to retain then cannot prune rather than merely promising not
// to. journalops' package doc states the split; this is the half it excludes.
//
// ReadEventsJournal is also a SECOND read body, not a narrowing of the role's:
// it pays for the head read only in the cases where the truncation verdict is
// ambiguous, because `bd events tail --follow` runs it every second. The two
// share the row query and the verdict (issueops.ComputeEventsTruncation) and
// differ in exactly that, so they are pinned separately.
type EventsJournalAccessor interface {
// ReadEventsJournal returns rows with seq greater than since, ordered by
// seq ascending, optionally capped by limit (0 = no cap). It returns
// *EventsJournalTruncatedError when since sits below the retained window.
// *EventsJournalTruncatedError when since sits below the retained window,
// on the terms journalops.Journal.ReadEventsJournalPage states.
ReadEventsJournal(ctx context.Context, since int64, limit int) ([]EventsJournalRow, error)
// PruneEventsJournal deletes rows with seq below before, honoring the
// retain-days / retain-rows floors (0 = floor disabled), and returns the
@ -867,6 +812,10 @@ type EventsJournalAccessor interface {
// parallel fixtures), and opening one with the journal enabled must not turn it
// on for any other. Callers type-assert; a store that does not implement it
// simply cannot journal.
//
// Activation is operator surface for the reason retention is: it is the
// workspace's answer to "do we record at all", read from the workspace's own
// config, and a consumer holding the read role has no business changing it.
type EventsJournalConfigurer interface {
SetEventsJournalEnabled(enabled bool)
}

View file

@ -0,0 +1,134 @@
package uow
import (
"context"
"testing"
"github.com/steveyegge/beads/backend/conformance"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/domain"
"github.com/steveyegge/beads/internal/types"
)
// TestJournalContract runs the Journal contract against the unit-of-work
// provider, which reaches issueops.ReadEventsPageInTx through
// domain.EventsJournalUseCase. All three legs share that one body, so this is
// an ENGINE CHECK rather than a second vote — and here it is a check on the
// composition, which is the part that genuinely differs: this leg is the only
// one whose read runs inside a unit of work it did not open, so a RunTx where
// RunTxRead belongs would commit a read of a dolt_ignored table.
//
// One provider for the whole suite (each newUOWRoleFixtureProvider boots a real
// Dolt sql-server) and NO t.Parallel: this backend has no per-test
// copy-on-write branch, and two of these cases prune the journal — one to
// nothing — while every case rebaselines off the live head (see
// journal_contract.go's seeding note).
func TestJournalContract(t *testing.T) {
ctx := context.Background()
fixture := newUOWJournalFixture(t, ctx, "jc")
t.Run("PagesAreSeqAscendingAndSinceExclusive", func(t *testing.T) {
conformance.RunJournalPagesAreSeqAscendingAndSinceExclusive(t, ctx, fixture)
})
t.Run("HeadArrivesWithItsRowsAndDetectsCaughtUp", func(t *testing.T) {
conformance.RunJournalHeadArrivesWithItsRowsAndDetectsCaughtUp(t, ctx, fixture)
})
t.Run("LimitCapsRowsNotHead", func(t *testing.T) {
conformance.RunJournalLimitCapsRowsNotHead(t, ctx, fixture)
})
t.Run("TruncationIsTypedAndNamesTheWindow", func(t *testing.T) {
conformance.RunJournalTruncationIsTypedAndNamesTheWindow(t, ctx, fixture)
})
t.Run("HeadSurvivesAFullPrune", func(t *testing.T) {
conformance.RunJournalHeadSurvivesAFullPrune(t, ctx, fixture)
})
t.Run("EveryMutationKindLandsARow", func(t *testing.T) {
conformance.RunJournalEveryMutationKindLandsARow(t, ctx, fixture)
})
}
func newUOWJournalFixture(t *testing.T, ctx context.Context, prefix string) conformance.JournalFixture {
t.Helper()
provider := newUOWRoleFixtureProvider(t, ctx, prefix)
// Through the capability accessor, not NewEventsJournalCursor: a provider
// that stopped offering the role is the regression a constructor call would
// hide.
source, ok := provider.(EventsJournalCursorSource)
if !ok {
t.Fatalf("provider %T does not offer the EventsJournalCursor accessor", provider)
}
cursor, err := source.EventsJournalCursor()
if err != nil {
t.Fatalf("EventsJournalCursor(): %v", err)
}
// Activation is the OTHER half, and it is a different interface on purpose:
// the read role cannot turn itself on, so the fixture reaches for the
// operator surface the way `bd serve` does.
configurer, ok := provider.(storage.EventsJournalConfigurer)
if !ok {
t.Fatalf("provider %T does not implement storage.EventsJournalConfigurer", provider)
}
return conformance.JournalFixture{
IssuePrefix: prefix,
Journal: cursor,
SetJournalEnabled: configurer.SetEventsJournalEnabled,
Prune: func(ctx context.Context, before int64, retainDays, retainRows int) (int64, error) {
// Ephemeral, exactly as `bd events prune` runs it here: the journal
// table is dolt_ignored, so the delete has to persist into the
// working set without minting a Dolt commit.
return RunTxEphemeral(ctx, provider, func(ctx context.Context, uw UnitOfWork) (int64, error) {
return uw.EventsJournalUseCase().Prune(ctx, before, retainDays, retainRows)
})
},
Mutations: conformance.JournalMutations{
Create: func(ctx context.Context, id string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
_, err := uw.IssueUseCase().CreateIssue(ctx, domain.CreateIssueParams{
Issue: &types.Issue{
ID: id, Title: "t-" + id, IssueType: types.TypeTask, Status: types.StatusOpen,
},
ExplicitID: id,
CreateOnly: true,
}, "actor")
return "create " + id, err
})
},
Update: func(ctx context.Context, id string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
return "update " + id, uw.IssueUseCase().UpdateIssue(ctx, id,
map[string]any{"title": "renamed " + id}, "actor")
})
},
Close: func(ctx context.Context, id string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
_, err := uw.IssueUseCase().CloseIssue(ctx, id, domain.CloseIssueParams{Reason: "done"}, "actor")
return "close " + id, err
})
},
Delete: func(ctx context.Context, id string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
_, err := uw.IssueUseCase().DeleteIssue(ctx, id, "actor")
return "delete " + id, err
})
},
AddDependency: func(ctx context.Context, from, to string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
return "dep add " + from, uw.DependencyUseCase().AddDependency(ctx, &types.Dependency{
IssueID: from, DependsOnID: to, Type: types.DepBlocks,
}, "actor")
})
},
RemoveDependency: func(ctx context.Context, from, to string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
return "dep remove " + from, uw.DependencyUseCase().RemoveDependency(ctx, from, to, "actor")
})
},
Comment: func(ctx context.Context, id, text string) error {
return RunTx(ctx, provider, func(ctx context.Context, uw UnitOfWork) (string, error) {
_, err := uw.CommentUseCase().AddCommentToIssue(ctx, id, "actor", text)
return "comment " + id, err
})
},
},
}
}

67
journalops/doc.go Normal file
View file

@ -0,0 +1,67 @@
// Package journalops describes the workspace's DURABLE MUTATION JOURNAL: the
// seq-ordered record of every committed bead mutation that `bd events tail`,
// `bd events export` and GET /v0/beads/events read, and that an external
// consumer replays to rebuild its own copy of the graph.
//
// It is a sibling of issueops and memoryops rather than more methods on either,
// and the reason is not that the role rule forbids appending (it does): a
// journal record is not a bead and not a setting. It is ENGINE STATE — rows in
// the clone-local, dolt_ignored bd_events_journal table, written inside the
// same transaction as the mutation they describe (migration 0064) — so it
// survives no merge, travels through no pull, and describes the history of ONE
// clone rather than the state of a project. A plane whose rows are deliberately
// unversioned has nothing in common with a plane whose rows are the versioned
// data, however similar an id column makes them look.
//
// WHAT A CONSUMER IS ACTUALLY HOLDING. The journal is a REPLAY FEED, not a
// query surface. Every promise here — seq order, the exclusive checkpoint, the
// head that travels with its rows, the typed truncation — exists so that a
// consumer polling from a stored checkpoint can prove it has missed nothing.
// That is why a checkpoint below the retained window is an ERROR rather than a
// smaller answer: at the SQL level "nothing new" and "your prefix was deleted"
// are one empty result set, and a reader that guessed would either stall
// forever or skip to the current floor and lose every record in between. Both
// are silent data loss, which is the one failure a replay feed must never ship.
//
// THE VOCABULARY OF Row.Op IS NOT DECLARED HERE, and that is deliberate. The
// engine journals seven ops and the public event vocabulary is six — the
// seventh, a comment write, is journaled but mints no wire event, because a
// comment's effect is already visible in the next issue snapshot. That
// reconciliation belongs to the emitter and to the wire contract
// (internal/storage/issueops: WireEventOps, EngineOnlyEventOps, IsWireEventOp),
// which are the two places that have to agree about it. Naming a closed set
// here would put a third copy one op away from drifting, and would tell a
// caller that reads records that it is entitled to an opinion about which ones
// a projector forwards. Op is a string, and what the strings mean is stated
// where they are minted.
//
// THERE IS NO PRUNE ON THIS ROLE, and no activation switch either. Both exist —
// `bd events prune`, the automatic bounding, and the per-instance enable that
// binds journaling to one open workspace — and both are OPERATOR surface,
// carried by storage.EventsJournalAccessor and storage.EventsJournalConfigurer
// beside the rows. The split is about who holds what: a surface that only
// PUBLISHES the journal, which is what `bd serve` documents itself as, must not
// be one line away from a delete. Narrowing the interface is what makes "this
// server cannot prune" a fact about the type rather than a promise about the
// handler. A role that carried a retention decision it is documented never to
// take would give that fact away for nothing.
//
// AND THERE IS NO ACCESSOR. Every other role in this tree is handed out by a
// method on a store or a unit-of-work provider; this one is reached by TYPE
// ASSERTION, because the journal is not part of storage.DoltStorage's published
// surface and a backend is free not to implement it at all. issueops.Importer is
// the precedent — a role with one accessor and none on the store interface —
// and the consequence is the same: the conformance census finds this role by
// parsing source rather than by reflecting over accessors
// (backend/conformance/role_coverage_scan_test.go), which is what keeps it from
// being invisible to the exhaustiveness gate.
//
// WHAT THIS PACKAGE IMPORTS: context and fmt, and nothing else — narrower even
// than memoryops, which needs beadserrors for a sentinel. This plane has no
// deterministic request-validation refusal to classify: a checkpoint is an
// int64, a limit is an int, and the one refusal that exists is the truncation,
// which is a TYPED error carrying a window rather than a sentinel a caller
// matches by identity. If a validation refusal ever appears here it aliases
// beadserrors.ErrValidation rather than minting a twin, for the reasons
// memoryops/errors.go gives.
package journalops

63
journalops/errors.go Normal file
View file

@ -0,0 +1,63 @@
package journalops
import "fmt"
// TruncatedCode is the stable machine-readable code a consumer matches on when
// its checkpoint has fallen below the retained journal window.
//
// It is the wire spelling of *TruncatedError, for the front doors that cannot
// hand back a Go type: the HTTP problem document carries it, and a client that
// speaks JSON dispatches on this string exactly as an in-process caller
// dispatches on errors.As. Both name one condition, which is why the string
// lives beside the error rather than in a handler.
const TruncatedCode = "events_journal_truncated"
// TruncatedError reports that a sequential read cannot resume from the caller's
// checkpoint because the records it needs next were pruned.
//
// Without it, `WHERE seq > since` cannot distinguish "nothing new" from "your
// prefix is gone": a consumer resuming past a prune would either see an empty
// success and stall forever, or silently skip to the current floor and lose
// every record in between. Both are silent data loss, so the read fails loudly
// instead and hands back the window it can actually serve.
//
// Floor is the lowest seq still retained, or Head+1 when the journal holds no
// rows at all. Head is the highest seq the counter has ever assigned; it never
// decreases under a prune, so Floor > Head means "fully pruned, caught up to
// Head". A consumer that receives this must decide explicitly — resume from
// Floor-1 and accept the gap, or rebuild from scratch — and the engine does not
// decide for it.
//
// It is returned as a POINTER, which is what errors.As matches against, and
// implementations return it rather than wrapping it in a message of their own
// making. A caller reads the FIELDS; the string below is for humans.
type TruncatedError struct {
// Since is the checkpoint the reported window begins after, which is the
// caller's own checkpoint in every case except one.
//
// When the rows the read can serve start above the caller's checkpoint —
// the ordinary "your prefix was pruned" case — Since IS that checkpoint and
// Floor is the first row still retained.
//
// When the prefix is intact but the retained window has an interior hole
// (a restored or hand-edited table; bd's own prune cannot produce one),
// Since is instead the last seq the engine could serve contiguously from
// the caller's checkpoint, and Floor is where the next intact island
// starts. A batch with BOTH shapes reports the prefix one; the interior
// hole is reported on the next read, once the caller has resumed past the
// first. Every gap is surfaced, one resume at a time.
//
// Since therefore never reports a value BELOW what the caller presented, so
// echoing it back can never make a consumer re-read records it already has.
Since int64
// Floor is the lowest retained seq (Head+1 when nothing is retained).
Floor int64
// Head is the highest seq ever assigned.
Head int64
}
func (e *TruncatedError) Error() string {
return fmt.Sprintf(
"events journal truncated: checkpoint %d is below the retained window [%d..%d]; records %d..%d were pruned",
e.Since, e.Floor, e.Head, e.Since+1, e.Floor-1)
}

114
journalops/journal.go Normal file
View file

@ -0,0 +1,114 @@
package journalops
import "context"
// Row is one raw record of the durable mutation journal.
//
// IssueJSON is empty when the op is a delete (no surviving row to snapshot);
// DepJSON is empty for non-dependency ops; CommentJSON is empty for non-comment
// ops. TS is the insert-time timestamp, stamped inside the committing
// transaction and normalized to a string.
//
// TS IS NOT MONOTONE IN SEQ, and a consumer that sorts by it is wrong. It is
// client-stamped, so two writers against one SQL server — or one writer whose
// clock steps back — can commit seq N with a later TS than seq N+1. Seq is the
// order; TS is a label. The retention floors resolve the same way, to a seq
// bound rather than a per-row age test, for exactly this reason.
type Row struct {
Seq int64
TS string
Op string
IssueID string
IssueJSON string
DepJSON string
CommentJSON string
}
// Page is one journal read that also reports how far behind the caller is: the
// rows it asked for, and the head of the journal's history at the moment they
// were read.
//
// The two travel together because they are only meaningful together. Rows alone
// cannot answer "poll again now, or wait?" — a full page might be the end of the
// journal or the first thousand of a hundred thousand — and a head read
// separately could be taken from a different instant and land BELOW the last row
// served, which a consumer reads as "past the end" and stalls on.
//
// Head is the highest seq the counter has ever assigned. It never decreases
// under a prune, so it is the journal's HISTORY rather than its CONTENTS: a
// fully pruned journal reports its rows gone and its head unchanged. Deriving
// it from the rows in hand would make it a fact about the page instead, which
// reads as "caught up" at the end of every bounded read.
type Page struct {
Rows []Row
Head int64
}
// Journal is the workspace's durable mutation journal, read as pages from a
// caller-held checkpoint — see the package doc for what that plane is and why
// its retention and activation are not on this role.
//
// IT IS ONE ROLE WITH ONE METHOD, and being born with one is the point rather
// than an accident of what shipped first. The plane's other verbs are operator
// decisions the read holder is deliberately not entitled to, so there is no
// second shape of this question waiting to be appended: `bd events prune` and
// the per-instance activation switch answer to storage.EventsJournalAccessor
// and storage.EventsJournalConfigurer, which the workspace holds and a
// publishing surface does not.
//
// The read/write entitlement test the role rule applies (can one caller be
// entitled to the read and not the write?) comes back YES here, loudly, which
// is why the split exists at all: `bd serve` publishes GET /v0/beads/events and
// documents that it never retains, so handing it a delete would make that
// documentation the only thing standing between a consumer's checkpoint and a
// prune.
//
// Result values are unspecified when error is non-nil — in particular a
// truncation carries its window in the ERROR and not in a partial Page, so a
// caller that ignores the error and reads Rows gets nothing rather than a
// plausible-looking suffix. Implementations never mutate caller-owned values,
// having none to mutate.
type Journal interface {
// ReadEventsJournalPage returns the journal records after the caller's
// checkpoint, with the head of the journal's history.
//
// SINCE IS EXCLUSIVE. The answer is the records whose seq is strictly
// greater than since, so a consumer stores the seq it last processed and
// hands back exactly that, with no arithmetic and no risk of an off-by-one
// replay. A since of 0 therefore means "from the beginning of history",
// because seq starts at 1.
//
// ROWS ARE SEQ-ASCENDING, and gapless within one answer. Seq is drawn from
// a counter inside the mutation's own transaction rather than assigned at
// insert, so the order rows are served in is the order they COMMITTED in —
// which is the order a replay has to apply them in, and is not what an
// AUTO_INCREMENT would have given.
//
// LIMIT CAPS THE ROWS AND NOT THE HEAD. A limit of 0 means uncapped: this
// role imposes no ceiling of its own, because the caller that pages a
// hundred thousand records out to a file is as legitimate as the one
// polling for ten. A FRONT DOOR may impose one — the HTTP handler behind
// GET /v0/beads/events caps its page, and that cap is the HANDLER's promise
// to its clients, stated on the operation — and a bounded page still
// reports the journal's head, which is how the caller learns there is more.
//
// HEAD COMES FROM THE SAME TRANSACTION AS THE ROWS, and is read after them.
// Reading it first would let a mutation commit in between and yield a head
// BELOW the last row served; taking it from a separate transaction would
// allow the same thing. This order can only report a head equal to or ahead
// of the last row returned, which a poller acts on correctly either way.
// Head never decreases across calls, prune included.
//
// A CHECKPOINT AT OR ABOVE HEAD IS CAUGHT UP: no rows, the head, and a nil
// error. That is the answer a poller gets on a quiet workspace and the
// answer it gets at the end of a fully pruned journal, and it is
// deliberately the same answer, because both mean "there is nothing after
// your checkpoint".
//
// A CHECKPOINT BELOW THE RETAINED WINDOW IS A TYPED FAILURE: the returned
// error matches errors.As against *TruncatedError, carrying the window the
// implementation can still serve. Resuming from that window's Floor-1
// succeeds and yields the retained records; the caller decides between that
// gap and a rebuild, and no implementation decides for it. See TruncatedError.
ReadEventsJournalPage(ctx context.Context, since int64, limit int) (Page, error)
}