Compare commits

...
Sign in to create a new pull request.

2 commits

Author SHA1 Message Date
Eddie the Engineer
538cec7eb8 fix(storage): apply review fixes for journal shape activation (adopt-pr #7211)
Review fixes (iteration 1), from the review synthesis on #7211:

- Blocker: the shape refusal applied to every open. With the journal
  enabled and bd_events_journal missing (or short a required column),
  eventsjournal.Apply refused read-only, preview, `bd dolt commit` and
  `bd dolt pull` opens as well as writers, so `bd list`, `bd create
  --dry-run` and `bd dolt commit` went from rc=0 on the base to rc=1.
  Apply now takes the open's posture and refuses only an open that is
  going to write beads. cmd/bd reads the posture off dolt.Config
  (opensForBeadWrites: not ReadOnly, Preview, LenientOpen or
  RemoteSyncOpen) and, for the proxied provider, off its options
  (uow.OpensForBeadWrites). A new uow.WithWorkingSetReconcile option
  marks the `bd dolt commit` open; rootProviderOptions sets it and it
  changes nothing about the open itself. The doctor repairs that open
  through openBeadMutatingStore all write beads and stay refused. The
  rest of bd doctor opens unactivated and still runs, as does #7213's
  journal-shape --fix (DatabaseVersion's raw dolt.NewFromConfig open).

- Deviation from the required fix: an open that writes no beads keeps
  the journal ENABLED, not disabled. A classified read is writable
  underneath. Its defer-wake sweep un-defers expired issues, and with
  the journal off that update would commit with no journal row. With
  the journal on, the write uses the canonical column list and fails
  on its INSERT, rolling back with the wake. The sweep's error is
  already advisory, so the read still exits 0. For the same reason,
  the fold that scoped a transaction as unjournaled whenever the probe
  produced no shape is gone. It lived in the dolt store, transaction.go,
  embedded commitConn and uow BeginTx. The base journals every write of
  an enabled store on the canonical shape, and a store enabled directly
  (bypassing Apply) must not silently stop journaling. T2.4
  (TestMissingRequiredColumnRefusesAtOpen) now requires the write to
  fail and its row to be absent; it used to require the write to
  succeed unrecorded.

- Major: a probe that failed for any reason refused the open, and all
  three probes ran unbounded on context.Background(). The new
  issueops.JournalActivationError refuses only a *JournalShapeError.
  Any other probe failure (a timeout, a dropped connection, the
  embedded probe's own OpenSQL) prints one warning, suppressed by quiet
  mode, and leaves a nil shape, which writes canonically. Each probe,
  including the embedded probe's connection, is bounded by
  issueops.JournalShapeProbeTimeout (30s); SetEventsJournalEnabled
  takes no context.

- Major: TestTxEndReleasesJournalScope passed whether or not the scope
  leaked. Its probe was unarmed, so activation failed and the fold
  scoped the transaction as disabled. Each subtest now arms the
  canonical probe, requires no activation error, and requires
  clearJournalShape to be released with the connection.

- Minor (test evidence): Apply/ActivateStore and the uow server path
  are now pinned directly.
  TestActivateStoreRefusesAnUnsupportedShapeOnlyForBeadWritingOpens
  checks that a bead-writing open refuses and closes the store, and
  that any other open returns it with the journal enabled. New sqlmock
  tests check three cases: a gci-shaped probe's INSERT names neither
  actor nor comment_json; an unsupported shape still journals and
  surfaces the INSERT error; and a failed probe does not refuse
  activation. TestRootProviderOptions and two TestOpensForBeadWrites
  tables pin the posture wiring.

Validation:
- e2e, built binaries (review head vs this commit), on a disposable
  embedded workspace with bd_events_journal dropped and
  BD_EVENTS_JOURNAL=1:
  - bd list --readonly, bd list, bd create --dry-run and bd dolt
    commit: rc=1 -> rc=0.
  - bd create: refused on both.
  - bd doctor: rc=0 on both.
  - bd ready with an expired defer: rc=1 -> rc=0, with "defer-wake
    sweep skipped: journal: record update ...: table not found"; the
    issue stays deferred.
- Ten go test -overlay mutants are each killed by the test named for
  them:
  - three that drop the uow transaction's journal or shape scope
    release (TestTxEndReleasesJournalScope)
  - restoring the fold in BeginTx, and unscoping its probed shape
  - dropping the probe classification, or Apply's posture gate
  - dropping the reconcile clause from uow.OpensForBeadWrites, or the
    LenientOpen or RemoteSyncOpen clause from opensForBeadWrites
- go test (gms_pure_go, cgo) passes for:
  - internal/storage/uow (whole package)
  - internal/eventsjournal
  - internal/storage/issueops
  - the embedded journal tests
  - the cmd/bd posture tests and construction guard
  - the cmd/bd events-journal e2e tests (BEADS_TEST_EMBEDDED_DOLT=1,
    BEADS_TEST_PROXIED_SERVER=1): proxied serve activation, routed
    create, delete actor, auto-prune, journal read/stream, all 12 run
    with none skipped
- make ci-pr-lint: 0 issues.

Left open (minor, not merge gates):
- An unsupported shape still refuses opens that are writable without
  writing beads: bd config set, embedded bd dolt push, and the
  proxied-server provider opens made with no options (backup, init,
  clean-databases, serve). bd prime is refused too, but degrades to
  "Skipped: beads storage unavailable" and exits 0.
- bd doctor gets its journal-shape finding in #7213.
- Unchanged: the per-command degraded-shape warning, lossy
  degraded-window rows, the once-per-open probe (A2's 0028 replay),
  reader adaptivity while the journal is disabled, and the five-site
  optional-column list (#7213 exports OptionalJournalColumns()). The
  three-atomics nit is also unchanged.

Review fixes (iteration 2, non-gating minors), from the iteration-2
synthesis on #7211 (scorecard 912/1000, approve):

- Minor (error handling): a probe that fails for a reason other than
  the table's shape still leaves a nil shape, so the store writes
  canonically. The warning now names what that costs: if the table is
  short a journal column, every journaled write through the store
  fails until it is reopened. Quiet mode still suppresses it on
  stderr, but it is now logged under BD_DEBUG or --verbose. Refusing a
  writable open on such a failure, or re-probing after the first
  failed journal write, is left as a follow-up: refusing on any probe
  failure was the iteration-1 major.

- Minor (test evidence): the posture arguments and the DoltStore
  server arm are now pinned.
  - TestRootOpensPassTheirPostureToActivation reads, as source text,
    the posture each root open passes to activation at eight sites:
    the newDoltStore and newRegisteredBackendStore build-tag twins,
    both proxied provider factories, and the root pre-run's
    rootProviderOptions and newRegisteredBackendStore calls.
  - TestDoltTransactionWritesTheProbedJournalShape and
    TestScopedRawTxWritesTheProbedJournalShape are the DoltStore
    counterparts of TestBeginTxWritesTheProbedJournalShape. On a table
    without actor and comment_json, a journaled write through
    runDoltTransactionRecording, and one through
    scopeEventsJournalTransaction, names only the probed columns.

- Not changed (minor, proxied pull/push posture): proxied `bd dolt
  pull` and `bd dolt push` never reach journal activation. The proxied
  capability gate (validateProxyRegistryBeforeProvider) refuses both
  as unimplemented before the root pre-run opens a provider, so a
  uow.WithRemoteSync() option would be dead code. What remains is the
  direct-path refusal already left open above (bd config set, embedded
  bd dolt push).

- Not changed (minor, doctor pointer): JournalShapeError still says
  "see `bd doctor`", and bd doctor has no journal-shape check until
  #7213. #7213 has to merge right behind this PR; until it does, the
  escape is BD_EVENTS_JOURNAL=0.

Validation (iteration 2):
- Mutants:
  - go test -overlay: dropping the shape binding in
    runDoltTransactionRecording fails only
    TestDoltTransactionWritesTheProbedJournalShape; dropping the one in
    scopeEventsJournalTransaction fails only
    TestScopedRawTxWritesTheProbedJournalShape; not caching the probed
    shape fails both.
  - On disk, restored after: replacing all eight posture arguments
    with constants fails all eight pins, each naming its site.
- go test (gms_pure_go) passes for internal/storage/issueops,
  internal/storage/uow, internal/eventsjournal, internal/storage/dolt
  (BEADS_TEST_SKIP=dolt), and the cmd/bd construction guard and
  posture tests.
- go vet, git diff --check, make bazel-sync-check and make ci-pr-lint:
  clean.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:40:01 +00:00
Julian Knutsen
010d981435 feat(storage): shape-adaptive events journal writes and reads (PR A1)
bd_events_journal writes and reads now probe the table's actual column
set on each store open and adapt accordingly, instead of assuming every
clone has comment_json and actor. Older table shapes (e.g. a clone that
predates those columns) no longer break every journaled mutation; the
adaptive path drops only the unsupported payload fields and logs once,
rather than failing every write.

This is PR A1 from BEADS-JOURNAL-PLAN.md §4 (T2.1-T2.9): the shape probe
runs once per store open (internal/storage/issueops/journal_shape.go)
and the dolt/embeddeddolt write and read paths, plus the uow layer that
wraps them, consult it. PR A2 (the converging ignored-plane migration
and `bd doctor` check) lands separately on top of this, since it depends
on the adaptive I/O introduced here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-05 23:40:01 +00:00
31 changed files with 1440 additions and 68 deletions

View file

@ -156,6 +156,7 @@
15 1 TestMergeWithStrategy_OursKeepsLocal
15 2 TestActiveDatabaseSizeScopesToActiveEmbeddedDatabase
15 2 TestAdaptiveInsert_GciShape
15 2 TestCleanGitRemoteCacheGarbage
15 2 TestCreateBatchFastPathsMatchPerRow_Embedded
15 2 TestDepRekeyMarkerStillInIgnoredSeries
@ -230,6 +231,7 @@
15 4 TestIgnoredCursorHealNeverResurrectsRolledBackVersions
15 4 TestMain
15 4 TestMemoriesContract
15 4 TestMissingRequiredColumnRefusesAtOpen
15 4 TestPromoteClearsNoHistory
15 4 TestRemoteVerbPlainRemoteKeepsAmbientEnv
15 4 TestReopenIssue
@ -237,6 +239,7 @@
15 4 TestWithPeerAuth_EnvFallbackDoesNotObservePeerCredentials
15 5 TestActiveDatabaseSizeFailsForMissingEmbeddedDatabase
15 5 TestAdaptiveRead_GciShape
15 5 TestBlockingAnnotatorContract
15 5 TestBootstrapperContract
15 5 TestBuildDSN_WithDatabase
@ -406,6 +409,7 @@
15 10 TestGraphWalkPlansHonorJoinHints
15 10 TestHelperProcess
15 10 TestIgnoredCursorHealResumeRespectsOperatorOverride
15 10 TestJournalShapeProbeOncePerOpen
15 10 TestReclaimRefusesForeignReplicaLease
15 10 TestRemoteEntryPointsUseVersionControlOps
15 10 TestTransactionCleanupBeforeSQLCommitRemainsDefinite

View file

@ -41,9 +41,11 @@ func openBeadMutatingStoreCreating(ctx context.Context, beadsDir string) (*dolt.
// activated applies the workspace's journal setting and closes the store if it
// cannot be honored. A failed open passes straight through, so the typed-nil
// store never reaches the activation.
// store never reaches the activation. Every caller writes beads, so a journal
// table the journal cannot run against refuses the repair rather than letting
// it mutate unrecorded.
func activated(beadsDir string, store *dolt.DoltStore, err error) (*dolt.DoltStore, error) {
if _, err = eventsjournal.ActivateStore(beadsDir, store, err); err != nil {
if _, err = eventsjournal.ActivateStore(beadsDir, true, store, err); err != nil {
return nil, err
}
return store, nil

View file

@ -10,6 +10,7 @@ import (
"github.com/steveyegge/beads/internal/debug"
"github.com/steveyegge/beads/internal/eventsjournal"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/uow"
)
@ -45,8 +46,19 @@ func eventsJournalEnabled() bool {
// activateEventsJournalStore is eventsjournal.ActivateStore under the name the
// construction guard matches. Kept as a wrapper rather than called directly so
// cmd/bd has one spelling of the idiom and the guard has one name to look for.
func activateEventsJournalStore(beadsDir string, s storage.DoltStorage, err error) (storage.DoltStorage, error) {
return eventsjournal.ActivateStore(beadsDir, s, err)
func activateEventsJournalStore(beadsDir string, writesBeads bool, s storage.DoltStorage, err error) (storage.DoltStorage, error) {
return eventsjournal.ActivateStore(beadsDir, writesBeads, s, err)
}
// opensForBeadWrites reports whether a store opened from cfg is going to write
// beads — the posture that decides whether an events-journal table the
// journal cannot run against refuses the open (eventsjournal.Apply). Read-only
// and preview opens write none. Neither do the working-set-reconcile (bd dolt
// commit, bd vc commit) and remote-sync (bd dolt pull) opens: they commit or
// merge a working set rather than mutate beads, and refusing them would lock
// an operator out of exactly the commands that recover a workspace.
func opensForBeadWrites(cfg *dolt.Config) bool {
return !cfg.ReadOnly && !cfg.Preview && !cfg.LenientOpen && !cfg.RemoteSyncOpen
}
// eventsJournalMaintenanceRunner returns the plumbing auto-prune should run
@ -241,12 +253,13 @@ func reportEventsJournalAutoPrune(n int64, err error) {
// activateEventsJournalProvider is the same for a unit-of-work provider — bd's
// second write plumbing, with its own transactions and its own activation.
func activateEventsJournalProvider(ctx context.Context, beadsDir string, p uow.UnitOfWorkProvider, err error) (uow.UnitOfWorkProvider, error) {
// writesBeads is the provider's posture (uow.OpensForBeadWrites).
func activateEventsJournalProvider(ctx context.Context, beadsDir string, writesBeads bool, p uow.UnitOfWorkProvider, err error) (uow.UnitOfWorkProvider, error) {
if err != nil || p == nil {
return p, err
}
configurer, _ := p.(storage.EventsJournalConfigurer)
if cfgErr := eventsjournal.Apply(configurer, eventsjournal.EnabledFor(beadsDir)); cfgErr != nil {
if cfgErr := eventsjournal.Apply(configurer, eventsjournal.EnabledFor(beadsDir), writesBeads); cfgErr != nil {
closeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), providerCloseTimeout)
defer cancel()
_ = p.Close(closeCtx)

View file

@ -4,6 +4,7 @@ import (
"go/ast"
"go/parser"
"go/token"
"go/types"
"io/fs"
"path/filepath"
"sort"
@ -11,6 +12,7 @@ import (
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage/dolt"
"github.com/steveyegge/beads/internal/storage/journalscan"
)
@ -208,6 +210,114 @@ func TestEveryStoreConstructionActivatesTheEventsJournal(t *testing.T) {
}
}
// TestOpensForBeadWrites pins which root opens an unsupported journal shape may
// refuse (eventsjournal.Apply): only one that is going to write beads. Read-only
// and preview opens are not refused, and neither are the working-set-reconcile
// and remote-sync opens an operator recovers a workspace with.
func TestOpensForBeadWrites(t *testing.T) {
for _, tt := range []struct {
name string
cfg dolt.Config
want bool
}{
{name: "ordinary write open", want: true},
{name: "read-only", cfg: dolt.Config{ReadOnly: true}},
{name: "preview", cfg: dolt.Config{Preview: true}},
{name: "working-set reconcile", cfg: dolt.Config{LenientOpen: true}},
{name: "remote sync", cfg: dolt.Config{RemoteSyncOpen: true}},
// Writable underneath for the defer-wake sweep, but opened read-only:
// a sweep write against an unsupported shape fails on its INSERT.
{name: "classified read", cfg: dolt.Config{ReadOnly: true, ClassifiedRead: true}},
} {
t.Run(tt.name, func(t *testing.T) {
if got := opensForBeadWrites(&tt.cfg); got != tt.want {
t.Fatalf("opensForBeadWrites = %t, want %t", got, tt.want)
}
})
}
}
// TestRootOpensPassTheirPostureToActivation pins the posture each root open
// hands to activation. TestOpensForBeadWrites and the uow package pin the
// classifiers, and the guard above pins that every site activates — but a site
// that passed a constant would satisfy both. `true` would refuse every
// read-only, preview, and recovery open on an unsupported journal shape again;
// `false` would let a bead-writing open through onto a shape where every
// journaled write fails. So the argument itself is compared, as source text,
// in the same syntactic way the guard works.
//
// It covers the root opens, whose posture depends on the command. The other
// activating factories (newDoltStoreFromConfig, the personal-migration
// planning store, bd doctor's bead-mutating repairs) pass a fixed `true` and
// are not pinned here.
func TestRootOpensPassTheirPostureToActivation(t *testing.T) {
for _, pin := range []struct {
file, fn, call string // fn "" searches the whole file
arg int
want string
}{
{"store_factory.go", "newDoltStore", "activateEventsJournalStore", 1, "opensForBeadWrites(cfg)"},
{"store_factory_nocgo.go", "newDoltStore", "activateEventsJournalStore", 1, "opensForBeadWrites(cfg)"},
{"store_factory.go", "newRegisteredBackendStore", "activateEventsJournalStore", 1, "writesBeads"},
{"store_factory_nocgo.go", "newRegisteredBackendStore", "activateEventsJournalStore", 1, "writesBeads"},
{"uow_factory.go", "newExternalProxiedServerUOWProvider", "activateEventsJournalProvider", 2, "uow.OpensForBeadWrites(opts...)"},
{"uow_factory.go", "newManagedProxiedServerUOWProvider", "activateEventsJournalProvider", 2, "uow.OpensForBeadWrites(opts...)"},
// The root pre-run, inside rootCmd's literal: the proxied provider reads
// its posture back out of these options, and the registry arm is handed
// it directly.
{"main.go", "", "rootProviderOptions", 2, "isWorkingSetReconcileCommand(cmd)"},
{"main.go", "", "newRegisteredBackendStore", 4, "opensForBeadWrites(doltCfg)"},
} {
where := pin.file
if pin.fn != "" {
where += ":" + pin.fn
}
args := callArgs(t, pin.file, pin.fn, pin.call, pin.arg)
if len(args) == 0 {
t.Errorf("%s: found no call to %s — the pin is checking nothing; update it", where, pin.call)
}
for _, got := range args {
if got != pin.want {
t.Errorf("%s: %s is passed %s, want %s — the open's posture must come from what the open does",
where, pin.call, got, pin.want)
}
}
}
}
// callArgs returns argument arg, as source text, of every call to the
// package-local function call inside file's function fn, or anywhere in file
// when fn is "".
func callArgs(t *testing.T, file, fn, call string, arg int) []string {
t.Helper()
parsed, err := parser.ParseFile(token.NewFileSet(), file, nil, 0)
if err != nil {
t.Fatalf("parse %s: %v", file, err)
}
var out []string
for _, decl := range parsed.Decls {
if fn != "" {
if fd, ok := decl.(*ast.FuncDecl); !ok || fd.Name.Name != fn {
continue
}
}
ast.Inspect(decl, func(n ast.Node) bool {
c, ok := n.(*ast.CallExpr)
if !ok {
return true
}
if ident, ok := c.Fun.(*ast.Ident); ok && ident.Name == call {
if arg >= len(c.Args) {
t.Fatalf("%s: %s has %d arguments; the pin reads argument %d", file, call, len(c.Args), arg)
}
out = append(out, types.ExprString(c.Args[arg]))
}
return true
})
}
return out
}
// exemptionFor resolves a site key against the exemption map: an exact match
// first, then the longest package prefix. It returns the exemption's own key so
// the staleness check can tell which entry is still earning its place.

View file

@ -1919,7 +1919,7 @@ var rootCmd = &cobra.Command{
// mode is where that is least visible, not where it is acceptable.
if proxiedServerMode {
p, err := newProxiedServerUOWProvider(rootCtx, beadsDir, databaseOverride,
rootProviderOptions(previewMode, useReadOnly)...)
rootProviderOptions(previewMode, useReadOnly, isWorkingSetReconcileCommand(cmd))...)
if err != nil {
// Same typed rendering the store path gets below: a schema
// skew or a migration-gate refusal here carries a whole
@ -2039,7 +2039,7 @@ var rootCmd = &cobra.Command{
// Dolt manages these files itself; external interference is never safe.
if _, ok := backends.Lookup(cfg.GetBackend()); ok {
store, err = newRegisteredBackendStore(rootCtx, cfg.GetBackend(), beadsDir, useReadOnly)
store, err = newRegisteredBackendStore(rootCtx, cfg.GetBackend(), beadsDir, useReadOnly, opensForBeadWrites(doltCfg))
} else {
store, err = newDoltStore(rootCtx, doltCfg)
}

View file

@ -27,7 +27,7 @@ import (
// journal if it has one enabled — and the setting is read from there, not from
// the workspace bd was launched in.
func openMigrationPlanningStore(ctx context.Context, planningBeadsDir string) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(planningBeadsDir, s, err) }()
defer func() { s, err = activateEventsJournalStore(planningBeadsDir, true, s, err) }()
return dolt.New(ctx, &dolt.Config{
Path: doltserver.ResolveDoltDir(planningBeadsDir),
BeadsDir: planningBeadsDir,

View file

@ -55,8 +55,9 @@ func usesProxiedServer() bool {
// construction path. A read-only open still goes through activation: a
// registered backend decides for itself what read-only means, and refusing to
// offer it the setting would be this factory guessing on its behalf.
func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readOnly bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
// writesBeads is the root open's posture (opensForBeadWrites).
func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readOnly, writesBeads bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, writesBeads, s, err) }()
backend, ok := backends.Lookup(name)
if !ok {
return nil, fmt.Errorf("storage backend %q is not registered", name)
@ -68,7 +69,7 @@ func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readO
}
func newDoltStore(ctx context.Context, cfg *dolt.Config) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(cfg.BeadsDir, s, err) }()
defer func() { s, err = activateEventsJournalStore(cfg.BeadsDir, opensForBeadWrites(cfg), s, err) }()
if cfg.ProxiedServer {
return nil, errProxiedStoreUnrouted()
}
@ -155,7 +156,7 @@ func acquireEmbeddedLock(beadsDir string, serverMode bool) (util.Unlocker, error
// exception is BEADS_DOLT_SHARED_SERVER, which is machine-global in every
// resolver in the tree (see sharedServerModeForWorkspace).
func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
defer func() { s, err = activateEventsJournalStore(beadsDir, true, s, err) }()
cfg, err := configfile.Load(beadsDir)
if err != nil {
// A present-but-unloadable metadata.json must not degrade to the

View file

@ -33,8 +33,8 @@ func usesProxiedServer() bool {
// factory: identical, since the backend registry does not depend on CGO. It
// exists here too so the root pre-run's registry arm has one activating
// construction path in both builds.
func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readOnly bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readOnly, writesBeads bool) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, writesBeads, s, err) }()
backend, ok := backends.Lookup(name)
if !ok {
return nil, fmt.Errorf("storage backend %q is not registered", name)
@ -48,7 +48,7 @@ func newRegisteredBackendStore(ctx context.Context, name, beadsDir string, readO
// newDoltStore applies events-journal activation for the same reason its CGO
// twin does — see the note at the top of events_journal.go.
func newDoltStore(ctx context.Context, cfg *dolt.Config) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(cfg.BeadsDir, s, err) }()
defer func() { s, err = activateEventsJournalStore(cfg.BeadsDir, opensForBeadWrites(cfg), s, err) }()
if cfg.ProxiedServer {
return nil, errProxiedStoreUnrouted()
}
@ -65,7 +65,7 @@ func acquireEmbeddedLock(_ string, _ bool) (util.Unlocker, error) {
// newDoltStoreFromConfig creates a SQL-server-backed storage backend from config.
func newDoltStoreFromConfig(ctx context.Context, beadsDir string) (s storage.DoltStorage, err error) {
defer func() { s, err = activateEventsJournalStore(beadsDir, s, err) }()
defer func() { s, err = activateEventsJournalStore(beadsDir, true, s, err) }()
cfg, err := configfile.Load(beadsDir)
if err != nil {
// Name the real cause: without this, a present-but-unloadable

View file

@ -138,21 +138,25 @@ type sqlServerUOWTopology struct {
}
// rootProviderOptions is the CLI-side half of the open-posture policy: it
// turns the root pre-run's two classifications into the uow.ProviderOption
// slice the proxied-server provider is opened with. Extracted (rather than
// inlined at the call site) so the wiring has something to unit test; the
// previous inline form had no test that would fail if a refactor dropped it.
// turns the root pre-run's classifications into the uow.ProviderOption slice
// the proxied-server provider is opened with. Extracted (rather than inlined
// at the call site) so the wiring has something to unit test; the previous
// inline form had no test that would fail if a refactor dropped it.
//
// The two are not exclusive but preview is stronger, and passing both would
// say two different things about an open that can only have one posture:
// preview neither creates nor migrates, while readOnly opens normally and only
// changes how a refused migration is handled. Preview wins.
func rootProviderOptions(preview, readOnly bool) []uow.ProviderOption {
// They are not exclusive but preview is stronger, and passing several would
// say different things about an open that can only have one posture: preview
// neither creates nor migrates, while readOnly opens normally and only changes
// how a refused migration is handled. Preview wins, then readOnly. reconcile
// (bd dolt commit) changes nothing about the open; it only tells events-journal
// activation that the command writes no beads (uow.OpensForBeadWrites).
func rootProviderOptions(preview, readOnly, reconcile bool) []uow.ProviderOption {
switch {
case preview:
return []uow.ProviderOption{uow.WithPreview()}
case readOnly:
return []uow.ProviderOption{uow.WithReadOnly()}
case reconcile:
return []uow.ProviderOption{uow.WithWorkingSetReconcile()}
default:
return nil
}
@ -449,7 +453,7 @@ func resolveServerModeUOWTopologyWithTransportResolver(ctx context.Context, bead
// the paths it resolves (root, log) are the same ones proxied mode uses because
// both modes root their server at the same directory.
func newExternalProxiedServerUOWProvider(ctx context.Context, beadsDir string, topology sqlServerUOWTopology, opts ...uow.ProviderOption) (p uow.UnitOfWorkProvider, err error) {
defer func() { p, err = activateEventsJournalProvider(ctx, beadsDir, p, err) }()
defer func() { p, err = activateEventsJournalProvider(ctx, beadsDir, uow.OpensForBeadWrites(opts...), p, err) }()
rootPath, err := resolveProxiedServerRootPath(beadsDir)
if err != nil {
return nil, fmt.Errorf("newExternalProxiedServerUOWProvider: resolve root path: %w", err)
@ -489,7 +493,7 @@ func newExternalProxiedServerUOWProvider(ctx context.Context, beadsDir string, t
}
func newManagedProxiedServerUOWProvider(ctx context.Context, beadsDir string, topology sqlServerUOWTopology, opts ...uow.ProviderOption) (p uow.UnitOfWorkProvider, err error) {
defer func() { p, err = activateEventsJournalProvider(ctx, beadsDir, p, err) }()
defer func() { p, err = activateEventsJournalProvider(ctx, beadsDir, uow.OpensForBeadWrites(opts...), p, err) }()
// Resolve and hardened-probe the external dolt binary before spawning
// it: an env/sidecar override that is explicitly named but broken
// should fail loudly here rather than surface as a confusing spawn

View file

@ -7,42 +7,51 @@ import (
"testing"
"github.com/steveyegge/beads/internal/configfile"
"github.com/steveyegge/beads/internal/storage/uow"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestRootProviderOptions pins the CLI-side wiring the reviewer flagged as
// untested: the root pre-run turns its previewMode/useReadOnly classification
// into providerOpts by calling this function, and a refactor that dropped or
// inverted that could not fail any existing test. uow.providerOptions is
// unexported, so this cannot poke inside the returned uow.ProviderOption
// values (see internal/storage/uow/preview_provider_test.go's
// untested: the root pre-run turns its previewMode/useReadOnly/working-set
// classification into providerOpts by calling this function, and a refactor
// that dropped or inverted that could not fail any existing test.
// uow.providerOptions is unexported, so this cannot poke inside the returned
// uow.ProviderOption values (see internal/storage/uow/preview_provider_test.go's
// TestApplyProviderOptions for the same-package introspection); it instead
// pins what an external caller can observe — how many options each posture
// yields — which is what the CLI wiring is responsible for.
// yields, and whether events-journal activation will treat the open as one
// that writes beads — which is what the CLI wiring is responsible for.
func TestRootProviderOptions(t *testing.T) {
for _, tt := range []struct {
name string
preview, readOnly bool
want int
name string
preview, readOnly, reconcile bool
want int
wantWritesBeads bool
}{
{name: "ordinary write open", want: 0},
{name: "ordinary write open", want: 0, wantWritesBeads: true},
{name: "preview", preview: true, want: 1},
{name: "read-only", readOnly: true, want: 1},
// bd dolt commit: an ordinary open that writes no beads, so a journal
// table the journal cannot run against must not refuse it.
{name: "working-set reconcile", reconcile: true, want: 1},
// Preview is the stronger posture and must win: it neither creates nor
// migrates, while read-only opens normally.
{name: "preview wins over read-only", preview: true, readOnly: true, want: 1},
} {
t.Run(tt.name, func(t *testing.T) {
opts := rootProviderOptions(tt.preview, tt.readOnly)
opts := rootProviderOptions(tt.preview, tt.readOnly, tt.reconcile)
if len(opts) != tt.want {
t.Fatalf("rootProviderOptions(%t, %t) len = %d, want %d", tt.preview, tt.readOnly, len(opts), tt.want)
t.Fatalf("rootProviderOptions(%t, %t, %t) len = %d, want %d", tt.preview, tt.readOnly, tt.reconcile, len(opts), tt.want)
}
for i, o := range opts {
if o == nil {
t.Fatalf("rootProviderOptions(%t, %t)[%d] is nil", tt.preview, tt.readOnly, i)
t.Fatalf("rootProviderOptions(%t, %t, %t)[%d] is nil", tt.preview, tt.readOnly, tt.reconcile, i)
}
}
if got := uow.OpensForBeadWrites(opts...); got != tt.wantWritesBeads {
t.Fatalf("uow.OpensForBeadWrites(rootProviderOptions(%t, %t, %t)...) = %t, want %t", tt.preview, tt.readOnly, tt.reconcile, got, tt.wantWritesBeads)
}
})
}
}

View file

@ -114,7 +114,7 @@ func parseBool(raw string, present bool) (bool, bool) {
// results:
//
// func newX(...) (s storage.DoltStorage, err error) {
// defer func() { s, err = eventsjournal.ActivateStore(beadsDir, s, err) }()
// defer func() { s, err = eventsjournal.ActivateStore(beadsDir, writesBeads, s, err) }()
// ... open and return ...
// }
//
@ -123,13 +123,14 @@ func parseBool(raw string, present bool) (bool, bool) {
// syntactic unit — which is what lets the construction guard check the property
// structurally instead of chasing a call graph. A failed open passes through
// untouched; a failed activation closes the store rather than returning one
// that would mutate unrecorded.
func ActivateStore(beadsDir string, s storage.DoltStorage, err error) (storage.DoltStorage, error) {
// that would mutate unrecorded. writesBeads is the open's posture, passed
// through to Apply.
func ActivateStore(beadsDir string, writesBeads bool, s storage.DoltStorage, err error) (storage.DoltStorage, error) {
if err != nil || s == nil {
return s, err
}
configurer, _ := storage.UnwrapStore(s).(storage.EventsJournalConfigurer)
if cfgErr := Apply(configurer, EnabledFor(beadsDir)); cfgErr != nil {
if cfgErr := Apply(configurer, EnabledFor(beadsDir), writesBeads); cfgErr != nil {
_ = s.Close()
return nil, cfgErr
}
@ -146,7 +147,19 @@ func ActivateStore(beadsDir string, s storage.DoltStorage, err error) (storage.D
// the journal is that a consumer can trust its cursor, and a plumbing that
// records nothing while reporting success is the one outcome that breaks that
// trust invisibly. A disabled workspace accepts any plumbing.
func Apply(configurer storage.EventsJournalConfigurer, enabled bool) error {
//
// It also fails there when the configurer DOES support the journal but its
// table's shape does not (storage.EventsJournalShapeChecker, PR A1) — but
// only for an open that is going to write beads (writesBeads). There a
// missing required column refuses at open rather than letting
// SetEventsJournalEnabled succeed and every write fail afterward, the gci
// failure mode this check replaces. An open that will not write beads (a
// read, a preview, a working-set commit, a pull) has no bead write to
// protect, so the shape does not refuse it: the journal stays enabled on the
// canonical column list, exactly as before adaptive I/O, and the rare
// journaled write such an open does make (a classified read's defer-wake
// sweep) fails loudly on its INSERT rather than landing unrecorded.
func Apply(configurer storage.EventsJournalConfigurer, enabled, writesBeads bool) error {
if configurer == nil {
if enabled {
return fmt.Errorf("storage backend does not support the events journal")
@ -154,5 +167,13 @@ func Apply(configurer storage.EventsJournalConfigurer, enabled bool) error {
return nil
}
configurer.SetEventsJournalEnabled(enabled)
if !writesBeads {
return nil
}
if checker, ok := configurer.(storage.EventsJournalShapeChecker); ok {
if err := checker.EventsJournalActivationError(); err != nil {
return err
}
}
return nil
}

View file

@ -1,12 +1,14 @@
package eventsjournal
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
)
func writeWorkspace(t *testing.T, yaml string) string {
@ -125,21 +127,21 @@ func (s *journalableStore) SetEventsJournalEnabled(enabled bool) {
// leaving a trace. A workspace that did NOT ask keeps working on any backend,
// because nothing is being promised.
func TestApplyFailsClosedOnAPlumbingThatCannotJournal(t *testing.T) {
err := Apply(nil, true)
err := Apply(nil, true, true)
if err == nil {
t.Fatal("Apply(nil, true) returned nil: an enabled workspace on a plumbing that cannot journal must fail the open")
t.Fatal("Apply(nil, true, true) returned nil: an enabled workspace on a plumbing that cannot journal must fail the open")
}
if !strings.Contains(err.Error(), "does not support the events journal") {
t.Errorf("error does not name the cause: %v", err)
}
if err := Apply(nil, false); err != nil {
t.Errorf("Apply(nil, false) = %v; a disabled workspace must accept any plumbing", err)
if err := Apply(nil, false, true); err != nil {
t.Errorf("Apply(nil, false, true) = %v; a disabled workspace must accept any plumbing", err)
}
recorder := &journalableStore{}
if err := Apply(recorder, true); err != nil {
t.Fatalf("Apply(configurer, true) = %v", err)
if err := Apply(recorder, true, true); err != nil {
t.Fatalf("Apply(configurer, true, true) = %v", err)
}
if !recorder.set || !recorder.enabled {
t.Errorf("activation was not applied: set=%v enabled=%v", recorder.set, recorder.enabled)
@ -153,7 +155,7 @@ func TestActivateStoreClosesAStoreItCannotActivate(t *testing.T) {
t.Setenv(EnvVar, "1")
closed := &closeRecordingStore{}
got, err := ActivateStore(writeWorkspace(t, ""), closed, nil)
got, err := ActivateStore(writeWorkspace(t, ""), true, closed, nil)
if err == nil {
t.Fatal("ActivateStore succeeded on a store that cannot journal")
}
@ -167,11 +169,63 @@ func TestActivateStoreClosesAStoreItCannotActivate(t *testing.T) {
// A failed open passes straight through, untouched — the activation must
// not mask the real error or try to close a store that was never opened.
sentinel := errOpenFailed
if _, err := ActivateStore("", nil, sentinel); err != sentinel {
if _, err := ActivateStore("", true, nil, sentinel); err != sentinel {
t.Errorf("ActivateStore rewrote a failed open: %v", err)
}
}
// shapeCheckingStore is a journalable store whose activation probe reported
// activationErr (storage.EventsJournalShapeChecker).
type shapeCheckingStore struct {
journalableStore
activationErr error
closed bool
}
func (s *shapeCheckingStore) EventsJournalActivationError() error { return s.activationErr }
func (s *shapeCheckingStore) Close() error {
s.closed = true
return nil
}
// TestActivateStoreRefusesAnUnsupportedShapeOnlyForBeadWritingOpens pins the
// posture split on a table the journal cannot run against.
//
// An open that is going to write beads must refuse, and close the store:
// otherwise the open succeeds and every write fails one at a time afterward,
// the gci failure mode. An open that will not write beads must NOT refuse —
// bd list, a preview, bd dolt commit and bd dolt pull all worked against such
// a table before adaptive I/O — and it must leave the journal ENABLED, so a
// journaled write it does make (the defer-wake sweep) fails loudly on its
// INSERT instead of committing with no journal row.
func TestActivateStoreRefusesAnUnsupportedShapeOnlyForBeadWritingOpens(t *testing.T) {
t.Setenv(EnvVar, "1")
beadsDir := writeWorkspace(t, "")
shapeErr := &issueops.JournalShapeError{Table: "bd_events_journal", Missing: []string{"op"}}
writer := &shapeCheckingStore{activationErr: shapeErr}
got, err := ActivateStore(beadsDir, true, writer, nil)
if !errors.Is(err, issueops.ErrJournalShapeUnsupported) {
t.Fatalf("bead-writing open: ActivateStore error = %v; want the shape refusal", err)
}
if got != nil || !writer.closed {
t.Errorf("bead-writing open: a refused store must be closed, not returned (returned=%v closed=%v)", got != nil, writer.closed)
}
reader := &shapeCheckingStore{activationErr: shapeErr}
got, err = ActivateStore(beadsDir, false, reader, nil)
if err != nil {
t.Fatalf("non-writing open: ActivateStore error = %v; the shape must not refuse an open that writes no beads", err)
}
if got != reader || reader.closed {
t.Errorf("non-writing open: the store must be returned open (returned=%v closed=%v)", got == reader, reader.closed)
}
if !reader.set || !reader.enabled {
t.Errorf("non-writing open: journal left disabled (set=%v enabled=%v); a write it makes would commit unrecorded", reader.set, reader.enabled)
}
}
type closeRecordingStore struct {
unjournalableStore
closed bool
@ -190,6 +244,7 @@ func (*openError) Error() string { return "open failed" }
// compile-time proof the doubles above stand in for a real store.
var (
_ storage.DoltStorage = (*unjournalableStore)(nil)
_ storage.EventsJournalConfigurer = (*journalableStore)(nil)
_ storage.DoltStorage = (*unjournalableStore)(nil)
_ storage.EventsJournalConfigurer = (*journalableStore)(nil)
_ storage.EventsJournalShapeChecker = (*shapeCheckingStore)(nil)
)

View file

@ -260,6 +260,7 @@ go_test(
"journal_contract_test.go",
"journal_coverage_test.go",
"journal_scope_completeness_test.go",
"journal_shape_scope_test.go",
"journal_test.go",
"keyset_search_test.go",
"label_rename_test.go",

View file

@ -0,0 +1,109 @@
package dolt
import (
"context"
"errors"
"fmt"
"regexp"
"testing"
sqlmock "github.com/DATA-DOG/go-sqlmock"
"github.com/stretchr/testify/require"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
)
// These are the DoltStore counterparts of the provider's
// TestBeginTxWritesTheProbedJournalShape: on a table that never gained actor
// or comment_json (gas-city-inc's lineage), a journaled write names only the
// columns SetEventsJournalEnabled's probe found. The store binds its cached
// shape to a transaction in two places — runDoltTransactionRecording, behind
// the whole doltTransaction mutator surface, and scopeEventsJournalTransaction,
// behind every raw-transaction writer — and TestEveryRawTxJournalScopeIsScopedOrExempt
// checks only the activation half of either. Dropping a shape binding falls
// back to the canonical INSERT, which is the Error 1054 that rolled back every
// journaled write on that lineage.
// degradedJournalInsert is the INSERT a table missing actor and comment_json gets.
var degradedJournalInsert = regexp.QuoteMeta(
"INSERT INTO bd_events_journal (seq, ts, op, issue_id, issue_json, dep_json) VALUES")
// newDegradedJournalStore returns a store whose journal was activated against
// a table missing only the optional actor and comment_json columns.
func newDegradedJournalStore(t *testing.T) (*DoltStore, sqlmock.Sqlmock) {
t.Helper()
db, mock, err := sqlmock.New()
require.NoError(t, err)
t.Cleanup(func() { _ = db.Close() })
columns := sqlmock.NewRows([]string{"COLUMN_NAME"})
for _, column := range []string{"seq", "ts", "op", "issue_id", "issue_json", "dep_json"} {
columns.AddRow(column)
}
mock.ExpectQuery("SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS").
WithArgs("bd_events_journal").
WillReturnRows(columns)
store := &DoltStore{db: db}
store.SetEventsJournalEnabled(true)
require.NoError(t, store.EventsJournalActivationError(),
"a table missing only OPTIONAL columns is degraded, not unsupported")
return store, mock
}
// expectDegradedJournalWrite arms one journal row: the seq allocation, then
// the INSERT naming only the columns the probe found.
func expectDegradedJournalWrite(mock sqlmock.Sqlmock) {
mock.ExpectExec("UPDATE bd_events_seq SET next_seq").WillReturnResult(sqlmock.NewResult(0, 1))
mock.ExpectQuery("SELECT next_seq FROM bd_events_seq").
WillReturnRows(sqlmock.NewRows([]string{"next_seq"}).AddRow(7))
mock.ExpectExec(degradedJournalInsert).WillReturnResult(sqlmock.NewResult(0, 1))
}
func TestDoltTransactionWritesTheProbedJournalShape(t *testing.T) {
store, mock := newDegradedJournalStore(t)
mock.ExpectQuery(regexp.QuoteMeta("SELECT active_branch()")).
WillReturnRows(sqlmock.NewRows([]string{"active_branch()"}).AddRow("main"))
mock.ExpectBegin()
expectDegradedJournalWrite(mock)
mock.ExpectRollback()
// The callback always returns, and stops before the commit: what is pinned
// here is the INSERT, not the Dolt commit sequence after it. It must not
// fail the test itself — FailNow inside the callback would skip the
// rollback, and the deferred close of the pinned connection would then
// wait on the open transaction forever.
errStop := errors.New("stop before commit")
var writeErr error
ctx := context.Background()
_, err := store.runDoltTransactionRecording(ctx, "test", func(tx storage.Transaction) error {
dtx, ok := tx.(*doltTransaction)
if !ok {
return fmt.Errorf("callback got a %T, want *doltTransaction", tx)
}
writeErr = issueops.RecordDeleteInTx(ctx, dtx.regularTx, "bd-1", "test-actor")
return errStop
})
require.ErrorIs(t, err, errStop)
require.NoError(t, writeErr)
require.NoError(t, mock.ExpectationsWereMet())
}
func TestScopedRawTxWritesTheProbedJournalShape(t *testing.T) {
store, mock := newDegradedJournalStore(t)
mock.ExpectBegin()
expectDegradedJournalWrite(mock)
mock.ExpectRollback()
ctx := context.Background()
tx, err := store.db.BeginTx(ctx, nil)
require.NoError(t, err)
clearJournalScope := store.scopeEventsJournalTransaction(tx)
writeErr := issueops.RecordDeleteInTx(ctx, tx, "bd-1", "test-actor")
clearJournalScope()
_ = tx.Rollback()
require.NoError(t, writeErr)
require.NoError(t, mock.ExpectationsWereMet())
}

View file

@ -308,6 +308,7 @@ var _ storage.Compactor = (*DoltStore)(nil)
var _ storage.SchemaMigrator = (*DoltStore)(nil)
var _ storage.ExternalRefHistoryQuerier = (*DoltStore)(nil)
var _ storage.EventsJournalConfigurer = (*DoltStore)(nil)
var _ storage.EventsJournalShapeChecker = (*DoltStore)(nil)
var _ storage.VersionedHistoryConfigurer = (*DoltStore)(nil)
// DoltStore implements the Storage interface using Dolt
@ -320,6 +321,17 @@ type DoltStore struct {
// eventsJournalEnabled activates the durable events journal for THIS store
// instance only (storage.EventsJournalConfigurer); never process-global.
eventsJournalEnabled atomic.Bool
// journalShape caches the ONE INFORMATION_SCHEMA probe SetEventsJournalEnabled
// runs on activation (issueops.ProbeJournalShape, PR A1). Nil means the
// journal is disabled or the probe produced no shape; transactions then get
// the canonical column list — see journalActivationErr.
journalShape atomic.Pointer[issueops.JournalShape]
// journalActivationErr holds the probe's *issueops.JournalShapeError when
// SetEventsJournalEnabled(true) found a table shape the journal cannot run
// against (a missing required column, or a missing table). Nil otherwise,
// including when the probe could not run at all
// (issueops.JournalActivationError). See storage.EventsJournalShapeChecker.
journalActivationErr atomic.Pointer[error]
// versionedHistoryEnabled activates dual-write issue-version history for
// THIS store instance only (storage.VersionedHistoryConfigurer); never
// process-global.
@ -1083,6 +1095,13 @@ func (s *DoltStore) withReadTx(ctx context.Context, fn func(tx *sql.Tx) error) e
return fmt.Errorf("begin read tx: %w", err)
}
defer func() { _ = tx.Rollback() }()
// ReadEventsJournal/ReadEventsJournalPage read through this helper, and
// the column list they SELECT depends on the probed shape, not on
// whether the journal is enabled for writes — scope the shape only, so
// readEventsRowsInTx sees this store's actual table instead of
// defaulting to canonicalJournalShape (PR A1).
clearShape := issueops.ScopeEventsJournalShape(tx, s.journalShape.Load())
defer clearShape()
return fn(tx)
})
}
@ -1395,12 +1414,56 @@ func logBlockedRecheckFailure(ctx context.Context, pending issueops.BlockedReche
}
// SetEventsJournalEnabled activates the journal for this store instance only.
//
// Activating probes bd_events_journal's actual shape ONCE
// (issueops.ProbeJournalShape, PR A1), bounded by
// issueops.JournalShapeProbeTimeout, and caches the result for this
// instance's lifetime; EventsJournalActivationError reports whether that
// probe found a shape the journal cannot run against. A probe that could not
// run degrades to the canonical shape (issueops.JournalActivationError).
// Deactivating clears both and skips the probe entirely — a disabled store
// accepts any shape.
func (s *DoltStore) SetEventsJournalEnabled(enabled bool) {
s.eventsJournalEnabled.Store(enabled)
if !enabled {
s.journalShape.Store(nil)
s.journalActivationErr.Store(nil)
return
}
ctx, cancel := context.WithTimeout(context.Background(), issueops.JournalShapeProbeTimeout)
defer cancel()
shape, err := issueops.ProbeJournalShape(ctx, s.db)
s.journalShape.Store(shape)
if activationErr := issueops.JournalActivationError(err); activationErr != nil {
s.journalActivationErr.Store(&activationErr)
return
}
s.journalActivationErr.Store(nil)
}
// EventsJournalActivationError implements storage.EventsJournalShapeChecker:
// nil unless the last SetEventsJournalEnabled(true) call found a table shape
// the journal cannot run against.
func (s *DoltStore) EventsJournalActivationError() error {
if p := s.journalActivationErr.Load(); p != nil {
return *p
}
return nil
}
// scopeEventsJournalTransaction binds this store's cached activation state and
// probed shape to tx. The activation switch deliberately does NOT depend on
// the shape: an enabled store whose probe produced no shape writes with the
// canonical column list, so a table the journal cannot run against fails the
// write loudly instead of letting the mutation commit with no journal row
// (storage.EventsJournalShapeChecker).
func (s *DoltStore) scopeEventsJournalTransaction(tx *sql.Tx) func() {
return issueops.ScopeEventsJournalTransaction(tx, s.eventsJournalEnabled.Load())
clearActivation := issueops.ScopeEventsJournalTransaction(tx, s.eventsJournalEnabled.Load())
clearShape := issueops.ScopeEventsJournalShape(tx, s.journalShape.Load())
return func() {
clearShape()
clearActivation()
}
}
// SetVersionedHistoryEnabled activates dual-write issue-version history for

View file

@ -241,6 +241,18 @@ func (s *DoltStore) runDoltTransactionRecording(ctx context.Context, commitMsg s
}
clearJournalScope := issueops.ScopeEventsJournalTransaction(regularTx, journalEnabled)
defer clearJournalScope()
// Shape counterpart of the activation switch just above (PR A1):
// insertEventRow/readEventsRowsInTx reach this store's probed shape only
// through a tx-scoped value, never through the store itself, so without
// this call every write on this path — CreateIssue, UpdateIssue, comments,
// the whole doltTransaction mutator surface reached from
// runDoltTransactionRecording — would silently fall back to
// canonicalJournalShape and defeat adaptive I/O for this store's main
// write path. The switch does not depend on the shape: a nil shape (no
// probe result) writes canonically, as scopeEventsJournalTransaction
// explains.
clearJournalShape := issueops.ScopeEventsJournalShape(regularTx, s.journalShape.Load())
defer clearJournalShape()
// Versioned history binds to the SAME regular transaction the mutation runs
// in, for the same reason the journal does: RecordVersionInTx no-ops unless
// this scope is set. Without it every issues-plane mutation routed through

View file

@ -159,6 +159,7 @@ go_test(
"journal_concurrency_test.go",
"journal_contract_test.go",
"journal_retention_test.go",
"journal_shape_adaptive_test.go",
"journal_truncation_test.go",
"label_list_all_benchmark_test.go",
"label_rename_test.go",

View file

@ -0,0 +1,221 @@
//go:build cgo
package embeddeddolt_test
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
)
// Adaptive I/O for bd_events_journal (BEADS-JOURNAL-PLAN.md §4.2a/§4.3, PR
// A1). These tests reproduce gas-city-inc's exact shape — a table built
// before `comment_json` and `actor` ever existed — directly on an
// otherwise-ordinary embedded store, and prove the writer and reader adapt to
// it instead of failing every journaled write (T2.1/T2.2), that adaptation
// probes the shape once per activation rather than once per write (T2.3),
// and that a table missing a REQUIRED column reports a typed activation
// error — which refuses a bead-writing open (eventsjournal.Apply) — rather
// than silently disabling the journal (T2.4).
// newGciShapeStore returns a store whose bd_events_journal has had
// comment_json and actor dropped, reproducing gas-city-inc's shape
// (BEADS-JOURNAL-PLAN.md §1.1: "gci.bd_events_journal has the columns seq,
// ts, op, issue_id, issue_json, dep_json" — no actor, no comment_json). The
// journal is NOT activated; callers that want to observe activation itself
// (the probe, its warning, its cached result) call SetEventsJournalEnabled
// themselves.
func newGciShapeStore(t *testing.T, prefix string) *testEnv {
t.Helper()
te := newTestEnv(t, prefix)
ctx := context.Background()
te.exec(t, ctx, "ALTER TABLE bd_events_journal DROP COLUMN comment_json")
te.exec(t, ctx, "ALTER TABLE bd_events_journal DROP COLUMN actor")
return te
}
// TestAdaptiveInsert_GciShape is T2.1: on the gci shape, create, update and
// comment all succeed, the rows land, and the comment payload is dropped with
// exactly one warning. A fixed INSERT column list (the pre-A1 behavior) fails
// every one of these three writes, because each one names either actor or
// comment_json.
func TestAdaptiveInsert_GciShape(t *testing.T) {
te := newGciShapeStore(t, "ai")
ctx := context.Background()
stderr := captureStderr(t, func() {
te.store.SetEventsJournalEnabled(true)
})
if err := te.store.EventsJournalActivationError(); err != nil {
t.Fatalf("activation on the gci shape (missing only OPTIONAL columns) must succeed, got: %v", err)
}
if n := strings.Count(stderr, "events journal:"); n != 1 {
t.Fatalf("activation warning count = %d, want exactly 1 (one warning per store, not per write):\n%s", n, stderr)
}
for _, want := range []string{"actor", "comment_json"} {
if !strings.Contains(stderr, want) {
t.Errorf("activation warning does not name missing column %q:\n%s", want, stderr)
}
}
issue := &types.Issue{Title: "gci create", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := te.store.CreateIssue(ctx, issue, "tester"); err != nil {
t.Fatalf("CreateIssue on gci shape: %v", err)
}
if err := te.store.UpdateIssue(ctx, issue.ID, map[string]interface{}{"title": "gci updated"}, "tester"); err != nil {
t.Fatalf("UpdateIssue on gci shape: %v", err)
}
if _, err := te.store.AddIssueComment(ctx, issue.ID, "tester", "gci comment"); err != nil {
t.Fatalf("AddIssueComment on gci shape: %v", err)
}
rows, err := te.store.ReadEventsJournal(ctx, 0, 0)
if err != nil {
t.Fatalf("ReadEventsJournal on gci shape: %v", err)
}
var ops []string
for _, r := range rows {
if r.IssueID != issue.ID {
continue
}
ops = append(ops, r.Op)
// Dropped, not merely blank: the column does not exist on this shape,
// so the adaptive reader must never have tried to select it.
if r.Actor != "" {
t.Errorf("row op=%s: Actor = %q, want \"\" (column does not exist on this shape)", r.Op, r.Actor)
}
if r.CommentJSON != "" {
t.Errorf("row op=%s: CommentJSON = %q, want \"\" (column does not exist on this shape)", r.Op, r.CommentJSON)
}
}
wantOps := []string{"create", "update", "comment"}
if len(ops) != len(wantOps) {
t.Fatalf("journaled ops for %s = %v, want %v (a fixed INSERT column list fails every one of these on the gci shape)", issue.ID, ops, wantOps)
}
for i, want := range wantOps {
if ops[i] != want {
t.Errorf("journaled op[%d] = %q, want %q", i, ops[i], want)
}
}
}
// TestAdaptiveRead_GciShape is T2.2: a page read on the gci shape succeeds,
// and actor/comment come back empty rather than failing the SELECT. A fixed
// SELECT list (the pre-A1 behavior) fails the read outright, because it names
// both actor and comment_json.
func TestAdaptiveRead_GciShape(t *testing.T) {
te := newGciShapeStore(t, "ar")
ctx := context.Background()
te.store.SetEventsJournalEnabled(true)
if err := te.store.EventsJournalActivationError(); err != nil {
t.Fatalf("activation on the gci shape: %v", err)
}
issue := &types.Issue{ID: "ar-1", Title: "gci read", Status: types.StatusOpen, IssueType: types.TypeTask}
if err := te.store.CreateIssue(ctx, issue, "tester"); err != nil {
t.Fatalf("CreateIssue on gci shape: %v", err)
}
page, err := te.store.ReadEventsJournalPage(ctx, 0, 0)
if err != nil {
t.Fatalf("ReadEventsJournalPage on gci shape: %v", err)
}
if len(page.Rows) == 0 {
t.Fatal("ReadEventsJournalPage on gci shape returned no rows")
}
for _, r := range page.Rows {
if r.Actor != "" {
t.Errorf("row op=%s: Actor = %q, want \"\" on a shape with no actor column", r.Op, r.Actor)
}
if r.CommentJSON != "" {
t.Errorf("row op=%s: CommentJSON = %q, want \"\" on a shape with no comment_json column", r.Op, r.CommentJSON)
}
}
}
// TestJournalShapeProbeOncePerOpen is T2.3: 100 writes against one already-
// activated store issue no INFORMATION_SCHEMA probes of their own — only the
// ONE SetEventsJournalEnabled ran. A per-write probe would show up here as a
// nonzero delta across the 100 writes.
func TestJournalShapeProbeOncePerOpen(t *testing.T) {
te := newTestEnv(t, "pr")
ctx := context.Background()
before := issueops.JournalShapeProbeCountForTest()
te.store.SetEventsJournalEnabled(true)
if err := te.store.EventsJournalActivationError(); err != nil {
t.Fatalf("activation on canonical shape: %v", err)
}
afterActivate := issueops.JournalShapeProbeCountForTest()
if delta := afterActivate - before; delta != 1 {
t.Fatalf("probes issued by SetEventsJournalEnabled = %d, want exactly 1", delta)
}
for i := 0; i < 100; i++ {
issue := &types.Issue{
Title: fmt.Sprintf("probe-once %d", i), Status: types.StatusOpen,
Priority: 2, IssueType: types.TypeTask,
}
if err := te.store.CreateIssue(ctx, issue, "tester"); err != nil {
t.Fatalf("CreateIssue %d: %v", i, err)
}
}
after := issueops.JournalShapeProbeCountForTest()
if delta := after - afterActivate; delta != 0 {
t.Fatalf("probes issued by 100 writes against an already-activated store = %d, want 0 (a per-write probe, not a per-open probe)", delta)
}
}
// TestMissingRequiredColumnRefusesAtOpen is T2.4: a table missing a REQUIRED
// column (op) makes activation fail with a typed error when the journal is
// enabled, but leaves it succeeding when disabled — a disabled workspace
// accepts any shape. It also pins what a store enabled anyway (an open that
// writes no beads, which eventsjournal.Apply does not refuse) does with a
// write: the journal stays ON and the write fails loudly on its INSERT,
// rolling the mutation back with it. Treating the failed probe as "journal
// off" instead would commit the mutation with no journal row — a consumer's
// cursor silently missing a write, which is the one outcome the journal
// exists to rule out.
func TestMissingRequiredColumnRefusesAtOpen(t *testing.T) {
te := newTestEnv(t, "mr")
ctx := context.Background()
te.exec(t, ctx, "ALTER TABLE bd_events_journal DROP COLUMN op")
te.store.SetEventsJournalEnabled(true)
err := te.store.EventsJournalActivationError()
if err == nil {
t.Fatal("activation with the journal enabled against a table missing a required column (op) succeeded; want a typed error")
}
if !errors.Is(err, issueops.ErrJournalShapeUnsupported) {
t.Errorf("activation error = %v, does not match issueops.ErrJournalShapeUnsupported", err)
}
var shapeErr *issueops.JournalShapeError
if !errors.As(err, &shapeErr) {
t.Fatalf("activation error is not a *issueops.JournalShapeError: %v (%T)", err, err)
}
if len(shapeErr.Missing) != 1 || shapeErr.Missing[0] != "op" {
t.Errorf("JournalShapeError.Missing = %v, want [op]", shapeErr.Missing)
}
// Disabling must still succeed: a disabled workspace accepts any shape, and
// never runs the probe that would otherwise fail it.
te.store.SetEventsJournalEnabled(false)
if err := te.store.EventsJournalActivationError(); err != nil {
t.Errorf("activation with the journal disabled against the same broken shape failed: %v", err)
}
// Re-enable (probe fails again, same cached nil shape) and prove a write
// cannot land unrecorded: it must fail, and take its mutation with it.
te.store.SetEventsJournalEnabled(true)
issue := &types.Issue{ID: "mr-1", Title: "no op column", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := te.store.CreateIssue(ctx, issue, "tester"); err == nil {
t.Fatal("CreateIssue succeeded on an enabled store whose journal cannot record it; the write must fail loudly, never commit without a journal row")
}
te.assertRowNotExists(t, ctx, "issues", "mr-1")
}

View file

@ -32,6 +32,7 @@ var _ storage.Flattener = (*EmbeddedDoltStore)(nil)
var _ storage.Compactor = (*EmbeddedDoltStore)(nil)
var _ storage.SchemaMigrator = (*EmbeddedDoltStore)(nil)
var _ storage.EventsJournalConfigurer = (*EmbeddedDoltStore)(nil)
var _ storage.EventsJournalShapeChecker = (*EmbeddedDoltStore)(nil)
var _ storage.VersionedHistoryConfigurer = (*EmbeddedDoltStore)(nil)
var _ storage.ExternalRefHistoryQuerier = (*EmbeddedDoltStore)(nil)
@ -53,6 +54,18 @@ type EmbeddedDoltStore struct {
// eventsJournalEnabled activates the durable events journal for THIS store
// instance only (storage.EventsJournalConfigurer); never process-global.
eventsJournalEnabled atomic.Bool
// journalShape caches the ONE INFORMATION_SCHEMA probe SetEventsJournalEnabled
// runs on activation (issueops.ProbeJournalShape, PR A1), over a short-lived
// connection since this store has no persistent db handle. Nil means the
// journal is disabled or the probe produced no shape; transactions then get
// the canonical column list — see journalActivationErr.
journalShape atomic.Pointer[issueops.JournalShape]
// journalActivationErr holds the probe's *issueops.JournalShapeError when
// SetEventsJournalEnabled(true) found a table shape the journal cannot run
// against (a missing required column, or a missing table). Nil otherwise,
// including when the probe could not run at all
// (issueops.JournalActivationError). See storage.EventsJournalShapeChecker.
journalActivationErr atomic.Pointer[error]
// versionedHistoryEnabled activates dual-write issue-version history for
// THIS store instance only (storage.VersionedHistoryConfigurer); never
// process-global.
@ -457,8 +470,17 @@ func (s *EmbeddedDoltStore) commitConn(ctx context.Context, commit bool, fn func
err = fmt.Errorf("embeddeddolt: begin tx: %w", err)
return
}
// The activation switch deliberately does NOT depend on the cached shape:
// an enabled store whose probe produced no shape writes with the canonical
// column list, so a table the journal cannot run against fails the write
// loudly instead of letting the mutation commit with no journal row
// (storage.EventsJournalShapeChecker). Reads (commit=false callers) are
// never gated by this switch — only by the shape — since ReadEventsJournal
// has no enabled/disabled concept of its own.
clearJournalScope := issueops.ScopeEventsJournalTransaction(tx, s.eventsJournalEnabled.Load())
defer clearJournalScope()
clearJournalShapeScope := issueops.ScopeEventsJournalShape(tx, s.journalShape.Load())
defer clearJournalShapeScope()
clearVersionScope := issueops.ScopeVersionedHistoryTransaction(tx, s.versionedHistoryEnabled.Load())
defer clearVersionScope()
clearRecheckScope := issueops.ScopeBlockedRecheckTransaction(tx)
@ -484,8 +506,53 @@ func (s *EmbeddedDoltStore) commitConn(ctx context.Context, commit bool, fn func
}
// SetEventsJournalEnabled activates the journal for this store instance only.
//
// Activating probes bd_events_journal's actual shape ONCE
// (issueops.ProbeJournalShape, PR A1), over a short-lived connection opened
// and closed just for the probe since this store keeps no persistent db
// handle, and caches the result for this instance's lifetime;
// EventsJournalActivationError reports whether that probe found a shape the
// journal cannot run against. The open and the probe share one
// issueops.JournalShapeProbeTimeout, and either failing to run degrades to
// the canonical shape (issueops.JournalActivationError). Deactivating clears
// both and skips the probe entirely — a disabled store accepts any shape.
func (s *EmbeddedDoltStore) SetEventsJournalEnabled(enabled bool) {
s.eventsJournalEnabled.Store(enabled)
if !enabled {
s.journalShape.Store(nil)
s.journalActivationErr.Store(nil)
return
}
ctx, cancel := context.WithTimeout(context.Background(), issueops.JournalShapeProbeTimeout)
defer cancel()
shape, err := s.probeJournalShape(ctx)
s.journalShape.Store(shape)
if activationErr := issueops.JournalActivationError(err); activationErr != nil {
s.journalActivationErr.Store(&activationErr)
return
}
s.journalActivationErr.Store(nil)
}
// probeJournalShape runs issueops.ProbeJournalShape over a connection opened
// and closed just for it.
func (s *EmbeddedDoltStore) probeJournalShape(ctx context.Context) (*issueops.JournalShape, error) {
db, cleanup, err := OpenSQL(ctx, s.dataDir, s.database, s.branch)
if err != nil {
return nil, fmt.Errorf("journal: open connection to probe shape: %w", err)
}
defer func() { _ = cleanup() }()
return issueops.ProbeJournalShape(ctx, db)
}
// EventsJournalActivationError implements storage.EventsJournalShapeChecker:
// nil unless the last SetEventsJournalEnabled(true) call found a table shape
// the journal cannot run against.
func (s *EmbeddedDoltStore) EventsJournalActivationError() error {
if p := s.journalActivationErr.Load(); p != nil {
return *p
}
return nil
}
// SetVersionedHistoryEnabled activates dual-write issue-version history for

View file

@ -136,6 +136,20 @@ func (p *uowProvider) SetEventsJournalEnabled(enabled bool) {
}
}
// EventsJournalActivationError forwards the inner provider's shape-probe
// result (storage.EventsJournalShapeChecker, PR A1). This wrapper is the
// OUTERMOST provider on both real chains (see SetVersionedHistoryEnabled), so
// without this forwarder eventsjournal.Apply's type assertion for the checker
// would fail here even when the inner provider's own probe found an
// unsupported table shape — the open would then succeed against a shape it
// cannot journal, the gci failure mode this interface exists to avoid.
func (p *uowProvider) EventsJournalActivationError() error {
if checker, ok := p.UnitOfWorkProvider.(storage.EventsJournalShapeChecker); ok {
return checker.EventsJournalActivationError()
}
return nil
}
// SetVersionedHistoryEnabled forwards dual-write issue-version activation
// inward, for the same reason SetEventsJournalEnabled does.
//
@ -164,6 +178,7 @@ func (p *uowProvider) RunEventsMaintenanceTx(ctx context.Context, fn func(contex
var (
_ uow.PoolTuner = (*uowProvider)(nil)
_ storage.EventsJournalConfigurer = (*uowProvider)(nil)
_ storage.EventsJournalShapeChecker = (*uowProvider)(nil)
_ storage.VersionedHistoryConfigurer = (*uowProvider)(nil)
_ issueops.EventsMaintenanceRunner = (*uowProvider)(nil)
_ uow.IssueLifecycleSource = (*uowProvider)(nil)

View file

@ -64,6 +64,7 @@ go_library(
"journal_autoprune.go",
"journal_dependencies.go",
"journal_prune.go",
"journal_shape.go",
"labels.go",
"lease.go",
"metadata.go",

View file

@ -7,6 +7,7 @@ import (
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
@ -420,6 +421,15 @@ func getJournalIssueInTx(ctx context.Context, tx DBTX, issueID string) (*types.I
// (non-dependency ops). ts is the insert time, stamped inside the committing
// transaction. actor is stored as-is — "" for the genuinely unattributable
// paths — in the NOT NULL DEFAULT ” actor column.
//
// The column list is shape-adaptive (journal_shape.go, PR A1): seq, ts, op
// and issue_id are always named, and each optional column — actor,
// issue_json, dep_json, comment_json — is named only when journalShapeFor(tx)
// says this store's table actually has it. On a canonical table that is every
// column, in the same order this INSERT has always used; on an old-lineage
// table missing one, the write still lands and that one payload is simply not
// recorded, rather than failing the whole mutation the journal row shares a
// transaction with.
func insertEventRow(ctx context.Context, tx DBTX, op EventOp, issueID string, issue *types.Issue, dep *EventDep, comment *EventComment, actor string) error {
var issueJSON any
if issue != nil {
@ -445,11 +455,41 @@ func insertEventRow(ctx context.Context, tx DBTX, op EventOp, issueID string, is
}
commentJSON = string(b)
}
shape := journalShapeFor(tx)
type optionalJournalValue struct {
column string
value any
}
var optionalValues []optionalJournalValue
if shape.HasActor {
optionalValues = append(optionalValues, optionalJournalValue{"actor", actor})
}
if shape.HasIssueJSON {
optionalValues = append(optionalValues, optionalJournalValue{"issue_json", issueJSON})
}
if shape.HasDepJSON {
optionalValues = append(optionalValues, optionalJournalValue{"dep_json", depJSON})
}
if shape.HasCommentJSON {
optionalValues = append(optionalValues, optionalJournalValue{"comment_json", commentJSON})
}
columns := make([]string, 0, 4+len(optionalValues))
columns = append(columns, "seq", "ts", "op", "issue_id")
for _, ov := range optionalValues {
columns = append(columns, ov.column)
}
placeholders := strings.TrimSuffix(strings.Repeat("?, ", len(columns)), ", ")
//nolint:gosec // columns is built entirely from the fixed name literals above, never from caller input.
query := fmt.Sprintf("INSERT INTO bd_events_journal (%s) VALUES (%s)", strings.Join(columns, ", "), placeholders)
insert := func(seq int64) error {
_, err := tx.ExecContext(ctx, `
INSERT INTO bd_events_journal (seq, ts, op, issue_id, actor, issue_json, dep_json, comment_json)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
`, seq, time.Now().UTC(), string(op), issueID, actor, issueJSON, depJSON, commentJSON)
args := make([]any, 0, len(columns))
args = append(args, seq, time.Now().UTC(), string(op), issueID)
for _, ov := range optionalValues {
args = append(args, ov.value)
}
_, err := tx.ExecContext(ctx, query, args...)
return err
}

View file

@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"strconv"
"strings"
"time"
"github.com/steveyegge/beads/internal/storage"
@ -251,10 +252,30 @@ func readEventsHeadInTx(ctx context.Context, tx DBTX) (int64, error) {
return head, nil
}
// readEventsRowsInTx's column list is shape-adaptive (journal_shape.go, PR
// A1): seq, ts, op and issue_id are always selected, and each optional column
// is selected only when journalShapeFor(tx) says this store's table actually
// has it. A column the probe did not find is simply never requested, so the
// corresponding field on storage.EventsJournalRow reads as "" — never a scan
// error — for a consumer on an old-lineage table.
func readEventsRowsInTx(ctx context.Context, tx DBTX, since int64, limit int) ([]storage.EventsJournalRow, error) {
shape := journalShapeFor(tx)
// CAST(ts AS CHAR) normalizes the DATETIME to a stable string across drivers.
q := `SELECT seq, CAST(ts AS CHAR), op, issue_id, actor, issue_json, dep_json, comment_json
FROM bd_events_journal WHERE seq > ? ORDER BY seq ASC`
columns := []string{"seq", "CAST(ts AS CHAR)", "op", "issue_id"}
if shape.HasActor {
columns = append(columns, "actor")
}
if shape.HasIssueJSON {
columns = append(columns, "issue_json")
}
if shape.HasDepJSON {
columns = append(columns, "dep_json")
}
if shape.HasCommentJSON {
columns = append(columns, "comment_json")
}
//nolint:gosec // columns is built entirely from the fixed name literals above, never from caller input.
q := "SELECT " + strings.Join(columns, ", ") + " FROM bd_events_journal WHERE seq > ? ORDER BY seq ASC"
if limit > 0 {
q += " LIMIT " + strconv.Itoa(limit)
}
@ -268,14 +289,29 @@ func readEventsRowsInTx(ctx context.Context, tx DBTX, since int64, limit int) ([
for rows.Next() {
var (
r storage.EventsJournalRow
actor sql.NullString
issueJS sql.NullString
depJS sql.NullString
commentJS sql.NullString
)
if err := rows.Scan(&r.Seq, &r.TS, &r.Op, &r.IssueID, &r.Actor, &issueJS, &depJS, &commentJS); err != nil {
dest := []any{&r.Seq, &r.TS, &r.Op, &r.IssueID}
if shape.HasActor {
dest = append(dest, &actor)
}
if shape.HasIssueJSON {
dest = append(dest, &issueJS)
}
if shape.HasDepJSON {
dest = append(dest, &depJS)
}
if shape.HasCommentJSON {
dest = append(dest, &commentJS)
}
if err := rows.Scan(dest...); err != nil {
return nil, fmt.Errorf("journal: scan row: %w", err)
}
r.TS = normalizeEventsTimestamp(r.TS)
r.Actor = actor.String
r.IssueJSON = issueJS.String
r.DepJSON = depJS.String
r.CommentJSON = commentJS.String

View file

@ -0,0 +1,304 @@
package issueops
import (
"context"
"errors"
"fmt"
"os"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/steveyegge/beads/internal/debug"
)
// Adaptive I/O for bd_events_journal (BEADS-JOURNAL-PLAN.md §4.2a, PR A1).
//
// The writer and reader used to name a fixed column list. That is fine on a
// table built by the current migrations, but a table built by an earlier
// lineage — gas-city-inc's is the live case — can lack an optional column
// forever: the ignored series only ever creates the table when it is wholly
// absent, and its one repair (ignored/0023) probes column TYPES, never
// EXISTENCE. A fixed INSERT naming a missing column then fails every
// journaled write, and because the journal row commits in the SAME
// transaction as the mutation it records, that failure rolls back the user's
// write along with it.
//
// The fix probes once, at activation, rather than guessing or failing per
// write:
// - REQUIRED columns {seq, ts, op, issue_id} have existed on every lineage
// this table has ever shipped under; a table missing one of them, or
// missing entirely, cannot journal at all. ProbeJournalShape refuses with
// a typed error so an open that is going to write beads fails at open,
// before any mutation is attempted — never as a per-write failure after
// the work is otherwise done, which is the gci failure mode above
// (eventsjournal.Apply decides which opens that is). A probe that could
// not run at all is not a shape verdict: JournalActivationError degrades
// it to the canonical shape with a warning instead of refusing.
// - OPTIONAL columns {actor, issue_json, dep_json, comment_json} may be
// missing on an old-lineage table. The writer drops whichever of them the
// probe did not find; the reader returns "" for them. One warning is
// logged per probe (i.e. once per store instance, not once per write),
// and ignored/0028 (PR A2) is what actually converges the shape — this
// file only keeps a drifted table WRITABLE and READABLE in the meantime.
//
// A store instance probes exactly once, when storage.EventsJournalConfigurer
// activates the journal (see each backend's SetEventsJournalEnabled), and
// caches the *JournalShape for its own lifetime. ScopeEventsJournalShape
// carries that cached value onto one transaction, next to
// ScopeEventsJournalTransaction's activation switch, so insertEventRow and
// readEventsRowsInTx — both of which only ever see a tx, never the store —
// can reach it without a per-write probe (TestJournalShapeProbeOncePerOpen).
// journalTableName is the clone-local table every probe, insert and read in
// this file targets.
const journalTableName = "bd_events_journal"
// requiredJournalColumns are the columns ProbeJournalShape treats as
// non-negotiable: every lineage of bd_events_journal has always carried them,
// and the seq/ts/op/issue_id contract (nextEventSeq, the reader's ordering)
// is meaningless without them.
var requiredJournalColumns = []string{"seq", "ts", "op", "issue_id"}
// optionalJournalColumns are the columns a pre-merge lineage may lack. Order
// matches the canonical INSERT column order a healthy table has carried since
// migration 0066/ignored 0025 added actor.
var optionalJournalColumns = []string{"actor", "issue_json", "dep_json", "comment_json"}
// JournalShape records which OPTIONAL bd_events_journal columns exist on one
// store's table. A nil *JournalShape is never handed to a caller; probing a
// table missing a REQUIRED column returns an error instead (see
// ProbeJournalShape).
type JournalShape struct {
HasActor bool
HasIssueJSON bool
HasDepJSON bool
HasCommentJSON bool
}
// missingOptional names the optional columns this shape does not have, in
// optionalJournalColumns order, for the doctor finding and the one-per-probe
// warning.
func (s *JournalShape) missingOptional() []string {
if s == nil {
return nil
}
var missing []string
if !s.HasActor {
missing = append(missing, "actor")
}
if !s.HasIssueJSON {
missing = append(missing, "issue_json")
}
if !s.HasDepJSON {
missing = append(missing, "dep_json")
}
if !s.HasCommentJSON {
missing = append(missing, "comment_json")
}
return missing
}
// canonicalJournalShape is every optional column present — the shape every
// workspace created from main migration 0064 plus ignored 0022/0025 has, and
// the default journalShapeFor returns for a transaction nobody scoped. That
// default matters: it is what every call site written before this file
// existed (and every test that activates the journal through the ctx-only
// WithEventsJournal test helper rather than a real store) keeps getting, so
// adaptive I/O is additive and changes nothing for a canonical table.
var canonicalJournalShape = &JournalShape{
HasActor: true,
HasIssueJSON: true,
HasDepJSON: true,
HasCommentJSON: true,
}
// ErrJournalShapeUnsupported is the sentinel errors.Is matches against any
// *JournalShapeError returned by ProbeJournalShape, regardless of which
// required columns a particular table happens to be missing.
var ErrJournalShapeUnsupported = errors.New("events journal: table shape does not support the events journal")
// JournalShapeError reports that bd_events_journal (or the table itself) is
// missing one or more REQUIRED columns. It is returned by ProbeJournalShape
// and is the typed error storage.EventsJournalConfigurer implementations
// surface from activation, so an open that is going to write beads fails at
// open rather than the gci failure mode: succeeding at open and then failing
// every write thereafter.
type JournalShapeError struct {
Table string
Missing []string
}
func (e *JournalShapeError) Error() string {
return fmt.Sprintf(
"events journal: table %s is missing required column(s) %s; the events journal cannot be enabled against this shape (see `bd doctor`)",
e.Table, strings.Join(e.Missing, ", "),
)
}
// Is makes errors.Is(err, ErrJournalShapeUnsupported) match any
// *JournalShapeError, so a caller can test for the class of failure without
// caring which columns a particular table lacks.
func (e *JournalShapeError) Is(target error) bool {
return target == ErrJournalShapeUnsupported //nolint:errorlint // sentinel identity comparison is the documented contract of Is.
}
// journalShapeProbeCount counts how many times ProbeJournalShape has actually
// queried INFORMATION_SCHEMA in this process. It exists for
// TestJournalShapeProbeOncePerOpen, which asserts that a batch of writes
// against one already-activated store issues none of its own.
var journalShapeProbeCount atomic.Int64
// JournalShapeProbeCountForTest reports journalShapeProbeCount's current
// value. Test-only instrumentation, read by TestJournalShapeProbeOncePerOpen;
// production code never calls it.
func JournalShapeProbeCountForTest() int64 { return journalShapeProbeCount.Load() }
// ProbeJournalShape runs the one INFORMATION_SCHEMA.COLUMNS query that
// determines what q's bd_events_journal table actually supports. Callers run
// it exactly once per store/provider instance, from
// storage.EventsJournalConfigurer.SetEventsJournalEnabled, and cache the
// result for that instance's lifetime — never per write or per transaction.
//
// A table missing any required column, including a wholly absent table
// (which reads as every column missing, since a COLUMNS query against a
// nonexistent table returns zero rows — the same shape ignored/0023's
// existing probes rely on), returns *JournalShapeError instead of a shape.
func ProbeJournalShape(ctx context.Context, q DBTX) (*JournalShape, error) {
journalShapeProbeCount.Add(1)
rows, err := q.QueryContext(ctx, `
SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = ?`, journalTableName)
if err != nil {
return nil, fmt.Errorf("journal: probe %s shape: %w", journalTableName, err)
}
defer rows.Close()
present := make(map[string]bool, len(requiredJournalColumns)+len(optionalJournalColumns))
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, fmt.Errorf("journal: scan %s column: %w", journalTableName, err)
}
present[strings.ToLower(name)] = true
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("journal: read %s shape: %w", journalTableName, err)
}
var missing []string
for _, col := range requiredJournalColumns {
if !present[col] {
missing = append(missing, col)
}
}
if len(missing) > 0 {
return nil, &JournalShapeError{Table: journalTableName, Missing: missing}
}
shape := &JournalShape{
HasActor: present["actor"],
HasIssueJSON: present["issue_json"],
HasDepJSON: present["dep_json"],
HasCommentJSON: present["comment_json"],
}
warnJournalShapeDegraded(shape)
return shape, nil
}
// JournalShapeProbeTimeout bounds the activation probe, including the
// connection the embedded store opens just for it. SetEventsJournalEnabled
// takes no context, so without a ceiling a wedged server — or an embedded
// open waiting out another handle on the same database — would hang the open
// that activates the journal indefinitely.
const JournalShapeProbeTimeout = 30 * time.Second
// JournalActivationError reduces a failed activation probe to the error a
// store caches for storage.EventsJournalShapeChecker.
//
// Only a *JournalShapeError is a verdict about the table, and only that
// refuses activation. Any other failure — a timeout, a dropped connection, an
// embedded open that could not get a handle — says nothing about the shape, so
// it must not refuse an open that would otherwise succeed. It returns nil for
// those, and the store keeps a nil shape: its transactions then use
// canonicalJournalShape, the fixed column list every write used before
// adaptive I/O existed.
//
// That guess is wrong on exactly the tables this file exists for: on a lineage
// that lacks a journal column, every journaled write through the store fails,
// and nothing re-probes until the store is reopened. So the one warning names
// that consequence, and quiet mode, which suppresses it on stderr, still
// leaves it in the debug log.
func JournalActivationError(probeErr error) error {
if probeErr == nil {
return nil
}
var shapeErr *JournalShapeError
if errors.As(probeErr, &shapeErr) {
return probeErr
}
warning := fmt.Sprintf(
"events journal: could not probe %s shape (%v); assuming the canonical shape, so if the table lacks a journal column every journaled write through this store will fail until it is reopened",
journalTableName, probeErr)
if debug.IsQuiet() {
debug.Logf("%s\n", warning)
return nil
}
fmt.Fprintln(os.Stderr, warning)
return nil
}
// warnJournalShapeDegraded writes one audit line to STDERR, suppressed only
// by quiet mode, the first (and every) time a store probes a table missing an
// optional column — one call per ProbeJournalShape invocation, which is one
// per store activation, which is "one warning per store" regardless of how
// many writes that store goes on to make.
func warnJournalShapeDegraded(shape *JournalShape) {
missing := shape.missingOptional()
if len(missing) == 0 {
return
}
if debug.IsQuiet() {
return
}
fmt.Fprintf(os.Stderr,
"events journal: %s is missing optional column(s) %s; affected payload(s) will be dropped until it converges (see `bd doctor`)\n",
journalTableName, strings.Join(missing, ", "))
}
// journalShapes associates a probed *JournalShape with ONE concrete
// transaction, the same per-tx lifetime journalTransactions gives the
// activation switch. Entries live for one transaction only.
var journalShapes sync.Map // map[DBTX]*JournalShape
// ScopeEventsJournalShape binds shape to tx so insertEventRow and
// readEventsRowsInTx can reach the store's already-cached probe result
// without taking the store itself as a parameter. Store implementations call
// it immediately after BeginTx, alongside ScopeEventsJournalTransaction,
// passing the *JournalShape SetEventsJournalEnabled cached at activation —
// never a fresh probe.
//
// A nil tx or nil shape is a no-op; journalShapeFor then returns
// canonicalJournalShape, today's fixed eight-column behavior.
func ScopeEventsJournalShape(tx DBTX, shape *JournalShape) func() {
if tx == nil || shape == nil {
return func() {}
}
journalShapes.Store(tx, shape)
return func() { journalShapes.Delete(tx) }
}
// journalShapeFor returns the shape scoped to tx, or canonicalJournalShape
// when nobody scoped one — which is every call site that predates this file,
// and every test driving the journal through the ctx-only WithEventsJournal
// helper rather than a real store's activation.
func journalShapeFor(tx DBTX) *JournalShape {
if v, ok := journalShapes.Load(tx); ok {
if shape, ok := v.(*JournalShape); ok && shape != nil {
return shape
}
}
return canonicalJournalShape
}

View file

@ -921,6 +921,36 @@ type EventsJournalConfigurer interface {
SetEventsJournalEnabled(enabled bool)
}
// EventsJournalShapeChecker reports whether the LAST SetEventsJournalEnabled
// call on this instance found a table shape the journal cannot run against.
//
// It exists because SetEventsJournalEnabled itself stays void: dozens of
// tests across three backends already call it as a bare statement, and
// adaptive I/O (PR A1) must not force every one of them to start checking an
// error it never had before. Instead, enabling the journal probes the table's
// shape internally and caches the verdict; this is the seam a factory's open
// path checks immediately afterward. Only a typed shape verdict
// (issueops.JournalShapeError: a missing required column, or a missing table)
// is reported. A probe that could not run at all says nothing about the
// table, so it degrades to the canonical shape with a warning and reports nil.
//
// The journal stays enabled either way, and an enabled store with an
// unsupported shape is NOT a no-op for writes: it writes with the canonical
// column list, so each journaled write fails loudly on its INSERT and rolls
// back with the mutation it records — never a mutation committed with no
// journal row. eventsjournal.Apply turns the verdict into a refused open for
// an open that is going to write beads, so that path fails once, at open,
// instead of on every write (the gci failure mode this interface exists to
// avoid); an open that writes no beads is not refused. Reads are never gated
// by the enabled flag: a table that cannot answer fails the read with a loud
// SQL error, never a silently wrong answer.
//
// A store that does not implement it is assumed never to fail activation
// (today, every implementation does).
type EventsJournalShapeChecker interface {
EventsJournalActivationError() error
}
// VersionedHistoryConfigurer controls dual-write issue-version history
// activation on ONE storage instance. Implementations must never use
// process-global state, for the same reason as EventsJournalConfigurer:

View file

@ -57,6 +57,17 @@ type doltSQLProvider struct {
// eventsJournalEnabled activates the durable events journal for THIS
// provider instance only. See SetEventsJournalEnabled.
eventsJournalEnabled atomic.Bool
// journalShape caches the ONE INFORMATION_SCHEMA probe SetEventsJournalEnabled
// runs on activation (issueops.ProbeJournalShape, PR A1). Nil means the
// journal is disabled or the probe produced no shape; transactions then get
// the canonical column list — see journalActivationErr.
journalShape atomic.Pointer[issueops.JournalShape]
// journalActivationErr holds the probe's *issueops.JournalShapeError when
// SetEventsJournalEnabled(true) found a table shape the journal cannot run
// against (a missing required column, or a missing table). Nil otherwise,
// including when the probe could not run at all
// (issueops.JournalActivationError). See storage.EventsJournalShapeChecker.
journalActivationErr atomic.Pointer[error]
// versionedHistoryEnabled activates dual-write issue-version history for
// THIS provider instance only. See SetVersionedHistoryEnabled.
versionedHistoryEnabled atomic.Bool
@ -72,8 +83,41 @@ type doltSQLProvider struct {
// BeginTx). Without that binding the uow plumbing writes mutations while
// journaling nothing — the failure is invisible, because the code runs and the
// write lands and the journal is simply empty.
//
// Activating also probes bd_events_journal's actual shape ONCE
// (issueops.ProbeJournalShape, PR A1), bounded by
// issueops.JournalShapeProbeTimeout, and caches the result for this
// instance's lifetime; EventsJournalActivationError reports whether that
// probe found a shape the journal cannot run against. A probe that could not
// run degrades to the canonical shape (issueops.JournalActivationError).
// Deactivating clears both and skips the probe entirely — a disabled provider
// accepts any shape.
func (p *doltSQLProvider) SetEventsJournalEnabled(enabled bool) {
p.eventsJournalEnabled.Store(enabled)
if !enabled {
p.journalShape.Store(nil)
p.journalActivationErr.Store(nil)
return
}
ctx, cancel := context.WithTimeout(context.Background(), issueops.JournalShapeProbeTimeout)
defer cancel()
shape, err := issueops.ProbeJournalShape(ctx, p.db)
p.journalShape.Store(shape)
if activationErr := issueops.JournalActivationError(err); activationErr != nil {
p.journalActivationErr.Store(&activationErr)
return
}
p.journalActivationErr.Store(nil)
}
// EventsJournalActivationError implements storage.EventsJournalShapeChecker:
// nil unless the last SetEventsJournalEnabled(true) call found a table shape
// the journal cannot run against (PR A1).
func (p *doltSQLProvider) EventsJournalActivationError() error {
if ptr := p.journalActivationErr.Load(); ptr != nil {
return *ptr
}
return nil
}
// SetVersionedHistoryEnabled activates dual-write issue-version history for
@ -134,6 +178,12 @@ type providerOptions struct {
// working through the upgrade window. That is the same warn-and-continue
// contract embeddeddolt gives its read-only-command intent.
readOnly bool
// workingSetReconcile opens for a command that commits the working set
// rather than writing beads (bd dolt commit; the provider-side
// counterpart of dolt.Config.LenientOpen's commands). It changes nothing
// about the open itself. It exists so OpensForBeadWrites can tell such an
// open from one that writes beads.
workingSetReconcile bool
}
// WithPreview opens the provider for a non-mutating preview command.
@ -146,6 +196,21 @@ func WithReadOnly() ProviderOption {
return func(o *providerOptions) { o.readOnly = true }
}
// WithWorkingSetReconcile marks the provider as opened for a command that
// commits the working set rather than writing beads.
func WithWorkingSetReconcile() ProviderOption {
return func(o *providerOptions) { o.workingSetReconcile = true }
}
// OpensForBeadWrites reports whether a provider opened with opts is going to
// write beads: false for a preview, a read-only open, or a working-set
// reconcile. Events-journal activation reads it to decide whether a journal
// table the journal cannot run against refuses the open (eventsjournal.Apply).
func OpensForBeadWrites(opts ...ProviderOption) bool {
resolved := applyProviderOptions(opts)
return !resolved.preview && !resolved.readOnly && !resolved.workingSetReconcile
}
func applyProviderOptions(opts []ProviderOption) providerOptions {
var resolved providerOptions
for _, opt := range opts {
@ -160,6 +225,7 @@ var (
_ UnitOfWorkProvider = (*doltSQLProvider)(nil)
_ TxProvider = (*doltSQLProvider)(nil)
_ storage.EventsJournalConfigurer = (*doltSQLProvider)(nil)
_ storage.EventsJournalShapeChecker = (*doltSQLProvider)(nil)
_ storage.VersionedHistoryConfigurer = (*doltSQLProvider)(nil)
)
@ -195,9 +261,18 @@ func (p *doltSQLProvider) BeginTx(ctx context.Context) (Tx, error) {
// (doltServerTx.releaseConn / poisonConn), so an entry cannot outlive its
// transaction. The blocked-recheck scope is bound and released the same
// way; Commit takes what it recorded once the transaction has committed.
// The shape scope carries the probe SetEventsJournalEnabled cached. The
// activation switch deliberately does NOT depend on it: an enabled provider
// whose probe produced no shape writes with the canonical column list, so a
// table the journal cannot run against fails the write loudly instead of
// letting the mutation commit with no journal row
// (storage.EventsJournalShapeChecker). Reads share this same BeginTx, and
// are never gated by the activation switch — only by the shape — since
// EventsJournalCursor has no enabled/disabled concept of its own.
return &doltServerTx{
conn: conn,
clearJournalScope: issueops.ScopeEventsJournalTransaction(conn, p.eventsJournalEnabled.Load()),
clearJournalShape: issueops.ScopeEventsJournalShape(conn, p.journalShape.Load()),
clearVersionScope: issueops.ScopeVersionedHistoryTransaction(conn, p.versionedHistoryEnabled.Load()),
clearRecheckScope: issueops.ScopeBlockedRecheckTransaction(conn),
}, nil

View file

@ -18,6 +18,10 @@ type doltServerTx struct {
// releaseConn and poisonConn — so the activation entry cannot outlive the
// transaction it describes, whichever way the transaction ends.
clearJournalScope func()
// clearJournalShape releases the probed *issueops.JournalShape BeginTx bound
// to conn (PR A1), the shape counterpart of clearJournalScope's enabled/disabled
// switch. Same binding at BeginTx, same release at releaseConn/poisonConn.
clearJournalShape func()
// clearVersionScope is clearJournalScope's counterpart for dual-write
// issue-version history: same binding at BeginTx, same release at
// releaseConn/poisonConn.
@ -197,6 +201,7 @@ func (t *doltServerTx) rollbackConn(ctx context.Context) error {
func (t *doltServerTx) releaseConn() {
t.releaseJournalScope()
t.releaseJournalShapeScope()
t.releaseVersionScope()
t.releaseRecheckScope()
if t.conn != nil {
@ -215,6 +220,15 @@ func (t *doltServerTx) releaseJournalScope() {
}
}
// releaseJournalShapeScope is releaseJournalScope's counterpart for the probed
// *issueops.JournalShape (PR A1). Idempotent for the same reason.
func (t *doltServerTx) releaseJournalShapeScope() {
if t.clearJournalShape != nil {
t.clearJournalShape()
t.clearJournalShape = nil
}
}
// releaseVersionScope is releaseJournalScope's counterpart for dual-write
// issue-version history. Idempotent for the same reason.
func (t *doltServerTx) releaseVersionScope() {
@ -241,6 +255,7 @@ func (t *doltServerTx) releaseRecheckScope() {
// database/sql close the connection and drop it from the pool.
func (t *doltServerTx) poisonConn() {
t.releaseJournalScope()
t.releaseJournalShapeScope()
t.releaseVersionScope()
t.releaseRecheckScope()
if t.conn == nil {

View file

@ -2,6 +2,8 @@ package uow
import (
"context"
"errors"
"regexp"
"testing"
"github.com/DATA-DOG/go-sqlmock"
@ -22,8 +24,30 @@ import (
// commits, and the journal is simply empty — so it must be pinned from both
// directions: enabled emits, disabled does not.
// expectCanonicalJournalShapeProbe arms the one INFORMATION_SCHEMA.COLUMNS
// query SetEventsJournalEnabled(true) now issues to probe bd_events_journal's
// shape (issueops.ProbeJournalShape, PR A1), returning every column a
// canonical table carries so activation succeeds with the shape every test in
// this file was written against before shape probing existed.
func expectCanonicalJournalShapeProbe(mock sqlmock.Sqlmock) {
expectJournalShapeProbe(mock,
"seq", "ts", "op", "issue_id", "actor", "issue_json", "dep_json", "comment_json")
}
// expectJournalShapeProbe arms the shape probe to report exactly columns.
func expectJournalShapeProbe(mock sqlmock.Sqlmock, columns ...string) {
rows := sqlmock.NewRows([]string{"COLUMN_NAME"})
for _, column := range columns {
rows.AddRow(column)
}
mock.ExpectQuery("SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS").
WithArgs("bd_events_journal").
WillReturnRows(rows)
}
func TestProviderImplementsEventsJournalConfigurer(t *testing.T) {
p, _ := newMockTxProvider(t)
p, mock := newMockTxProvider(t)
expectCanonicalJournalShapeProbe(mock)
var configurer storage.EventsJournalConfigurer = p
configurer.SetEventsJournalEnabled(true)
require.True(t, p.eventsJournalEnabled.Load(),
@ -37,7 +61,10 @@ func TestProviderImplementsEventsJournalConfigurer(t *testing.T) {
// internal bookkeeping.
func TestBeginTxScopesJournalActivationToThePinnedConn(t *testing.T) {
p, mock := newMockTxProvider(t)
expectCanonicalJournalShapeProbe(mock)
p.SetEventsJournalEnabled(true)
require.NoError(t, p.EventsJournalActivationError(),
"activation against the canonical shape the probe expectation below describes must succeed")
mock.ExpectExec("START TRANSACTION").WillReturnResult(sqlmock.NewResult(0, 0))
mock.ExpectExec("UPDATE bd_events_seq SET next_seq").WillReturnResult(sqlmock.NewResult(0, 1))
@ -98,7 +125,13 @@ func TestTxEndReleasesJournalScope(t *testing.T) {
} {
t.Run(tc.name, func(t *testing.T) {
p, mock := newMockTxProvider(t)
// Arm the shape probe so activation succeeds outright. This test
// pins the release, so it must not lean on how a failed probe is
// handled: while a failed probe left the transaction unjournaled,
// the emit below passed with or without the release.
expectCanonicalJournalShapeProbe(mock)
p.SetEventsJournalEnabled(true)
require.NoError(t, p.EventsJournalActivationError())
mock.ExpectExec("START TRANSACTION").WillReturnResult(sqlmock.NewResult(0, 0))
tx, err := p.BeginTx(context.Background())
@ -111,6 +144,7 @@ func TestTxEndReleasesJournalScope(t *testing.T) {
require.NoError(t, mock.ExpectationsWereMet())
require.Nil(t, serverTx.clearJournalScope, "the scope must be released with the connection")
require.Nil(t, serverTx.clearJournalShape, "the probed shape must be released with the connection")
// Emitting against the released connection must be a no-op. Had the
// activation entry survived, the emit would still consider itself
// enabled and run SQL on a connection that is back in the pool —
@ -120,3 +154,93 @@ func TestTxEndReleasesJournalScope(t *testing.T) {
})
}
}
// expectJournalSeqAllocation arms the counter round trip an emit makes before
// its INSERT.
func expectJournalSeqAllocation(mock sqlmock.Sqlmock) {
mock.ExpectExec("UPDATE bd_events_seq SET next_seq").WillReturnResult(sqlmock.NewResult(0, 1))
mock.ExpectQuery("SELECT next_seq FROM bd_events_seq").
WillReturnRows(sqlmock.NewRows([]string{"next_seq"}).AddRow(7))
}
// canonicalJournalInsert is the INSERT a table carrying every optional column
// gets, and the one a store with no probed shape falls back to.
var canonicalJournalInsert = regexp.QuoteMeta(
"INSERT INTO bd_events_journal (seq, ts, op, issue_id, actor, issue_json, dep_json, comment_json) VALUES")
// TestBeginTxWritesTheProbedJournalShape is the proxied-server counterpart of
// the embedded TestAdaptiveInsert_GciShape: on a table that never gained actor
// or comment_json (gas-city-inc's lineage), a unit of work's INSERT names only
// the columns the probe found. Naming either missing column is the 1054 error
// that rolled back every journaled write on that lineage.
func TestBeginTxWritesTheProbedJournalShape(t *testing.T) {
p, mock := newMockTxProvider(t)
expectJournalShapeProbe(mock, "seq", "ts", "op", "issue_id", "issue_json", "dep_json")
p.SetEventsJournalEnabled(true)
require.NoError(t, p.EventsJournalActivationError(),
"a table missing only OPTIONAL columns is degraded, not unsupported")
mock.ExpectExec("START TRANSACTION").WillReturnResult(sqlmock.NewResult(0, 0))
expectJournalSeqAllocation(mock)
mock.ExpectExec(regexp.QuoteMeta(
"INSERT INTO bd_events_journal (seq, ts, op, issue_id, issue_json, dep_json) VALUES")).
WillReturnResult(sqlmock.NewResult(0, 1))
ctx := context.Background()
tx, err := p.BeginTx(ctx)
require.NoError(t, err)
require.NoError(t, issueops.RecordDeleteInTx(ctx, tx.Runner(), "bd-1", "test-actor"))
require.NoError(t, mock.ExpectationsWereMet())
}
// TestUnsupportedJournalShapeStillJournals pins both halves of a shape
// verdict. A table missing a REQUIRED column is the activation error a
// bead-writing open is refused with. A provider enabled anyway keeps
// journaling: the canonical INSERT fails and takes the mutation's transaction
// with it, rather than the mutation committing with no journal row.
func TestUnsupportedJournalShapeStillJournals(t *testing.T) {
p, mock := newMockTxProvider(t)
expectJournalShapeProbe(mock, "seq", "ts", "issue_id", "actor", "issue_json", "dep_json", "comment_json")
p.SetEventsJournalEnabled(true)
require.ErrorIs(t, p.EventsJournalActivationError(), issueops.ErrJournalShapeUnsupported)
mock.ExpectExec("START TRANSACTION").WillReturnResult(sqlmock.NewResult(0, 0))
expectJournalSeqAllocation(mock)
mock.ExpectExec(canonicalJournalInsert).
WillReturnError(errors.New("Error 1054 (42S22): Unknown column 'op' in 'field list'"))
ctx := context.Background()
tx, err := p.BeginTx(ctx)
require.NoError(t, err)
require.Error(t, issueops.RecordDeleteInTx(ctx, tx.Runner(), "bd-1", "test-actor"),
"a mutation must not commit on a journaling provider without its journal row")
require.NoError(t, mock.ExpectationsWereMet())
}
// TestFailedShapeProbeDoesNotRefuseActivation is the other arm: a probe that
// never reached the table says nothing about its shape, so it must not refuse
// an open that would otherwise succeed. Activation reports no error and the
// journal stays on, writing the canonical column list.
func TestFailedShapeProbeDoesNotRefuseActivation(t *testing.T) {
p, mock := newMockTxProvider(t)
mock.ExpectQuery("SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS").
WithArgs("bd_events_journal").
WillReturnError(errors.New("dial tcp 127.0.0.1:3307: connect: connection refused"))
p.SetEventsJournalEnabled(true)
require.NoError(t, p.EventsJournalActivationError(),
"only a *JournalShapeError is a verdict about the table")
mock.ExpectExec("START TRANSACTION").WillReturnResult(sqlmock.NewResult(0, 0))
expectJournalSeqAllocation(mock)
mock.ExpectExec(canonicalJournalInsert).WillReturnResult(sqlmock.NewResult(0, 1))
ctx := context.Background()
tx, err := p.BeginTx(ctx)
require.NoError(t, err)
require.NoError(t, issueops.RecordDeleteInTx(ctx, tx.Runner(), "bd-1", "test-actor"))
require.NoError(t, mock.ExpectationsWereMet(),
"a failed probe must leave the journal on; turning it off lets writes commit unrecorded")
}

View file

@ -356,6 +356,18 @@ func (p *notifyingProvider) SetEventsJournalEnabled(enabled bool) {
}
}
// EventsJournalActivationError forwards the inner provider's shape-probe
// result (storage.EventsJournalShapeChecker, PR A1), for the same reason
// SetEventsJournalEnabled forwards activation itself: the wrapper holds no
// journal state of its own, and eventsjournal.Apply must see the inner
// provider's real probe result through whichever wrapper sits on top of it.
func (p *notifyingProvider) EventsJournalActivationError() error {
if checker, ok := p.inner.(storage.EventsJournalShapeChecker); ok {
return checker.EventsJournalActivationError()
}
return nil
}
// SetVersionedHistoryEnabled forwards dual-write issue-version history
// activation to the provider that actually binds it to a transaction
// (doltSQLProvider.BeginTx), for the same reason and in the same shape as

View file

@ -300,9 +300,13 @@ func TestNotifyingProviderBuildsRolesOnItself(t *testing.T) {
// for both binds on the inner provider's transactions, and a retention
// pass writes no bead. None of these build a role, so none of them can
// build one "on itself".
"SetEventsJournalEnabled": true,
"RunEventsMaintenanceTx": true,
"SetVersionedHistoryEnabled": true,
"SetEventsJournalEnabled": true,
// EventsJournalActivationError (PR A1) forwards the inner provider's
// shape-probe result the same way SetEventsJournalEnabled forwards
// activation: a query, not a role constructor.
"EventsJournalActivationError": true,
"RunEventsMaintenanceTx": true,
"SetVersionedHistoryEnabled": true,
}
var checked int

View file

@ -71,3 +71,26 @@ func TestApplyProviderOptions(t *testing.T) {
t.Fatal("applyProviderOptions(WithPreview()).preview = false, want true")
}
}
// TestOpensForBeadWrites pins which postures events-journal activation treats
// as writing beads: only the ordinary open. A preview, a read-only open and a
// working-set reconcile write none, so a journal table the journal cannot run
// against must not refuse them — and the reconcile marker must not change the
// open itself.
func TestOpensForBeadWrites(t *testing.T) {
if !OpensForBeadWrites() {
t.Fatal("OpensForBeadWrites() = false, want true for the ordinary open")
}
for name, opt := range map[string]ProviderOption{
"preview": WithPreview(),
"read-only": WithReadOnly(),
"working-set reconcile": WithWorkingSetReconcile(),
} {
if OpensForBeadWrites(opt) {
t.Errorf("OpensForBeadWrites(%s) = true, want false", name)
}
}
if got := applyProviderOptions([]ProviderOption{WithWorkingSetReconcile()}); got.preview || got.readOnly {
t.Errorf("WithWorkingSetReconcile changed the open itself: %+v", got)
}
}