beads/backend/backend.go
Julian Knutsen 2f1e8be3ca
feat(backends): registry Remote + OpenWith options; beads.OpenBestAvailableWith for embedders (#7188)
* feat(backends): registry Remote + optional OpenWith(ctx, dir, OpenOptions)

Slice S1: backends.Backend gains Remote bool and an optional
OpenWith(ctx, beadsDir, OpenOptions{Credential, HTTPClient, UserAgent}) so a
registered extension backend can honor a per-open credential instead of
dialing with a process-global client — the motivating case is one embedder
process serving many workspaces with distinct credentials. Open keeps
working unmodified for every existing backend: Backend.OpenWithOptions calls
OpenWith when present, otherwise falls back to Open and ignores
HTTPClient/UserAgent, but refuses (ErrCredentialWithoutOpenWith) rather than
silently dropping a non-nil Credential.

Public beads.OpenBestAvailableWith(ctx, beadsDir, opts) carries this through
the SDK surface (mirrored in beads_nocgo.go and aliased in backend/backend.go
and beads.go); OpenBestAvailable is now that call with a zero OpenOptions,
identical to its prior behavior for Dolt, embedded Dolt, and every registered
backend. The plain-Dolt path (no registered backend) applies the same
Credential refusal, since Dolt has no per-open credential seam either.

doltserver.ResolvePhysicalRoots now checks backends.IsRemote(cfg.GetBackend())
before every Dolt-mode branch, including the shared-server check (which reads
only env/config.yaml and never consults cfg) — without this ordering, a global
BEADS_DOLT_SHARED_SERVER=1 would hijack a remote-backend workspace into
reporting a bogus local Dolt root it will never open.

Tests: backends_test new file open_with_test.go covers OpenWithOptions
dispatch/back-compat/Credential refusal and Remote/IsRemote; root package
open_backend_selection_test.go adds a fake OpenWith registrant proving
dispatch and option pass-through through OpenBestAvailableWith, zero-options
back-compat for every path (registered w/ and w/o OpenWith, plain Dolt), and
the Credential-without-OpenWith refusal on both paths; doltserver
physical_root_test.go adds the registered-remote-backend case including the
shared-server-override ordering regression test. Verified under both cgo and
nocgo builds. Live-mutated each new guard (the Credential refusal, the
OpenWith dispatch, the ResolvePhysicalRoots remote-backend case, and the
plain-Dolt Credential refusal) one at a time and confirmed RED before
reverting to GREEN.

No HTTP client code added; that is a later slice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fixup(backends): address Opus review findings on OpenWith (M1-M3, L1-L2)

- M1: frame cmd/bd store-open errors for remote backends as unreachable
  server failures instead of Dolt-shaped "failed to open database"
  advice (openStoreErrorMessage, backendNameForErrorFraming), with tests.
- M2: rewrite the OpenBestAvailable/OpenBestAvailableWith back-compat
  subtest to assert fixed expected outcomes (error identity/message,
  returned store behavior) now that the two functions are identical,
  instead of comparing them to each other.
- M3: document and enforce the OpenWith contract: an implementation that
  receives a non-nil Credential it cannot type-assert to its expected
  type must refuse with the new ErrUnsupportedCredential sentinel,
  verified with a fake registrant.
- L1: document that a nil OpenOptions.Credential means the backend's
  default/ambient auth, and that multi-tenant embedders must pass a
  non-nil Credential (RequireCredential deferred to S2).
- L2: fail closed on HTTPClient/UserAgent passed to a backend without
  OpenWith, mirroring the existing Credential refusal, via new
  ErrHTTPClientWithoutOpenWith/ErrUserAgentWithoutOpenWith sentinels.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37195230844
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

* fix(backends): keep the store-open error chain through the remote framing

Maintainer review fixes for the adopted PR (iteration 1).

openStoreErrorMessage returned a string, so ensureStoreActiveWithContext
rebuilt its error from text (%s) where base wrapped the open error with %w,
and the chain ended at the CLI's framing. Prime tells a store-open timeout
apart from a generic "storage unavailable" only through
errors.Is(err, context.DeadlineExceeded) on what ensureStoreActiveWithContext
returns (cmd/bd/prime.go), so the flattening silently downgraded that
diagnostic for every backend, Dolt included, not just the new Remote ones.

Replace the helper with openStoreError, which returns an error wrapping err
with %w on both arms. The message text is unchanged: "failed to open
database: <err>" for Dolt and any registered backend that is not Remote,
"failed to reach remote backend %q: <err>" for a Remote one. direct_mode.go
wraps the result with %w ahead of its Hint line; main.go keeps printing it
through HandleError("%v", ...), which only formats.

Tests:

- TestOpenStoreErrorFramesRemoteBackends (renamed from
  TestOpenStoreErrorMessageFramesRemoteBackends) now also asserts errors.Is
  on every arm, and the exact original text for the non-Remote arms.
- TestEnsureStoreActiveKeepsOpenErrorChain drives the real
  ensureStoreActiveWithContext against a registered backend whose open
  outlives its context, for a local and a Remote backend, and requires
  errors.Is(err, context.DeadlineExceeded) plus the framing prefix and the
  Hint line.
- TestFormatMemoriesForPrimeTimesOutOpeningRegisteredBackend is the
  unstubbed twin of TestFormatMemoriesForPrimeTimesOutOpeningStore and
  requires the timeout banner, not the "storage unavailable" one.

Measured at this head: turning direct_mode.go's outer %w into %v reddens
both chain-test rows and the prime test; turning the helper's %w into %v
reddens the framing test as well.

Non-gating cleanup in the same area:

- beads_cgo.go, beads_nocgo.go: OpenBestAvailableWith's doc and dispatch
  comments said only a non-nil Credential is refused. The code also refuses
  a non-nil HTTPClient and a non-empty UserAgent, each with its own
  ErrXWithoutOpenWith sentinel.
- internal/doltserver/physical_root.go: the Mode doc lists "remote-backend",
  and the RemoteBackend doc covers a registered Remote backend as well as a
  Dolt server on a non-local host.
- open_backend_selection_test.go: the plain-Dolt refusal test is a table
  over Credential, HTTPClient and UserAgent (renamed
  TestOpenBestAvailableWithRefusesPerOpenOptionsOnPlainDolt). Disabling the
  nocgo HTTPClient arm or the cgo UserAgent arm reddens only that row. The
  zero-options test's comment had the delegation backwards and pointed at
  beads.go; OpenBestAvailable delegates to OpenBestAvailableWith in
  beads_cgo.go and beads_nocgo.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-04 09:34:36 -07:00

231 lines
9.9 KiB
Go

// Package backend is the public surface for OUT-OF-TREE storage backends: it
// exports, by type alias, the storage contract an external Go module must
// implement (the engine interface storage.DoltStorage and every type its
// method signatures reach), the process-init registry that plugs an
// implementation into bd, and — via the backend/conformance subpackage — the
// tests that prove an implementation behaves like the Dolt reference.
//
// Every name here is an alias or thin wrapper over the in-tree definition, so
// an external implementation and the in-tree consumers see IDENTICAL types:
// a store whose methods are written against these aliases satisfies
// storage.DoltStorage, and sentinel errors re-exported here match errors.Is
// across the module boundary. The internal packages stay the single source of
// truth; this package adds no behavior.
//
// A minimal external backend looks like:
//
// package mybackend
//
// import "github.com/steveyegge/beads/backend"
//
// type Store struct{ ... } // implements backend.DoltStorage
//
// func init() {
// backend.Register("mybackend", backend.Backend{
// Open: open,
// OpenReadOnly: openReadOnly,
// })
// }
//
// and proves itself with the conformance package:
//
// func TestConformance(t *testing.T) {
// // What the backend declares it does not do.
// conformance.RunUnsupportedContract(t, &Store{}, unsupported)
// // What it does: the portable raw surface.
// conformance.RunAll(t, func(t *testing.T) backend.DoltStorage {
// return newTestStore(t)
// })
// }
//
// The two halves are not interchangeable. RunAll covers the portable raw
// surface — roughly half of core Storage plus five capability sub-interfaces —
// and calls NO method of VersionControl, HistoryViewer, RemoteStore, SyncStore,
// FederationStore, CompactionStore or FastStatisticsStore. Those seven are
// proved only by the allowlist: a backend that answers them with a typed
// ErrUnsupported and runs RunUnsupportedContract has satisfied everything the
// suite asks of them. The conformance package doc measures the split, and the
// role contracts in that package are the third piece, covering the issueops
// roles the accessors on Storage return.
//
// Registration is init-time wiring only; see Register. bd init / bd bootstrap
// do not provision registered backends — an external backend supplies its own
// workspace-creation path and is opened by name via metadata.json.
//
// # Stability
//
// EXPERIMENTAL. This surface is exported so external backends can be built
// and conformance-gated while the extension door stabilizes; the contract
// itself still evolves. Adding a required method to the engine interface is a
// breaking change for out-of-tree implementations — such additions are called
// out in CHANGELOG.md — and no compatibility promise beyond that is made yet.
// Pin an exact beads version and re-run the conformance suite on every bump.
package backend
import (
"context"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/backends"
)
// DoltStorage is the full engine interface an external backend implements:
// the core Storage interface plus every capability sub-interface. The name is
// historical — the Dolt stores were its first implementations — but the
// contract is backend-agnostic.
//
// Implementing it is not the same as being exercised by the conformance suite,
// which reaches roughly half of it; the rest is declared through the unsupported
// allowlist. See the conformance package doc for which half is which.
type DoltStorage = storage.DoltStorage
// The engine interface decomposes into the core Storage interface plus these
// capability sub-interfaces; each is aliased so an implementation can be
// organized (and documented) per capability.
type (
Storage = storage.Storage
IssueLifecycleStore = storage.IssueLifecycleStore
VersionControl = storage.VersionControl
HistoryViewer = storage.HistoryViewer
RemoteStore = storage.RemoteStore
SyncStore = storage.SyncStore
FederationStore = storage.FederationStore
BulkIssueStore = storage.BulkIssueStore
DependencyQueryStore = storage.DependencyQueryStore
EventQueryStore = storage.EventQueryStore
AnnotationStore = storage.AnnotationStore
ConfigMetadataStore = storage.ConfigMetadataStore
CompactionStore = storage.CompactionStore
AdvancedQueryStore = storage.AdvancedQueryStore
FastStatisticsStore = storage.FastStatisticsStore
)
// Transaction types: Storage.RunInTransaction yields a Transaction;
// IssueLifecycleStore's guarded mutations run inside an
// IssueLifecycleTransaction.
type (
Transaction = storage.Transaction
IssueLifecycleTransaction = storage.IssueLifecycleTransaction
)
// Value types reached by the engine interface's method signatures.
type (
BatchCreateOptions = storage.BatchCreateOptions
CloseIssueOptions = storage.CloseIssueOptions
CloseIssueResult = storage.CloseIssueResult
CommentPageCursor = storage.CommentPageCursor
CommitInfo = storage.CommitInfo
Conflict = storage.Conflict
DependencyAddOptions = storage.DependencyAddOptions
DependencyRemoveOptions = storage.DependencyRemoveOptions
DiffEntry = storage.DiffEntry
EventCursor = storage.EventCursor
FederationPeer = storage.FederationPeer
HistoryEntry = storage.HistoryEntry
MergeSlotResult = storage.MergeSlotResult
MergeSlotStatus = storage.MergeSlotStatus
RemoteInfo = storage.RemoteInfo
StatusEntry = storage.StatusEntry
SyncResult = storage.SyncResult
SyncStatus = storage.SyncStatus
UpdateIssueOptions = storage.UpdateIssueOptions
)
// VCStatus is the version-control status report (storage.Status). Named
// VCStatus here — matching the root beads package — because Status is the
// issue-status enum from the domain types.
type VCStatus = storage.Status
// Backend describes a registered storage backend: how to open its workspace
// store. See the field docs on the aliased struct.
type Backend = backends.Backend
// OpenOptions carries per-open injections (Credential, HTTPClient,
// UserAgent) that Backend.OpenWith may use. See the field docs on the
// aliased struct.
type OpenOptions = backends.OpenOptions
// Credential is the opaque per-open credential marker OpenOptions.Credential
// carries. A backend that needs one declares its own narrower interface and
// type-asserts the value its OpenWith receives. See the aliased type's docs.
type Credential = backends.Credential
// ErrCredentialWithoutOpenWith is returned by Backend.OpenWithOptions when
// OpenOptions.Credential is set but the backend has no OpenWith to honor it.
var ErrCredentialWithoutOpenWith = backends.ErrCredentialWithoutOpenWith
// ErrHTTPClientWithoutOpenWith is returned by Backend.OpenWithOptions when
// OpenOptions.HTTPClient is set but the backend has no OpenWith to honor it.
// Same fail-closed family as ErrCredentialWithoutOpenWith.
var ErrHTTPClientWithoutOpenWith = backends.ErrHTTPClientWithoutOpenWith
// ErrUserAgentWithoutOpenWith is returned by Backend.OpenWithOptions when
// OpenOptions.UserAgent is set but the backend has no OpenWith to honor it.
// Same fail-closed family as ErrCredentialWithoutOpenWith.
var ErrUserAgentWithoutOpenWith = backends.ErrUserAgentWithoutOpenWith
// ErrUnsupportedCredential is the typed refusal an OpenWith implementation
// MUST return (directly or wrapped, so errors.Is still matches) when it
// receives a non-nil Credential that does not type-assert to the narrower
// credential interface that backend's OpenWith expects. See the aliased
// error's docs in internal/storage/backends for the full contract.
var ErrUnsupportedCredential = backends.ErrUnsupportedCredential
// Register adds a backend under name. It is process-start wiring: call it
// once during initialization (typically from a registrant package's init that
// the embedding binary blank-imports), before any concurrent store access.
// Invalid or duplicate registrations panic; the name "dolt" is reserved.
func Register(name string, backend Backend) {
backends.Register(name, backend)
}
// Deregister removes a backend. It exists for isolated single-threaded
// contract tests; production registrants never deregister.
func Deregister(name string) bool {
return backends.Deregister(name)
}
// Lookup returns the backend registered under name.
func Lookup(name string) (Backend, bool) {
return backends.Lookup(name)
}
// Registered reports whether name has a backend implementation.
func Registered(name string) bool {
return backends.Registered(name)
}
// WorkspaceIsBeadsDir reports whether name is a registered backend whose
// workspace is identified by the .beads directory alone (metadata.json plus a
// remote store), with no separately discoverable local database.
func WorkspaceIsBeadsDir(name string) bool {
return backends.WorkspaceIsBeadsDir(name)
}
// IsRemote reports whether name is a registered backend that is a pure
// network client of a bd serve with no local database (see Backend.Remote).
func IsRemote(name string) bool {
return backends.IsRemote(name)
}
// Iter is the streaming-read cursor returned by the Iter* methods.
type Iter[T any] = storage.Iter[T]
// SliceIter adapts an in-memory slice to Iter — the simplest way for a
// backend to satisfy the Iter* methods before it streams natively.
type SliceIter[T any] = storage.SliceIter[T]
// NewSliceIter returns an Iter over items.
func NewSliceIter[T any](items []*T) *SliceIter[T] {
return storage.NewSliceIter(items)
}
// ForEach drains it, invoking fn per element, and closes it.
func ForEach[T any](ctx context.Context, it Iter[T], fn func(*T) error) error {
return storage.ForEach(ctx, it, fn)
}
// Collect drains it into a slice and closes it.
func Collect[T any](ctx context.Context, it Iter[T]) ([]*T, error) {
return storage.Collect(ctx, it)
}