Commit graph

8 commits

Author SHA1 Message Date
Julian Knutsen
f3c47390e7
fix(storage): refuse graph_mode workspaces on library opens (#7721)
#7671 fences graph workspaces in cmd/bd: command admission, the store
factories and the other CLI paths refuse one, and #7722 extends that
fence to a .beads holding the graph-preview-format marker whatever its
metadata says. But the public library opens (beads.OpenBestAvailable,
OpenBestAvailableWith, OpenFromConfig, OpenGated) and the Dolt store's
metadata-driven constructors (NewFromConfig, NewFromConfigWithOptions,
NewFromConfigWithCLIOptions) still opened either kind of graph workspace
as an Issue store and ran Issue schema migrations into the graph
database (or, with no metadata beside the marker, created an Issue
database inside the graph workspace).

- Move the typed refusal into internal/configfile
  (UnsupportedWorkspaceFormatError with BeadsDir, GraphMode and
  FormatMarker; CheckWorkspaceFormat; GraphFormatMarkerFile;
  HasGraphFormatMarker) with byte-identical text and hints for both the
  graph_mode and the marker refusal, and move cmd/bd's discovery
  pre-checks there as LoadAndCheckWorkspaceFormat and
  CheckDiscoveredWorkspaceFormat (LoadForDiscovery, which never migrates
  a legacy config.json; unreadable metadata beside the marker is refused
  as the marker's format). cmd/bd's checkWorkspaceFormat,
  loadAndCheckWorkspaceFormat, hasGraphFormatMarker and
  checkDiscoveredWorkspaceFormat become thin delegates, so the CLI keeps
  its behaviour (exit 16 with the same hints) and every caller of them is
  unchanged; bootstrap's ancestor walk builds the configfile type.
- Every public metadata-driven open runs CheckDiscoveredWorkspaceFormat
  as its first statement: before OpenGated takes its gates and before
  OpenBestAvailableWith's Load can migrate config.json. A refused open
  leaves the workspace byte-identical. The type is exported as
  beads.UnsupportedWorkspaceFormatError.
- The Dolt constructors run CheckDiscoveredWorkspaceFormat before
  configfile.Load, so a graph workspace marked only by a legacy
  config.json, or by the marker beside stripped, missing or unparseable
  metadata, is refused before Load can migrate it into metadata.json
  (and drop graph_* keys this build does not know). requireDoltBackend
  re-checks the loaded config as a backstop, so they also refuse before
  resolving a server or writing a port file or Dolt root.

Tests: configfile unit tests for both refusals (exact marker text and
hint, graph_mode winning over the marker, unparseable metadata with and
without the marker); the public opens and the three Dolt constructors
against graph_mode workspaces and against marker-only, stripped
(metadata.json and legacy config.json), server, explicit dependency and
unparseable-metadata marker workspaces, each refused with the typed
error and the workspace left unchanged.

Agent-Signature: claude-code-claude-opus-5-5-unknown-reasoning on behalf of Julian Knutsen

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 09:29:54 +00:00
Julian Knutsen
2f1e8be3ca
feat(backends): registry Remote + OpenWith options; beads.OpenBestAvailableWith for embedders (#7188)
* feat(backends): registry Remote + optional OpenWith(ctx, dir, OpenOptions)

Slice S1: backends.Backend gains Remote bool and an optional
OpenWith(ctx, beadsDir, OpenOptions{Credential, HTTPClient, UserAgent}) so a
registered extension backend can honor a per-open credential instead of
dialing with a process-global client — the motivating case is one embedder
process serving many workspaces with distinct credentials. Open keeps
working unmodified for every existing backend: Backend.OpenWithOptions calls
OpenWith when present, otherwise falls back to Open and ignores
HTTPClient/UserAgent, but refuses (ErrCredentialWithoutOpenWith) rather than
silently dropping a non-nil Credential.

Public beads.OpenBestAvailableWith(ctx, beadsDir, opts) carries this through
the SDK surface (mirrored in beads_nocgo.go and aliased in backend/backend.go
and beads.go); OpenBestAvailable is now that call with a zero OpenOptions,
identical to its prior behavior for Dolt, embedded Dolt, and every registered
backend. The plain-Dolt path (no registered backend) applies the same
Credential refusal, since Dolt has no per-open credential seam either.

doltserver.ResolvePhysicalRoots now checks backends.IsRemote(cfg.GetBackend())
before every Dolt-mode branch, including the shared-server check (which reads
only env/config.yaml and never consults cfg) — without this ordering, a global
BEADS_DOLT_SHARED_SERVER=1 would hijack a remote-backend workspace into
reporting a bogus local Dolt root it will never open.

Tests: backends_test new file open_with_test.go covers OpenWithOptions
dispatch/back-compat/Credential refusal and Remote/IsRemote; root package
open_backend_selection_test.go adds a fake OpenWith registrant proving
dispatch and option pass-through through OpenBestAvailableWith, zero-options
back-compat for every path (registered w/ and w/o OpenWith, plain Dolt), and
the Credential-without-OpenWith refusal on both paths; doltserver
physical_root_test.go adds the registered-remote-backend case including the
shared-server-override ordering regression test. Verified under both cgo and
nocgo builds. Live-mutated each new guard (the Credential refusal, the
OpenWith dispatch, the ResolvePhysicalRoots remote-backend case, and the
plain-Dolt Credential refusal) one at a time and confirmed RED before
reverting to GREEN.

No HTTP client code added; that is a later slice.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fixup(backends): address Opus review findings on OpenWith (M1-M3, L1-L2)

- M1: frame cmd/bd store-open errors for remote backends as unreachable
  server failures instead of Dolt-shaped "failed to open database"
  advice (openStoreErrorMessage, backendNameForErrorFraming), with tests.
- M2: rewrite the OpenBestAvailable/OpenBestAvailableWith back-compat
  subtest to assert fixed expected outcomes (error identity/message,
  returned store behavior) now that the two functions are identical,
  instead of comparing them to each other.
- M3: document and enforce the OpenWith contract: an implementation that
  receives a non-nil Credential it cannot type-assert to its expected
  type must refuse with the new ErrUnsupportedCredential sentinel,
  verified with a fake registrant.
- L1: document that a nil OpenOptions.Credential means the backend's
  default/ambient auth, and that multi-tenant embedders must pass a
  non-nil Credential (RequireCredential deferred to S2).
- L2: fail closed on HTTPClient/UserAgent passed to a backend without
  OpenWith, mirroring the existing Credential refusal, via new
  ErrHTTPClientWithoutOpenWith/ErrUserAgentWithoutOpenWith sentinels.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* build(bazel): auto-sync BUILD files

Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37195230844
(`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy).
Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see
scripts/bazel-autofix-push.sh.

* fix(backends): keep the store-open error chain through the remote framing

Maintainer review fixes for the adopted PR (iteration 1).

openStoreErrorMessage returned a string, so ensureStoreActiveWithContext
rebuilt its error from text (%s) where base wrapped the open error with %w,
and the chain ended at the CLI's framing. Prime tells a store-open timeout
apart from a generic "storage unavailable" only through
errors.Is(err, context.DeadlineExceeded) on what ensureStoreActiveWithContext
returns (cmd/bd/prime.go), so the flattening silently downgraded that
diagnostic for every backend, Dolt included, not just the new Remote ones.

Replace the helper with openStoreError, which returns an error wrapping err
with %w on both arms. The message text is unchanged: "failed to open
database: <err>" for Dolt and any registered backend that is not Remote,
"failed to reach remote backend %q: <err>" for a Remote one. direct_mode.go
wraps the result with %w ahead of its Hint line; main.go keeps printing it
through HandleError("%v", ...), which only formats.

Tests:

- TestOpenStoreErrorFramesRemoteBackends (renamed from
  TestOpenStoreErrorMessageFramesRemoteBackends) now also asserts errors.Is
  on every arm, and the exact original text for the non-Remote arms.
- TestEnsureStoreActiveKeepsOpenErrorChain drives the real
  ensureStoreActiveWithContext against a registered backend whose open
  outlives its context, for a local and a Remote backend, and requires
  errors.Is(err, context.DeadlineExceeded) plus the framing prefix and the
  Hint line.
- TestFormatMemoriesForPrimeTimesOutOpeningRegisteredBackend is the
  unstubbed twin of TestFormatMemoriesForPrimeTimesOutOpeningStore and
  requires the timeout banner, not the "storage unavailable" one.

Measured at this head: turning direct_mode.go's outer %w into %v reddens
both chain-test rows and the prime test; turning the helper's %w into %v
reddens the framing test as well.

Non-gating cleanup in the same area:

- beads_cgo.go, beads_nocgo.go: OpenBestAvailableWith's doc and dispatch
  comments said only a non-nil Credential is refused. The code also refuses
  a non-nil HTTPClient and a non-empty UserAgent, each with its own
  ErrXWithoutOpenWith sentinel.
- internal/doltserver/physical_root.go: the Mode doc lists "remote-backend",
  and the RemoteBackend doc covers a registered Remote backend as well as a
  Dolt server on a non-local host.
- open_backend_selection_test.go: the plain-Dolt refusal test is a table
  over Credential, HTTPClient and UserAgent (renamed
  TestOpenBestAvailableWithRefusesPerOpenOptionsOnPlainDolt). Disabling the
  nocgo HTTPClient arm or the cgo UserAgent arm reddens only that row. The
  zero-options test's comment had the delegation backwards and pointed at
  beads.go; OpenBestAvailable delegates to OpenBestAvailableWith in
  beads_cgo.go and beads_nocgo.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
2026-10-04 09:34:36 -07:00
Julian Knutsen
b13fab0151
fix(storage): truthful legacy-backend tombstone rejection — detect present Dolt data, give the exact heal (D-8) (#6044)
* fix(configfile): detect present Dolt data and give the exact metadata heal (D-8)

A workspace whose metadata.json names sqlite/postgres/mysql stays fail-closed,
but the rejection text was not truthful for the common case. bd v1.2.x opened
those workspaces as Dolt regardless of the "backend" value, so for them the
stale field is the only problem — and the old remedy sent the user to export
and reinitialize, destroying a database that is one JSON edit from healthy.

ExistingDoltDataDir stats the embedded (.beads/embeddeddolt) and server-mode
(.beads/dolt, dolt_data_dir, BEADS_DOLT_DATA_DIR) layouts for a .dolt directory,
either at the root or one level down. Detection is stat/readdir only — every
caller is on a path that has promised not to touch storage. The detection
approach is ported from the obsolete PR #4740.

The removed- and unknown-backend messages now branch on it: with Dolt data
present they name metadata.json and the exact edit ("backend": "dolt", or drop
the field); without it they keep today's export/reinitialize path, so the
substrings existing tests pin are unchanged. GetDoltDataDir becomes nil-safe
like GetBackend, since rejection paths resolve it for configs that may be nil.

Co-authored-by: Steve Yegge <steve.yegge@gmail.com>

* fix(storage): thread beadsDir into every legacy-backend rejection (D-8)

All three rejection surfaces now branch on whether the workspace actually holds
a Dolt database: cmd/bd's validateConfiguredBackend (12 call sites, every one
already had beadsDir in scope), internal/storage/dolt's requireDoltBackend, and
the public beads.OpenBestAvailable path. Widening the signatures rather than
adding parallel *At helpers is deliberate — it makes each site state which
workspace it is rejecting, so none can silently keep the generic message.

Behavior is unchanged for workspaces with no Dolt data; test call sites take the
new argument but no assertion changes.

* test(storage): cover the tombstone heal branch and its round-trip (D-8)

The headline test is the round-trip: init a real Dolt workspace, hand-edit
metadata.json to backend:"sqlite", watch bd list exit 1 with the heal message,
apply exactly the edit that message prescribes, and confirm bd list opens the
workspace with its issue intact. The message's stated alternative — deleting the
"backend" field — is exercised too, so both remedies it offers are proven rather
than asserted.

The same test pins the fail-closed guarantee at a level a stat loop could not:
metadata.json stays byte-identical and the whole embedded Dolt tree is unchanged
in paths and sizes after the refusal.

Around it: the ExistingDoltDataDir layout matrix (embedded and server layouts at
both depths, relative and absolute dolt_data_dir, BEADS_DOLT_DATA_DIR, decoys, no
workspace dir), string-level branch coverage in internal/configfile, a public
OpenBestAvailable case proving beadsDir reaches the library refusal, and an
explicit assertion that the no-Dolt-data branch still refuses to suggest a heal
that would open an empty store.

---------

Co-authored-by: Steve Yegge <steve.yegge@gmail.com>
2026-08-28 23:27:39 -07:00
Julian Knutsen
9fddc56055
feat(storage): add extension-safe backend registry seam (#4859)
* feat(storage): add downstream backend registry seam (bd-7oj)

Salvage the name-keyed extension contract from PR #4859 on current main while preserving the Dolt-only OSS policy. Stock builds register no alternate backends; tombstones and unknown names still fail closed. Downstream builds can register read-write/read-only opens and workspace discovery without editing shared dispatch.

Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of CI Bot

* fix(storage): dispatch and fail-closed for registered backends

Address review majors on the backend registry seam:

- bd init rejects a registered backend (open-existing-workspace-only)
  instead of silently provisioning Dolt and persisting backend: dolt.
- OpenBestAvailable dispatches registered backends before any Dolt path
  in both CGO and non-CGO builds, mirroring the CLI store factories.
- Public FindDatabasePath discovers WorkspaceIsBeadsDir workspaces via
  the shared registry authority instead of returning no database.
- Root command reuses the resolved .beads directory for hook-runner and
  molecule-catalog setup instead of filepath.Dir(dbPath). For a registered
  WorkspaceIsBeadsDir backend dbPath is the .beads directory itself, so
  filepath.Dir(dbPath) loaded hooks from <repo>/hooks and molecules from
  <repo>/molecules.jsonl instead of .beads/hooks and .beads/molecules.jsonl.

Add a single WorkspaceIsBeadsDir authority, document the init-time
registration invariant, and cover the paths with tests, including a
root-command test proving the .beads hook and molecule directories are
selected for WorkspaceIsBeadsDir backends.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-31 03:05:48 -07:00
Test User
6fd9dbda7d refactor(storage)!: remove the SQLite backend and dialect
Extend #4847's rollback to the last alternative backend: bd now ships a
single storage engine (Dolt, embedded and server mode).

- Delete internal/storage/{sqlite,sqlitedialect} and the packages the
  removal orphans: sqlkit (sole importer was the sqlite store),
  unsupportedgen (sole go:generate consumer), the NonCommitGraphBackend
  marker and its two dead consumer checks, and the domain-db parity test
  that used sqlite as its harness. Drop modernc.org/sqlite from go.mod
  and regenerate the nix vendorHash.
- Tombstone backend="sqlite" exactly like postgres/mysql: recognized in
  metadata (never silently aliased to a fresh Dolt DB), rejected before
  any open with its own rationale and the export -> re-init -> import
  guidance. The fail-closed guarantee now also covers never creating the
  beads.db file the removed backend would have provisioned.
- bd init: --backend=sqlite reaches removed-backend guidance;
  --sqlite-path becomes a hidden parser tombstone like the pg/mysql
  flags; help text and supported-backend enumerations say Dolt only.
- Doctor: sqlite workspaces fail closed like the other tombstones; the
  modern sqlite stub payloads are gone. The legacy pre-Dolt surface
  (artifact scans, migration validation, migrate-sqlite-to-current.sh,
  the migration-test harness) is untouched; genuine pre-Dolt workspaces
  carry no backend field and never hit the tombstone.
- Conformance: drop the sqlite profile; the CLI differential runner
  idles (skip, not fail) until another backend profile is registered,
  and the export/import round-trip keeps running against the embedded
  Dolt reference. Tier-1 oracle legs are unchanged.

Backend selection has never shipped in a release (v1.1.0 predates it),
so released workflows are unaffected and the pinned CLI docs already
describe the resulting behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 02:43:56 +00:00
Test User
0d7cf0022d Reapply "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857)
This reverts commit 0cf3b3ca09.
2026-07-18 02:43:56 +00:00
Julian Knutsen
0cf3b3ca09
Revert "Simplify the supported backends: roll back the direct PostgreSQL and MySQL adapters" (#4857) 2026-07-16 16:35:51 -07:00
Test User
cafb6888bd refactor(storage): remove direct PostgreSQL and MySQL adapters (bd-sadcd)
Roll back the direct adapters, dialect layers, credential handling, dependencies, CI services, and conformance profiles for general-purpose server databases. That support adds operational and testing complexity at odds with our goal of keeping Beads as simple as possible and consuming as few resources as possible.

Keep the Storage and DoltStorage interfaces unchanged. Embedded Dolt, Dolt server, and SQLite remain supported; the MySQL wire driver remains because Dolt server uses it.

Retain legacy metadata identifiers and hidden init flags only as compatibility tombstones. Removed, unknown, or corrupt backend metadata now fails safely across CLI, SDK, bootstrap, doctor, factories, and Dolt administration before storage is opened or modified, with guidance to export using a supporting build and reinitialize via bd help init-safety.

Agent-Signature: codex-gpt-5-unknown-reasoning on behalf of Test User
2026-07-16 03:35:18 +00:00