Commit graph

118 commits

Author SHA1 Message Date
Harry
be65f59cd0 fix(dolt): batch wisp-ID partition in bulk hydrators (GH#3414)
Every mutating bd command against a remote Dolt backend hung ~10–15s
with `auto-export failed: ... hydrate labels: ... context canceled`.

Root cause: five bulk fetchers in internal/storage/issueops partitioned
IDs into wisp vs permanent buckets by calling IsActiveWispInTx once per
ID, and each call is a round-trip to the SQL server. With ~600 issues
over WAN latency that's ~12s of sequential SELECTs *before* the real
label query runs — blowing past the context deadline.

Fix:
- Add PartitionWispIDsInTx that batches the check into one
  `SELECT id FROM wisps WHERE id IN (...)` per queryBatchSize chunk,
  reducing cost from O(N) to O(1 + N/200) round-trips. Falls back to
  "no wisps" when the wisps table is absent (pre-migration dbs).
- Rewrite GetLabelsForIssuesInTx, GetCommentsForIssuesInTx,
  GetDependencyRecordsForIssuesInTx, GetBlockingInfoForIssuesInTx, and
  DeleteIssuesInTx to use it.
- Add fast-path helpers GetLabelsForIssuesFromTableInTx and
  GetDependencyRecordsForIssuesFromTableInTx for callers (searchTableInTx)
  that already know every ID is in a single table, skipping the partition
  round-trip entirely on the hot auto-export path.

Tests:
- New wisp_partition_test.go covering the partitioner directly: empty,
  all-perm, all-wisp, mixed-with-unknown, input-order preservation, and
  a 450-ID large-batch case crossing the queryBatchSize boundary.
- End-to-end mixed-routing regression tests for GetCommentsForIssues,
  GetDependencyRecordsForIssues, and a 250-ID GetLabelsForIssues case
  that crosses the chunk boundary.

Made-with: Cursor
2026-05-02 13:27:24 -07:00
coffeegoddd☕️✨
e97532ab5e :remove uneccessary locking 2026-05-01 17:46:37 -07:00
coffeegoddd☕️✨
e211188bbd /{cmd,internal}: fix import jsonl 2026-05-01 14:34:45 -07:00
coffeegoddd☕️✨
7e0d09cbf8 /{internal,cmd}: fix CommitPending 2026-05-01 11:15:02 -07:00
coffeegoddd☕️✨
05251d1b7c /{cmd,internal}: some cleanup 2026-04-30 12:46:54 -07:00
coffeegoddd☕️✨
73f2ab613b /{cmd,internal}: maybe fix tool tip closed store bug 2026-04-29 16:35:16 -07:00
coffeegoddd☕️✨
58dc2e1b8b /{beads_cgo,cmd,internal}: cache open embedded stores deterministically to prevent deadlocking when multiple store open attempts are made against an opened store 2026-04-29 15:49:51 -07:00
matt wilkie
a97eec9739 Add atomic ready claim (GH-3570) 2026-04-28 07:11:13 -07:00
matt wilkie
e97b0bc581
Merge pull request #3510 from seanmartinsmith/fix/test-subprocess-build-tag
fix(test): thread -tags gms_pure_go to subprocess build in embeddeddolt concurrency test
2026-04-27 07:29:40 -07:00
matt wilkie
0bc4c7256a
feat: add JSONL bulk dep add (#3530)
* perf: add --no-cycle-check flag to bd dep add for bulk wiring

On large dependency graphs (230+ edges), bd dep add performs a full
DetectCycles traversal after every insertion — O(V+E) per call. This
causes individual dep add calls to hang for 3+ minutes as the graph
grows beyond ~150 edges.

Add --no-cycle-check to both `dep add` and the `dep --blocks`
shorthand. When set, the post-insert warnIfCyclesExist call is skipped.
Callers doing bulk wiring can run `bd dep cycles` once at the end to
verify the complete graph.

The flag is intentionally not the default: single-edge interactive use
keeps cycle detection on. Bulk callers opt in explicitly.

* Add bulk JSONL dependency wiring (bd-wjl)

* Fix bulk dep file lint warning

---------

Co-authored-by: Robin Carswell <robin@robincarswell.com>
2026-04-27 04:51:46 -07:00
sms
8475f8bc4a fix(test): thread -tags gms_pure_go to subprocess build in embeddeddolt concurrency test
TestConcurrencyMultiProcess shells out to `go test -c` to build a child
test binary that subprocesses then exec. The build invocation didn't
pass `-tags gms_pure_go`, so on Windows the subprocess build pulled in
the cgo-backed ICU regex package and failed at compile time on a missing
`unicode/uregex.h` header.

Linux/macOS toolchains ship ICU dev headers by default so the cgo path
compiled silently without the tag — Windows was structurally broken.

Per CLAUDE.md, `gms_pure_go` is mandatory project-wide. Pass it through
to the subprocess build and add a comment explaining the constraint
since it's not obvious from Go conventions alone.

Fixes #3508
2026-04-25 22:21:54 -04:00
seanmartinsmith
f368f98830
fix: wire embedded backend compat migrations by relocating runner (#3413)
The embedded Dolt backend open path ran only the embedded SQL
migrations via schema.MigrateUp. The defensive backward-compat
migration runner was invoked only from the server-mode path in
dolt.initSchemaOnDB. Pre-existing embedded databases opened by a
newer bd binary therefore skipped every compat migration 001-017,
leaving missing columns and tables unrepaired.

The runner itself has no coupling to package dolt -- it drives
idempotent migration functions that already live in the migrations/
subpackage. Relocating CompatMigration, compatMigrationsList,
RunCompatMigrations, and ListCompatMigrations into package migrations
puts them alongside the functions they drive, and lets both the
server and embedded backends share them without a cross-backend
import.

Changes:
- New internal/storage/dolt/migrations/runner.go with the runner,
  registry list, and the exported List function.
- internal/storage/dolt/migrations.go trimmed to CreateIgnoredTables.
- internal/storage/dolt/store.go and cmd/bd/migrate.go updated to
  call through package migrations.
- internal/storage/embeddeddolt/store.go opens a fresh sql.DB after
  the schema-migration tx commits and calls RunCompatMigrations.
- Regression test: synthetic old-schema DB (drop issues.started_at),
  reopen, assert the compat runner re-adds the column.

Refs GH#3412.

Hook-bypass-reason: local pre-commit hook fails on Windows with
CGO_ENABLED=0 because cmd/bd/init.go references sanitizeDBName,
defined in cmd/bd/store_factory.go which is gated //go:build cgo.
Reproduced against pristine origin/main; not introduced by this
change. Upstream fix in flight at GH#3402 / GH#3403.
2026-04-24 13:56:41 -07:00
Jim Wordelman
331bc3aed2
perf(storage): amortize per-ID wisp routing via WispIDSetInTx (#3453)
* perf(storage): amortize per-ID wisp routing via WispIDSetInTx

Implements D2 build per ADR be-nu4 §4.D2 (bead be-nu4.2.1).

Add WispIDSetInTx helper that returns the set of currently-active
wisp IDs, consistent for the tx's lifetime under Dolt MVCC. The hot
batch-routing paths (GetLabelsForIssuesInTx, GetIssuesByIDsInTx)
gain a trailing wispSet parameter; callers that pass nil trigger a
single internal SELECT id FROM wisps rather than the N-per-ID
IsActiveWispInTx loop the helpers previously ran.

IsActiveWispInTx is retained unchanged for single-ID callers
(AddLabelInTx etc.) per the ADR audit note.

Mixed-ID routing is the sacred invariant (ADR §11.5); covered by a
new hard-gate test TestWispIDSetInTx_HardGate that exercises both
refactored helpers with a permanent issue and an active wisp in the
same input slice. Pure-fn unit coverage (partitionByWispSet) runs
without a testcontainer.

Adds BenchmarkGetLabelsForIssues_Mixed and
BenchmarkGetIssuesByIDs_Mixed at 1K / 10K / 50K with 25% wisp share.

Note for reviewer: two additional per-ID hot loops share the same
pattern (GetDependencyRecordsForIssuesInTx,
GetBlockingInfoForIssuesInTx in dependency_queries.go) and would
benefit from the same treatment. Left alone here to respect ADR
scope; flagged for a follow-up bead.

Dolt testcontainer runs flaky on this worktree (Docker 29.4 + ryuk
0.13.0). TESTCONTAINERS_RYUK_DISABLED=true works when infra
cooperates; unit-level coverage (TestPartitionByWispSet) runs
container-free and is green.

Refs: be-nu4.2.1, be-nu4

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: release gate PASS for be-eqw

* fix(storage): scope WispIDSetInTx to input IDs

Reviewer maphew on #3453 flagged that the new default helper builds
the active wisp set with an unscoped SELECT id FROM wisps for
nil-set callers. That turns a small hydration batch (N IDs) into a
full scan of the wisps table (W rows), which is worse than the
pre-PR per-ID IsActiveWispInTx path for small N against large W.

Scope the helper query to the input IDs so cost becomes O(N·log W)
+ 1 round-trip — strictly better than both the old per-ID path and
the current unscoped path.

Signature change: WispIDSetInTx now takes ids []string. Both
internal call sites (GetIssuesByIDsInTx, GetLabelsForIssuesInTx)
pass their input IDs through; all 13 existing nil-set callers
continue to pass nil and get a scoped internal build. Doc updated
on the wispSet parameter to reflect "set scoped to cover these
IDs."

Note: committed with --no-verify (authorized) because release/be-eqw
inherits a pre-existing pre-commit hook blocker — sanitizeDBName is
only defined in the cgo-tagged store_factory.go and the hook runs the
linter under CGO_ENABLED=0. be-4lf (on main) deletes the duplicate
extraction file; once release/be-eqw rebases onto main post-merge the
hook passes naturally. Green-field build/vet/gofmt and all wisp
routing tests were run locally before committing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(storage): be-rgm WispIDSetInTx scoped path + small-N bench

be-3l2 fills the codecov gap on PR #3453 flagged by maphew:
WispIDSetInTx scope-to-input-IDs (14 missing lines on
wisp_routing.go, 0% on dependencies.go/labels.go hydration paths).

New tests in internal/storage/dolt/wisp_set_scoped_test.go:
- TestWispIDSetInTx_Scoped_ReturnsOnlyInputIDs — "lock the fix"
  regression: seeds 5 wisps, passes 1 ID, asserts the other 4
  are NOT in the set. Reverting to `SELECT id FROM wisps` fails this.
- TestWispIDSetInTx_Scoped_AcrossBatchBoundary — 250 wisps (past
  queryBatchSize=200) all pass through the batched IN-clause loop.
- TestWispIDSetInTx_Scoped_UnknownIDsReturnEmpty — unknown IDs
  return empty set; a bait wisp outside the input must not leak.
- TestGetLabelsForIssuesInTx_SmallInputLargeWispTable — end-to-end
  hydration with nil wispSet against 20 noise wisps; exercises the
  dependencies.go/labels.go callers that pass input IDs through.
- TestGetIssuesByIDsInTx_SmallInputLargeWispTable — same shape for
  issue hydration.

New benches in internal/storage/dolt/dolt_benchmark_test.go:
- BenchmarkGetLabelsForIssues_SmallNLargeW_10_5K / _10_10K / _100_5K
- BenchmarkGetIssuesByIDs_SmallNLargeW_10_5K / _10_10K / _100_5K
These measure the maphew regression shape: small input batch
against a large wisps table. Local 10_5K: ~2.6 ms/op, confirming
the scoped N·log(W) path wins cleanly over the pre-be-rgm
full-scan path (existing Mixed10K is ~1.35 s/op for the full
hydration, but that loads 10K not 10).

Note: committed with --no-verify (authorized). release/be-eqw
inherits a pre-existing pre-commit hook blocker — golangci-lint
on the hook is built with go1.25 while go.mod targets 1.26.2,
so the lint step errors before seeing the diff. Green-field
go vet, go build, and all five new tests pass locally against
an isolated Dolt test container.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jim Wordelman <jim@wordelman.name>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:39:42 -07:00
matt wilkie
aed707a103 fix remote sync follow-ups 2026-04-24 11:36:22 -07:00
matt wilkie
813fdfabf8
fix(ci): unblock main — gosec G122 + credentials_test signatures (#3389)
* fix(lint): silence G122 on dolt cache cleanup remove

golangci-lint v2.11.4 (released recently) enabled a new gosec check:

    G122: Filesystem operation in filepath.Walk/WalkDir callback uses
    race-prone path; consider root-scoped APIs (e.g. os.Root) to prevent
    symlink TOCTOU traversal (gosec)

This flags the `os.Remove(path)` inside cleanGitRemoteCacheGarbage.
Main went red on the Lint job with no code change to blame — it broke
on every PR that rebased past the golangci-lint bump.

The warning is not applicable here: the walk root is
`<beadsDir>/<database>/.dolt/git-remote-cache/`, owned by the user
running bd. An attacker who can symlink-swap a tmp_pack_*/tmp_idx_*
file mid-walk already has write access to the Dolt data directory,
which is a much bigger compromise than cache_cleanup.go doing anything
unsafe.

Add a targeted `#nosec G122` with justification rather than refactor
to `os.Root`-scoped APIs, which would be a larger change against a
file that exists to clean up a 102 GB leak (GH#3354) and shouldn't
take on additional complexity for a hypothetical threat. Matches the
project's existing pattern of per-site gosec exceptions with rationale.

Verification:
    golangci-lint run --timeout=5m --build-tags=gms_pure_go ./...
    # 0 issues.

* fix(dolt): update shared-server credential tests for new per-remote signatures

Two tests added by #3223 (f9ec9859, 2026-04-13) were not updated when
#3211 (73d72ec0, 2026-04-20) widened the credential-routing guards to
take per-remote parameters. The resulting signature drift landed on
main and broke the build on ubuntu-latest and macos-latest:

    internal/storage/dolt/credentials_test.go:445:39: not enough
      arguments in call to store.shouldUseCLIForCredentials
    internal/storage/dolt/credentials_test.go:486:6: not enough
      arguments in call to store.shouldUseCLIForCloudAuth

Fix:
- TestCredentialCLIRoutingSharedServerUsesSharedDoltRoot:
  pass store.remote and store.mainRemoteCredentials() to match the
  new shouldUseCLIForCredentials(ctx, remote, creds) signature,
  matching the pattern already used by the sibling
  TestCredentialCLIRoutingExternalServer/NoRemote tests.

- TestCloudAuthCLIRoutingSharedServerUsesSharedDoltRoot:
  pass store.remote to the new shouldUseCLIForCloudAuth(remote)
  signature, and change the dolt remote URL from https:// to az://.
  The old global "any cloud env var triggers CLI routing" heuristic
  was replaced by per-scheme matching (AZURE_STORAGE_* only triggers
  for az:// remotes, etc. — see the positive/negative cases in
  TestCloudAuthCLIRouting). The test sets AZURE_STORAGE_ACCOUNT, so
  the remote must be az:// for the assertion to hold.

Verification: both tests pass locally (`go test -tags gms_pure_go
-run 'SharedServerUsesSharedDoltRoot' ./internal/storage/dolt/`).
2026-04-20 19:02:11 -07:00
Harry Miller
73d72ec0fa
feat: add --remote flag to bd dolt push/pull commands (#3211)
Add PushRemote/PullRemote methods to RemoteStore interface, allowing
push/pull to target a specific named remote instead of the default.

Implementation:
- Refactor Push/ForcePush/Pull to delegate to internal pushToRemote/
  pullFromRemote methods with parameterized remote name
- Parameterize credential routing (isGitProtocolRemote,
  shouldUseCLIForCredentials, shouldUseCLIForCloudAuth) to accept
  remote name instead of using s.remote
- Make shouldUseCLIForCloudAuth per-scheme-aware: Azure env vars only
  trigger CLI routing for az:// remotes, not dolthub:// remotes
- Add credentialsForRemote() helper that only returns creds when the
  target remote matches the default remote (prevents sending wrong
  creds to wrong host)
- Add --remote flag to doltPushCmd and doltPullCmd in CLI
- Implement PushRemote/PullRemote on EmbeddedDoltStore

Backwards compatibility:
- Push(ctx)/Pull(ctx)/ForcePush(ctx) signatures unchanged
- No --remote flag = exact same behavior as today
- PushTo/PullFrom for federation peers unchanged

Closes: bd-4v3

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-20 18:02:33 -07:00
Kevin Glynn
cfdb75e228
Fix git-remote-cache tmp_pack file leak in embedded mode (#3355)
The embedded Dolt backend accumulates orphaned tmp_pack_* files in
.dolt/git-remote-cache/*/repo.git/objects/pack/ when git fetch operations
fail or are interrupted. Dolt's built-in cleanup (GitBlobstore.maybeRunGC,
gated to once per 24h) either never fires in embedded mode or cannot keep
pace with the accumulation rate.

On a real machine with normal beads usage across ~10 workspaces, this leak
consumed 102 GB (412 files) in 7 days, filling the disk to capacity and
causing cascading system failures (swap exhaustion, shell temp file errors,
app crashes).

Add cleanGitRemoteCacheGarbage() to EmbeddedDoltStore that walks the
git-remote-cache pack directories and removes tmp_pack_*/tmp_idx_* files
older than 5 minutes (preserving in-progress fetches). The method is:

- Called from Close() to clean up on store teardown
- Called from withDBConn() deferred cleanup (covers fetch/push paths)
- Rate-limited to once per 10 minutes via a package-level throttle to
  avoid filesystem thrashing on hot paths

Fixes #3354

Co-authored-by: kev <kglynn@pryoninc.com>
2026-04-20 17:49:49 -07:00
Kevin Glynn
dbf793fdb6
fix: auto-sanitize hyphenated dolt_database on embedded-mode upgrade (#3231) (#3232)
Auto-sanitize hyphenated `dolt_database` names on embedded-mode upgrade. Projects initialized before #2142 retain hyphenated names in `metadata.json`; after the 1.0.0 embedded-mode default flip, those names are rejected by the embedded Dolt engine. This change detects legacy names on store open, renames the database directory, persists the fix to `metadata.json`, and adds a doctor check + init guard for future prevention.

Closes #3231. Related: #2142, #2160, #3128, #3129.

Thanks @kevglynn for the thorough fix, migration safety (collision detection, crash-safe ordering), and test coverage.
2026-04-20 13:17:26 -07:00
coffeegoddd☕️✨
65dbdbf566 /internal/storage: fix migration issue 2026-04-14 15:07:54 -07:00
coffeegoddd☕️✨
50f715b124 /{cmd,internal}: move local only state to dolt ignored tables 2026-04-14 14:47:30 -07:00
coffeegoddd☕️✨
31d512325c fix: ensure wisp tables exist after bootstrap clone
After `bd bootstrap` clones from a remote, dolt_ignore'd wisp tables
(wisps, wisp_dependencies, wisp_labels, wisp_events, wisp_comments)
are absent because they are never committed. This caused "table not
found: wisp_dependencies" errors on commands like `bd delete`.

Two fixes:
1. Add EnsureIgnoredTables call to embeddeddolt.New() — matches what
   server mode already does in newServerMode(). This is the primary
   fix: every embedded store open now creates missing wisp tables.
2. Add post-clone store warmup in executeSyncAction() — opens and
   closes a store after finalization to trigger wisp table creation
   immediately, regardless of storage mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:03:17 -07:00
matt wilkie
ebcb901d7a
fix(bootstrap): commit issue_prefix to config table (GH#3216) (#3247)
bd init/restore/import wrote issue_prefix via SetConfig but called
Commit(), which excludes the config table per GH#2455. The prefix row
sat orphaned in the working set forever, causing bd dolt pull to fail
with "cannot merge with uncommitted changes" on every fresh init.

Fix: use CommitWithConfig in the three bootstrap paths, matching the
pattern PR #3052 introduced for remember/forget/config set. Promote
CommitWithConfig to the VersionControl interface and add an alias on
EmbeddedDoltStore (whose Commit already includes config via DOLT_ADD -A).

Reported by @GraemeF.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 13:54:50 -07:00
matt wilkie
3c7dec4fe6
fix(dolt): reject empty database name in embeddeddolt.New (GH#3113) (#3179)
Guard against empty database name early with a clear error message
instead of failing deep in schema migrations with "no database
selected". Includes regression tests for the guard and for
bd create --repo with an uninitialized target directory.

Cherry-picked from thejosephstevens#3113 — thank you Jo Stevens!

Co-authored-by: Jo Stevens <thejosephstevens@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 20:32:26 -07:00
Gabe Lichten
69a13b6e88
Fix GetCustomTypes swallowing DB errors, breaking bd update --type for custom types (#3159)
GetCustomTypes in both DoltStore and EmbeddedDoltStore returns (nil, nil)
when the DB read fails and no YAML fallback is available. This silently
discards the error, causing callers (notably `bd update --type`) to see
an empty custom type list and reject valid custom types like "agent".

The update command (cmd/bd/update.go:146) already handles the error case
correctly — it logs a warning and falls through to the YAML fallback.
But it never fires because the error is swallowed at the storage layer.

Change `return nil, nil` to `return nil, err` so the error propagates
to callers that are prepared to handle it.

Reproducer:
  bd config set types.custom "agent"
  bd create "test" --type=agent    # works (validates in storage layer)
  bd update <id> --type=agent      # fails: "invalid issue type"

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-10 20:30:34 -07:00
Scott Parrish
cace7c52b7
fix(dolt): pass branch to DOLT_PULL to match DOLT_PUSH behavior (#3137)
DOLT_PULL('origin') with only the remote name causes a nil pointer
dereference in the embedded Dolt driver when resolving the branch from
tracking config. DOLT_PUSH already passes both remote and branch.

This aligns Pull() with Push() by passing the branch parameter:
  DOLT_PULL('origin')       → DOLT_PULL('origin', 'main')

Tested with GCS, file, and git+ssh remotes in embedded mode.

Fixes #2118 (related)
2026-04-08 12:50:48 -07:00
coffeegoddd☕️✨
40e5eb11a9 /internal/storage/schema: use shared schema package 2026-04-06 15:03:20 -07:00
coffeegoddd☕️✨
b4284a29be /internal/storage: shared ensure ignored 2026-04-06 14:21:23 -07:00
matt wilkie
98d8e72ea9
fix: queue concurrent embedded Dolt callers instead of failing (#3005)
* fix: queue concurrent embedded Dolt callers instead of failing

PR #2571 added an exclusive flock to prevent concurrent embedded Dolt
panics, but used a non-blocking TryLock that fails immediately when the
lock is held. This makes embedded mode effectively single-process: if
multiple concurrent `bd` commands run, all but one fail instantly.

Replace TryLock with WaitLock in the New() path. WaitLock uses
exponential backoff (50ms initial, 2s max) with non-blocking lock
attempts so the wait is interruptible via context cancellation. TryLock
is preserved for bd init which legitimately needs fail-fast behavior.

Refs: GH#2571

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: correct 'cancelled' → 'canceled' spelling for golangci-lint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 21:05:18 -07:00
coffeegoddd☕️✨
f74f9f2b75 /internal/storage/{dolt,embeddeddolt}: update embedded schema to match dolt server 2026-04-03 11:32:03 -07:00
coffeegoddd☕️✨
e46318e986 fix: address PR review — replace hardcoded version 23 with LatestVersion()
Export LatestVersion() and use it in tests so they don't break when new
migrations are added. String-based error detection kept as-is since the
embedded dolt driver (dolthub/driver) doesn't expose mysql.MySQLError,
consistent with the existing pattern in issueops/batching.go.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:10:45 -07:00
coffeegoddd☕️✨
231c88d6e3 refactor: deduplicate migration logic into runMigrations helper
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:48:05 -07:00
coffeegoddd☕️✨
457df7a8ee /internal/storage/embeddeddolt: make backfill schema robust 2026-04-03 10:45:13 -07:00
beads/crew/emma
543e1943b6 fix: prevent concurrent embedded Dolt panic with exclusive flock (#2571)
When two bd commands ran simultaneously against the same embedded Dolt
database, the second process could panic with a nil-pointer dereference
in DoltDB.SetCrashOnFatalError during engine initialization. The race
occurs because dolthub/driver.NewConnector is not safe for concurrent
initialization on the same data directory.

The fix acquires an exclusive flock in embeddeddolt.New() for the entire
store lifetime. The second process now receives a clear error message
instead of panicking. The lock is released in Close(). A WithLock option
allows bd init (which acquires the lock earlier) to pass its pre-held
lock and avoid double-locking on macOS where flock is per-fd.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Executed-By: beads/crew/emma
2026-04-02 21:27:37 -07:00
beads/crew/emma
2a71d7fc41 fix: preserve spaces in embedded Dolt paths
The buildDSN function used url.URL.String() to construct the DSN, which
percent-encodes the path component (spaces become %20). The embedded Dolt
driver ParseDataSource uses the path as a literal filesystem string, so
the encoded path fails os.Stat checks.

Build the DSN string directly to avoid URL encoding of filesystem paths.

Fixes #2920

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Executed-By: beads/crew/emma
2026-04-02 21:22:45 -07:00
Robin Carswell
a097723f28
fix: suppress auto-backup address conflict warning (#2927)
* fix: suppress auto-backup address conflict warning when default backup is configured

When a user runs `bd backup init <path>` it registers a Dolt backup named
"default". The auto-backup system also calls BackupDatabase() which registers
its own remote named "backup_export" pointing to the same .beads/backup/
directory. Dolt returns "Error 1105: address conflict with a remote: 'default'"
because two remotes cannot share a URL, causing a noisy warning on every write:

  Warning: auto-backup failed: register backup remote: add backup backup_export: \
  Error 1105: address conflict with a remote: 'default' -> file://.../backup

Fix: when BackupAdd fails with an address conflict, extract the name of the
already-registered remote from the error message and sync using that name
instead. The backup still runs — it just reuses the existing remote.

Adds ExtractAddressConflictName() to versioncontrolops for parsing the
conflicting name out of the Dolt error. Applies the fix to both the
EmbeddedDoltStore and DoltStore implementations of BackupDatabase().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add table-driven tests for ExtractAddressConflictName

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 20:28:47 -07:00
Bella-Giraffety
4e4d958baf
fix: prefer bootstrap for missing database recovery (#2940)
* fix: prefer bootstrap for missing database recovery

Executed-By: beads/crew/emma

* fix: align doctor recovery guidance with bootstrap

Executed-By: beads/crew/emma

* fix: let bootstrap recover missing server databases

Executed-By: beads/crew/emma

* fix: bootstrap missing server databases safely

Executed-By: beads/crew/emma

* fix: fail closed on bootstrap server probe errors

Executed-By: beads/crew/emma

* fix: honor runtime server config in bootstrap

Executed-By: beads/crew/emma

* docs: soften bootstrap-first recovery wording

Executed-By: beads/crew/emma

* fix: report resolved bootstrap server port

Executed-By: beads/crew/emma

* fix: repair test failures from build tag mismatch and mode detection

Three issues causing CI failures on macOS and Ubuntu:

1. detectBootstrapAction used isEmbeddedMode() (global state) to choose
   the database path, but tests set cfg.DoltMode without updating the
   global serverMode flag. Switch to cfg.GetDoltMode() so the detection
   logic respects the config object passed by the caller.

2. TestDetectBootstrapAction_NoneWhenDatabaseExists created a "dolt"
   directory but default (embedded) config looks for "embeddeddolt".
   Fix the test to create the correct directory.

3. embeddeddolt/slots.go and new_methods_test.go used //go:build
   embeddeddolt tag while all other files in the package use cgo,
   causing interface satisfaction failures. Also fix undefined
   isEmbeddedDolt reference (should be isEmbeddedMode()).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Executed-By: beads/crew/emma

---------

Co-authored-by: mayor <thalia.geraghty@fiserv.com>
Co-authored-by: beads/crew/emma <steve.yegge@gmail.com>
2026-04-02 19:52:58 -07:00
Jim Wordelman
677f7bcd3d
feat: add SlotSet/SlotGet/SlotClear to Storage interface (#2870)
* feat: add SlotSet/SlotGet/SlotClear to Storage interface

Metadata slot operations for key-value pairs stored in issue metadata
JSON. Used by gt for delegation tracking, hook state, and other
per-issue data.

- SlotSet: set or overwrite a metadata key
- SlotGet: read a metadata key (error if not found)
- SlotClear: remove a metadata key (no-op if absent)

Implemented in DoltStore, EmbeddedDoltStore, and InstrumentedStorage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add SlotSet/SlotGet/SlotClear stubs to configStore test mock

The configStore mock in jira/tracker_test.go was missing the three new
Storage interface methods, causing go vet to fail.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mayor <jim@wordelman.name>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: beads/crew/emma <steve.yegge@gmail.com>
2026-04-02 19:25:18 -07:00
matt wilkie
59b49c45f0 fix: add missing //go:build cgo tags to embedded test files and cmd
Several test files and one main.go that depend on CGO-only types
(EmbeddedDoltStore methods like Close, CreateIssue, GetIssue) were
missing //go:build cgo build tags, causing compilation failures when
building with CGO_ENABLED=0.

Files fixed:
- 67 cmd/bd/*_embedded_test.go files
- cmd/bd/metadata_filter_test.go (uses newTestStore from cgo-tagged helper)
- cmd/bd/doctor/federation_test.go (uses doctorTestServerPort/testSharedDB)
- cmd/bd/doctor/fix/migrate_import_test.go (uses fixTestServerPort)
- 6 internal/storage/embeddeddolt/*_test.go files
- internal/storage/embeddeddolt/cmd/main.go

Amp-Thread-ID: https://ampcode.com/threads/T-019d4938-40d4-73ae-89d1-1891a19a9a15
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 07:08:41 -07:00
Harry
047e506859 Enforce .beads/ directory permissions at runtime
- Add config.BeadsDirPerm (0700) and config.BeadsFilePerm (0600) constants
- Add config.CheckBeadsDirPermissions() for startup warning if permissions
  are too permissive (non-fatal, checks group/world bits)
- Add config.EnsureBeadsDir() helper for consistent directory creation
- Tighten all MkdirAll calls for .beads/ paths from 0750/0755 to 0700:
  init.go, doltserver.go, credentials.go, audit.go, recipes.go,
  embeddeddolt store.go and flock.go
- Add startup check in loadEnvironment() (cmd/bd/main.go)
- Windows build tag with no-op implementation (ACL-based permissions)
- 6 tests: constants, dir creation perms, warning on permissive, no
  warning on secure, no error on nonexistent

Fixes: bd-kl6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-30 09:15:49 -07:00
beads/crew/emma
0cac11fbb6 fix(release): add CGO build constraints for embeddeddolt, bump to v0.63.2
Gate embeddeddolt package on //go:build cgo with stub files for !cgo.
Android and freebsd targets use CGO_ENABLED=0 since zig lacks sysroot
headers for these platforms. Server mode still works on all targets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 16:57:50 -07:00
coffeegoddd☕️✨
9134355618 /{README.md,cmd,docs,internal,website}: doc updates 2026-03-28 13:47:20 -07:00
coffeegoddd☕️✨
4488df7077 /{cmd,internal}: get backups working 2026-03-28 13:47:20 -07:00
coffeegoddd☕️✨
9b30a7f911 /{cmd,internal}: fix restore 2026-03-28 13:47:20 -07:00
coffeegoddd☕️✨
fb0bdbc60a /{cmd,internal}: update backup command 2026-03-28 13:47:20 -07:00
coffeegoddd☕️✨
a0c277e871 /{cmd,internal}: fixes 2026-03-28 13:47:20 -07:00
coffeegoddd☕️✨
a35edfd28e /{internal,cmd,.goreleaser,.github}: embedded mode by default 2026-03-28 13:47:20 -07:00
matt wilkie
fe1c116950
Fix CI: gofmt, orchestrator root check, macOS symlink test, remove unimplemented slot tests, reopen exit code (#2863)
- Fix gofmt formatting in migrations/helpers.go (string concat spacing)
- Fix isOrchestratorRoot to check mayor/town.json, not just mayor/ dir
- Fix macOS symlink test by resolving /var -> /private/var with EvalSymlinks
- Remove TestSlotSetGetClear for unimplemented SlotSet/SlotGet/SlotClear methods
- Fix bd reopen to exit non-zero when resolution or reopening fails

Amp-Thread-ID: https://ampcode.com/threads/T-019d31c8-6463-701a-8a1f-16eab285d0c3

Co-authored-by: Amp <amp@ampcode.com>
2026-03-27 19:50:21 -07:00
emma
169f357cbb feat: add MergeSlot methods to Storage interface (fix-merge PR #2787)
Co-Authored-By: Jim Wordelman <jim@wordelman.name>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 19:34:21 -07:00
emma
46c8f13c57 feat: add ListWisps method to Storage interface (fix-merge PR #2783)
Co-Authored-By: Jim Wordelman <jim@wordelman.name>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 19:33:28 -07:00
emma
dc96e34812 feat: add ReopenIssue and UpdateIssueType to Storage interface (fix-merge PR #2782)
Co-Authored-By: Jim Wordelman <jim@wordelman.name>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 19:33:14 -07:00