* fix(dolt): pre-pull commit must include config; auto-resolve convergent kv.memory.* config conflicts
bd remember stores persistent memories as kv.memory.* rows in the synced
config table. Commit() deliberately excludes config (GH#2455, to avoid
sweeping a concurrent writer's stale issue_prefix into an unrelated commit),
so the GH#2474 pre-pull auto-commit at pullFromRemote (store.go) and PullFrom
(federation.go) leaves config permanently dirty once any memory exists. The
"clean the working set before merging" step then leaves config dirty and
DOLT_MERGE refuses to start ("cannot merge with uncommitted changes"). Two
correct PRs colliding over config. Diagnosed from a live server-mode repro
(Wyvern wy-71t); see PROPOSAL-pull-config-wedge.md.
Fix (server mode): add commitBeforePull, which stages config explicitly via
DOLT_ADD rather than relying on CommitWithConfig's DOLT_COMMIT('-Am'). The
proposal author observed -Am not staging config reliably under the server-mode
stored-procedure path, and the new embedded test TestEmbeddedConfigCommitStagesConfig
confirms -Am DOES stage config in the embedded engine — so the unreliability is
server-mode-specific and explicit staging is the robust choice. Commit() and
commitBeforePull now share commitWorkingSet(includeConfig bool).
Hardening (both modes): making config a committed/synced table means two clones
editing the same memory key collide on a config merge conflict. Extend
TryAutoResolveMergeConflicts to auto-resolve config conflicts with --theirs ONLY
when every conflicted row is a kv.memory.* key (machine-convergent, same policy
as metadata); any other config key (issue_prefix above all) is left for the
operator. Without this the primary fix would trade a silent pull wedge for an
operator-visible conflict wedge on the exact bd remember workload.
Embedded pre-pull (CommitPending -> -Am) already stages config and is unchanged;
it gains the resolver hardening.
Tests: server-mode pull_config_wedge_test.go (commit-includes-config + resolver
positive/negative, run under Docker in CI) and embedded pull_config_wedge_test.go
(in-process: -Am-stages-config probe + resolver positive/negative, all green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: add pull-config-wedge diagnosis/proposal (PR #4412)
The diagnosis-and-patch write-up behind this PR: how GH#2455 (Commit excludes
config) and GH#2474 (pre-pull auto-commit) collide over the synced config table,
leaving kv.memory.* rows permanently uncommitted so DOLT_MERGE refuses to start.
Includes the live-DB validation (Wyvern wy-71t) and the rejected alternatives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(dolt): adopt-pr maintainer fixes for pull/sync config wedge
Maintainer review fixups for PR #4412, squashed into one commit on top of
the contributor's branch.
Pre-merge config wedge (sync): bd federation sync committed with the
config-excluding Commit before its merge while this PR switched the pull
paths to commitBeforePull, so any DB that ran `bd remember` carried dirty
kv.memory.* config rows and hit the "cannot merge with uncommitted changes"
wedge. Sync now uses commitBeforePull too, with a regression test.
Pre-pull auto-commit safety (GH#2455): commitBeforePull no longer stages the
whole config table. It auto-commits only convergent memory rows
(kv.memory.*); if any other config key is dirty (issue_prefix above all) it
refuses with operator guidance instead of sweeping a stale value into the
merge basis, restoring the GH#2455 protection the broad pre-pull staging had
reopened.
Sync --strategy config commit: a config-only conflict resolution committed
through the config-excluding Commit, so it could report success without
creating the merge-resolution commit and re-wedge the next sync. Sync now
commits resolutions through a config-inclusive path (commitMergeResolution).
Embedded sync parity: EmbeddedDoltStore.Sync fetched and merged without the
pre-merge CommitPending guard already used by embedded Pull/PullRemote/PullFrom
and the new server Sync, so the same dirty kv.memory.* working set could still
wedge `bd federation sync` in embedded mode. Sync now commits pending changes
before fetch/merge (embedded Commit is DOLT_COMMIT('-Am'), which stages
config), with an embedded regression test that drives Sync with a dirty
kv.memory.* row.
KV/memory namespace collision: `bd kv set memory.*` stored to kv.memory.*,
indistinguishable from a `bd remember` memory the merge resolver auto-resolves
with --theirs. validateKVKey now reserves the memory. prefix so a generic KV
value can't be silently overridden by the remote on pull.
Shared prefix + tests: centralize the kv.memory. prefix in
internal/storage/kvkeys (used by cmd/bd and the merge resolver) with a
contract test; route the remaining cmd/bd kv. prefix literals (the doctor KV
check, the validateKVKey nested-prefix guard, and the memory listing) through
the kvkeys constants; and strengthen the server-mode non-memory config
conflict test to prove a conflict materialized before the resolver declines it.
Maintainer review fixup for PR #4412.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <ci@beads.test>
353 lines
10 KiB
Go
353 lines
10 KiB
Go
//go:build cgo
|
|
|
|
package embeddeddolt
|
|
|
|
import (
|
|
"context"
|
|
"crypto/aes"
|
|
"crypto/cipher"
|
|
"crypto/rand"
|
|
"database/sql"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/storage/issueops"
|
|
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
|
|
)
|
|
|
|
// credentialKeyFile is the filename for the random encryption key.
|
|
const credentialKeyFile = ".beads-credential-key" //nolint:gosec // G101: filename, not a credential
|
|
|
|
// ensureCredentialKey lazily initializes the credential encryption key.
|
|
func (s *EmbeddedDoltStore) ensureCredentialKey() error {
|
|
if s.credentialKey != nil {
|
|
return nil
|
|
}
|
|
if s.beadsDir == "" {
|
|
return fmt.Errorf("beads directory not set; credential encryption unavailable")
|
|
}
|
|
|
|
keyPath := filepath.Join(s.beadsDir, credentialKeyFile)
|
|
|
|
// Try to load existing key.
|
|
key, err := os.ReadFile(keyPath) //nolint:gosec // G304: keyPath derived from trusted beadsDir
|
|
if err == nil && len(key) == 32 {
|
|
s.credentialKey = key
|
|
return nil
|
|
}
|
|
|
|
// Generate new random 32-byte key (AES-256).
|
|
key = make([]byte, 32)
|
|
if _, err := io.ReadFull(rand.Reader, key); err != nil {
|
|
return fmt.Errorf("generate credential key: %w", err)
|
|
}
|
|
if err := os.WriteFile(keyPath, key, 0600); err != nil {
|
|
return fmt.Errorf("write credential key: %w", err)
|
|
}
|
|
|
|
s.credentialKey = key
|
|
return nil
|
|
}
|
|
|
|
func (s *EmbeddedDoltStore) encryptPassword(password string) ([]byte, error) {
|
|
if password == "" {
|
|
return nil, nil
|
|
}
|
|
if err := s.ensureCredentialKey(); err != nil {
|
|
return nil, err
|
|
}
|
|
block, err := aes.NewCipher(s.credentialKey)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
gcm, err := cipher.NewGCM(block)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
nonce := make([]byte, gcm.NonceSize())
|
|
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
|
|
return nil, err
|
|
}
|
|
return gcm.Seal(nonce, nonce, []byte(password), nil), nil
|
|
}
|
|
|
|
func (s *EmbeddedDoltStore) decryptPassword(encrypted []byte) (string, error) {
|
|
if len(encrypted) == 0 {
|
|
return "", nil
|
|
}
|
|
if err := s.ensureCredentialKey(); err != nil {
|
|
return "", err
|
|
}
|
|
block, err := aes.NewCipher(s.credentialKey)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
gcm, err := cipher.NewGCM(block)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
nonceSize := gcm.NonceSize()
|
|
if len(encrypted) < nonceSize {
|
|
return "", fmt.Errorf("ciphertext too short")
|
|
}
|
|
nonce, ciphertext := encrypted[:nonceSize], encrypted[nonceSize:]
|
|
plaintext, err := gcm.Open(nil, nonce, ciphertext, nil)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
return string(plaintext), nil
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// FederationStore implementation
|
|
// ---------------------------------------------------------------------------
|
|
|
|
func (s *EmbeddedDoltStore) AddFederationPeer(ctx context.Context, peer *storage.FederationPeer) error {
|
|
encryptedPwd, err := s.encryptPassword(peer.Password)
|
|
if err != nil {
|
|
return fmt.Errorf("encrypt password: %w", err)
|
|
}
|
|
|
|
if err := s.withConn(ctx, true, func(tx *sql.Tx) error {
|
|
if err := issueops.AddFederationPeerInTx(ctx, tx, peer, encryptedPwd); err != nil {
|
|
return err
|
|
}
|
|
// Also add the Dolt remote.
|
|
return issueops.AddRemoteIfNotExists(ctx, tx, peer.Name, peer.RemoteURL)
|
|
}); err != nil {
|
|
return err
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (s *EmbeddedDoltStore) GetFederationPeer(ctx context.Context, name string) (*storage.FederationPeer, error) {
|
|
var row *issueops.FederationPeerRow
|
|
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
|
|
var err error
|
|
row, err = issueops.GetFederationPeerInTx(ctx, tx, name)
|
|
return err
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
if len(row.EncryptedPwd) > 0 {
|
|
row.Peer.Password, err = s.decryptPassword(row.EncryptedPwd)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("decrypt password: %w", err)
|
|
}
|
|
}
|
|
return &row.Peer, nil
|
|
}
|
|
|
|
func (s *EmbeddedDoltStore) ListFederationPeers(ctx context.Context) ([]*storage.FederationPeer, error) {
|
|
var rows []*issueops.FederationPeerRow
|
|
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
|
|
var err error
|
|
rows, err = issueops.ListFederationPeersInTx(ctx, tx)
|
|
return err
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
peers := make([]*storage.FederationPeer, 0, len(rows))
|
|
for _, row := range rows {
|
|
if len(row.EncryptedPwd) > 0 {
|
|
pwd, err := s.decryptPassword(row.EncryptedPwd)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("decrypt password for peer %s: %w", row.Peer.Name, err)
|
|
}
|
|
row.Peer.Password = pwd
|
|
}
|
|
peers = append(peers, &row.Peer)
|
|
}
|
|
return peers, nil
|
|
}
|
|
|
|
func (s *EmbeddedDoltStore) RemoveFederationPeer(ctx context.Context, name string) error {
|
|
if err := s.withConn(ctx, true, func(tx *sql.Tx) error {
|
|
return issueops.RemoveFederationPeerInTx(ctx, tx, name)
|
|
}); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Also remove the Dolt remote (best-effort).
|
|
if rmErr := s.RemoveRemote(ctx, name); rmErr != nil {
|
|
if !strings.Contains(rmErr.Error(), "not found") {
|
|
// Silently ignore "not found" — the remote may not exist.
|
|
_ = rmErr
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// SyncStore implementation
|
|
// ---------------------------------------------------------------------------
|
|
|
|
// Sync performs a full bidirectional sync with a peer:
|
|
// 1. Fetch from peer
|
|
// 2. Merge peer's changes (handling conflicts per strategy)
|
|
// 3. Push local changes to peer
|
|
func (s *EmbeddedDoltStore) Sync(ctx context.Context, peer string, strategy string) (*storage.SyncResult, error) {
|
|
result := &storage.SyncResult{
|
|
Peer: peer,
|
|
StartTime: time.Now(),
|
|
}
|
|
|
|
// GH#2474 / bd-578h9.2: commit pending changes before the merge, matching
|
|
// embedded Pull/PullRemote/PullFrom and server-mode Sync. Embedded Commit is
|
|
// DOLT_COMMIT('-Am'), so it stages config — where kv.memory.* memories live —
|
|
// and a leftover dirty working set (e.g. a `bd remember` write) would
|
|
// otherwise make DOLT_MERGE refuse to start ("cannot merge with uncommitted
|
|
// changes"). CommitPending is a no-op when the working set is already clean.
|
|
if _, err := s.CommitPending(ctx, "beads"); err != nil {
|
|
result.Error = fmt.Errorf("commit pending before sync: %w", err)
|
|
return result, result.Error
|
|
}
|
|
|
|
// Step 1: Fetch
|
|
if err := s.Fetch(ctx, peer); err != nil {
|
|
result.Error = fmt.Errorf("fetch failed: %w", err)
|
|
return result, result.Error
|
|
}
|
|
result.Fetched = true
|
|
|
|
// Step 2: Get commit before merge for change detection
|
|
beforeCommit, _ := s.GetCurrentCommit(ctx)
|
|
|
|
// Step 3: Merge peer's branch
|
|
remoteBranch := fmt.Sprintf("%s/%s", peer, s.branch)
|
|
conflicts, err := s.Merge(ctx, remoteBranch)
|
|
if err != nil {
|
|
result.Error = fmt.Errorf("merge failed: %w", err)
|
|
return result, result.Error
|
|
}
|
|
|
|
// Step 4: Handle conflicts
|
|
if len(conflicts) > 0 {
|
|
result.Conflicts = conflicts
|
|
|
|
if strategy == "" {
|
|
result.Error = fmt.Errorf("merge conflicts require resolution (use --strategy ours|theirs)")
|
|
return result, result.Error
|
|
}
|
|
|
|
for _, c := range conflicts {
|
|
if err := s.ResolveConflicts(ctx, c.Field, strategy); err != nil {
|
|
result.Error = fmt.Errorf("conflict resolution failed for %s: %w", c.Field, err)
|
|
return result, result.Error
|
|
}
|
|
}
|
|
result.ConflictsResolved = true
|
|
|
|
if err := s.Commit(ctx, fmt.Sprintf("Resolve conflicts from %s using %s strategy", peer, strategy)); err != nil {
|
|
result.Error = fmt.Errorf("commit conflict resolution: %w", err)
|
|
return result, result.Error
|
|
}
|
|
|
|
// bd-578h9.11: the conflicted merge skipped the automatic is_blocked
|
|
// recompute (unresolved rows would have fed it garbage); now that the
|
|
// resolution is committed, cover the whole merge+resolution window.
|
|
if err := s.RecomputeBlockedAfterMerge(ctx, beforeCommit); err != nil {
|
|
result.Error = fmt.Errorf("conflicts resolved but is_blocked recompute failed: %w", err)
|
|
return result, result.Error
|
|
}
|
|
}
|
|
result.Merged = true
|
|
|
|
afterCommit, _ := s.GetCurrentCommit(ctx)
|
|
if beforeCommit != afterCommit {
|
|
result.PulledCommits = 1
|
|
}
|
|
|
|
// Step 5: Push
|
|
if err := s.PushTo(ctx, peer); err != nil {
|
|
result.PushError = err
|
|
} else {
|
|
result.Pushed = true
|
|
}
|
|
|
|
// Record last sync time in metadata.
|
|
_ = s.setLastSyncTime(ctx, peer)
|
|
|
|
result.EndTime = time.Now()
|
|
return result, nil
|
|
}
|
|
|
|
// SyncStatus returns the synchronization status with a peer.
|
|
func (s *EmbeddedDoltStore) SyncStatus(ctx context.Context, peer string) (*storage.SyncStatus, error) {
|
|
status := &storage.SyncStatus{
|
|
Peer: peer,
|
|
}
|
|
|
|
// Get ahead/behind counts by comparing refs.
|
|
// Dolt's AS OF requires a literal ref, not a parameterized expression.
|
|
remoteRef := peer + "/" + s.branch
|
|
if err := issueops.ValidateRef(remoteRef); err != nil {
|
|
status.LocalAhead = -1
|
|
status.LocalBehind = -1
|
|
} else if err := s.withDBConn(ctx, func(db versioncontrolops.DBConn) error {
|
|
query := fmt.Sprintf(`
|
|
SELECT
|
|
(SELECT COUNT(*) FROM dolt_log WHERE commit_hash NOT IN
|
|
(SELECT commit_hash FROM dolt_log AS OF '%s')) as ahead,
|
|
(SELECT COUNT(*) FROM dolt_log AS OF '%s' WHERE commit_hash NOT IN
|
|
(SELECT commit_hash FROM dolt_log)) as behind
|
|
`, remoteRef, remoteRef)
|
|
if err := db.QueryRowContext(ctx, query).
|
|
Scan(&status.LocalAhead, &status.LocalBehind); err != nil {
|
|
// Remote branch may not exist locally yet.
|
|
status.LocalAhead = -1
|
|
status.LocalBehind = -1
|
|
}
|
|
return nil
|
|
}); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// Check for conflicts.
|
|
conflicts, err := s.GetConflicts(ctx)
|
|
if err == nil && len(conflicts) > 0 {
|
|
status.HasConflicts = true
|
|
}
|
|
|
|
// Get last sync time.
|
|
status.LastSync = s.getLastSyncTime(ctx, peer)
|
|
|
|
return status, nil
|
|
}
|
|
|
|
// setLastSyncTime records the last sync time for a peer in metadata.
|
|
func (s *EmbeddedDoltStore) setLastSyncTime(ctx context.Context, peer string) error {
|
|
key := "last_sync_" + peer
|
|
value := time.Now().Format(time.RFC3339)
|
|
return s.withConn(ctx, true, func(tx *sql.Tx) error {
|
|
_, err := tx.ExecContext(ctx,
|
|
"REPLACE INTO metadata (`key`, value) VALUES (?, ?)", key, value)
|
|
return err
|
|
})
|
|
}
|
|
|
|
// getLastSyncTime retrieves the last sync time for a peer from metadata.
|
|
func (s *EmbeddedDoltStore) getLastSyncTime(ctx context.Context, peer string) time.Time {
|
|
key := "last_sync_" + peer
|
|
var value string
|
|
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
|
|
return tx.QueryRowContext(ctx, "SELECT value FROM metadata WHERE `key` = ?", key).Scan(&value)
|
|
})
|
|
if err != nil {
|
|
return time.Time{}
|
|
}
|
|
t, err := time.Parse(time.RFC3339, value)
|
|
if err != nil {
|
|
return time.Time{}
|
|
}
|
|
return t
|
|
}
|