Compare commits

...
Sign in to create a new pull request.

2 commits

Author SHA1 Message Date
Julian Knutsen
b429a36d68 test(dolt): guard that SnapshotCreateInputs undoes every write of the real create path
Runs the real issueops create path against a real Dolt server — the
store-internal path and the transaction path cmd/bd replays — in hash and
counter mode, for durable issues and wisps, with labels, metadata,
optional pointer fields, inline dependencies (one dangling) and inline
comments (one with an incoming id). It rolls the SQL back, restores, and
deep-compares every caller-owned field and pointee with a deep copy taken
before the create. It fails the day the create path starts writing into
caller-owned state that SnapshotCreateInputs does not restore.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 06:05:26 +00:00
Julian Knutsen
30c3fbc137 fix(dolt): replayed creates restart from the caller's request
withRetryTx replays a create whose transaction lost Dolt's commit-time
merge to a concurrent writer, but the create writes what it minted back
into the caller's issue, and the replay started from that instead of from
the request. This is the mechanism #7289 describes. The rolled-back
attempt's ID then looked caller-supplied, so the replay took the
explicit-ID upsert path. In counter mode, where the competing writer
minted the same number, that overwrote the competitor's issue and handed
one ID to both callers.

The create path writes back through three routes, and a replay has to
undo all of them:

- the Issue struct: the minted ID, the defaulted timestamps and the
  content hash;
- each inline dependency that named no source gets the minted ID as its
  source (PersistDependenciesWithOptionsResult);
- each inline comment without an ID gets one derived from the minted ID
  (PersistComments).

issueops.SnapshotCreateInputs records those fields before the first
attempt and returns a restore func. DoltStore.createIssue and both
createIssuesWithFullOptions paths call it at the top of every withRetryTx
attempt. A struct-only restore is not enough: the replay then mints a new
ID while the edge and the comment still point at the old one, and in hash
mode the inline edge fails its foreign key (Error 1452). An audit of the
create path found no other write into caller-owned state.

Tests:
- TestSnapshotCreateInputsRestoresWhatTheCreatePathWritesBack (unit).
  It fails against a struct-only restore ("inline edge source after
  restore = test-1").
- TestConcurrentCounterModeCreatesNeverShareAnID (real Dolt: 4 stores x
  3 rounds of single and batch creates, counter mode). On main it fails:
  "id test-1 handed to both "single 3-0" and "single 2-0"", and
  "issue test-12 holds "batch 2-2-a" ...: a replayed create overwrote it".
- TestReplayedBatchCreateKeepsInlineEdgeOnItsOwnIssue holds a batch
  create open in its transaction until a competitor commits. It runs in
  counter and hash mode. Counter mode fails on main ("loser and competitor
  were both handed test-1"). With a struct-only restore both modes fail;
  hash mode fails with "Error 1452 ... Foreign key violation on fk:
  fk_dep_issue".
- TestReplayedCreateDerivesInlineCommentIDFromItsOwnIssue holds a single
  create's COMMIT at a TCP proxy until a competitor commits. It also runs
  in counter and hash mode. Counter mode fails on main as above. With a
  struct-only restore both modes fail ("comment on test-2 has id ..., want
  ... (derived from its own issue)").

All pass with this change, against a dolt sql-server 2.2.0 test
container. The replay tests use only test-side seams: an
OnSkippedDependency callback and a MySQL-wire proxy in the test package.

Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 00:04:24 +00:00
9 changed files with 863 additions and 0 deletions

View file

@ -181,6 +181,8 @@ go_test(
"create_batch_equivalence_test.go",
"create_guard_test.go",
"create_issue_commit_test.go",
"create_replay_snapshot_complete_test.go",
"create_replay_test.go",
"create_rollback_test.go",
"credentials_test.go",
"cross_project_test.go",
@ -289,6 +291,7 @@ go_test(
"molecule_contract_test.go",
"multistore_multiprocess_test.go",
"open_test.go",
"peer_store_test.go",
"pool_timeouts_test.go",
"port_provenance_test.go",
"portsource_propagation_test.go",
@ -342,6 +345,7 @@ go_test(
"smart_gate_data_behind_test.go",
"smart_remote_migrate_gate_test.go",
"socket_fallback_test.go",
"sql_wire_proxy_test.go",
"stats_reporter_contract_test.go",
"store_branch_pin_test.go",
"store_unit_test.go",

View file

@ -0,0 +1,173 @@
package dolt
// SnapshotCreateInputs must undo EVERY write the real create path makes into
// caller-owned state — struct fields AND pointees. This runs the real issueops
// create path against a real Dolt server (the store-internal path and the
// transaction path cmd/bd replays), rolls the SQL back, restores, and
// deep-compares with a deep copy taken before the create. It fails the day
// the create path starts writing somewhere the snapshot does not restore.
import (
"encoding/json"
"errors"
"reflect"
"testing"
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/types"
)
func deepCopyValue(v reflect.Value) reflect.Value {
switch v.Kind() {
case reflect.Ptr:
if v.IsNil() {
return reflect.Zero(v.Type())
}
n := reflect.New(v.Elem().Type())
n.Elem().Set(deepCopyValue(v.Elem()))
return n
case reflect.Struct:
n := reflect.New(v.Type()).Elem()
n.Set(v) // shallow, including unexported fields (time.Time etc.)
for i := 0; i < v.NumField(); i++ {
if !n.Field(i).CanSet() {
continue
}
n.Field(i).Set(deepCopyValue(v.Field(i)))
}
return n
case reflect.Slice:
if v.IsNil() {
return reflect.Zero(v.Type())
}
n := reflect.MakeSlice(v.Type(), v.Len(), v.Len())
for i := 0; i < v.Len(); i++ {
n.Index(i).Set(deepCopyValue(v.Index(i)))
}
return n
case reflect.Map:
if v.IsNil() {
return reflect.Zero(v.Type())
}
n := reflect.MakeMapWithSize(v.Type(), v.Len())
for _, k := range v.MapKeys() {
n.SetMapIndex(k, deepCopyValue(v.MapIndex(k)))
}
return n
default:
return v
}
}
func createReplayFixtureIssues(t *testing.T) []*types.Issue {
t.Helper()
est := 30
due := time.Now().Add(48 * time.Hour).UTC().Truncate(time.Second)
ext := "gh-1"
return []*types.Issue{
{
Title: "auto-id durable", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
Labels: []string{"zeta", "alpha", "alpha"},
Metadata: json.RawMessage(`{"b":1,"a":2}`),
EstimatedMinutes: &est,
DueAt: &due,
ExternalRef: &ext,
Dependencies: []*types.Dependency{
{DependsOnID: "test-anchor", Type: types.DepBlocks},
{DependsOnID: "test-nosuch", Type: types.DepRelated},
},
Comments: []*types.Comment{
{Author: "a", Text: "fresh"},
{ID: "c-imported", Author: "b", Text: "imported", CreatedAt: time.Now().Add(-time.Hour).UTC().Truncate(time.Second)},
},
},
{
// closed without closed_at: PrepareIssueForInsert allocates one
Title: "closed auto", Status: types.StatusClosed, Priority: 1, IssueType: types.TypeBug,
},
{
ID: "test-anchor.7", Title: "hierarchical explicit", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
Dependencies: []*types.Dependency{{DependsOnID: "test-anchor", Type: types.DepParentChild}},
},
}
}
func assertCreateInputsRestored(t *testing.T, label string, before, after []*types.Issue) {
t.Helper()
for i := range before {
if !reflect.DeepEqual(before[i], after[i]) {
b, _ := json.Marshal(before[i])
a, _ := json.Marshal(after[i])
t.Errorf("%s: issue %d not restored\nbefore: %s\nafter: %s", label, i, b, a)
}
}
}
func TestSnapshotCreateInputsUndoesTheRealCreatePath(t *testing.T) {
base, cleanup := setupConcurrentTestStore(t)
defer cleanup()
ctx, cancel := concurrentTestContext(t)
defer cancel()
if err := base.CreateIssue(ctx, &types.Issue{ID: "test-anchor", Title: "anchor", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeEpic}, "seed"); err != nil {
t.Fatalf("seed: %v", err)
}
for _, counter := range []bool{false, true} {
mode := "hash"
if counter {
mode = "counter"
if err := base.SetConfig(ctx, "issue_id_mode", "counter"); err != nil {
t.Fatal(err)
}
}
for _, wisp := range []bool{false, true} {
// (a) the store-internal path (withRetryTx bodies): issueops directly.
issues := createReplayFixtureIssues(t)
if wisp {
for _, is := range issues {
is.Ephemeral = true
}
}
before := deepCopyValue(reflect.ValueOf(issues)).Interface().([]*types.Issue)
restore := issueops.SnapshotCreateInputs(issues)
tx, err := base.db.BeginTx(ctx, nil)
if err != nil {
t.Fatal(err)
}
_, cerr := issueops.CreateIssuesInTxWithResult(ctx, tx, issues, "tester", storage.BatchCreateOptions{SkipDependencyValidationErrors: true})
_ = tx.Rollback()
if cerr != nil {
t.Fatalf("%s/wisp=%v create: %v", mode, wisp, cerr)
}
if issues[0].ID == "" || issues[0].Dependencies[0].IssueID == "" || issues[0].Comments[0].ID == "" || issues[1].ClosedAt == nil {
t.Fatalf("%s/wisp=%v: the create path wrote nothing back; test is vacuous", mode, wisp)
}
restore()
assertCreateInputsRestored(t, mode+"/issueops/wisp="+map[bool]string{true: "y", false: "n"}[wisp], before, issues)
// (b) the transaction path cmd/bd replays (RunInTransaction + tx.CreateIssues).
issues = createReplayFixtureIssues(t)
if wisp {
for _, is := range issues {
is.Ephemeral = true
}
}
before = deepCopyValue(reflect.ValueOf(issues)).Interface().([]*types.Issue)
restore = issueops.SnapshotCreateInputs(issues)
sentinel := errors.New("roll back")
err = base.RunInTransaction(ctx, "test: snapshot restore", func(tx storage.Transaction) error {
if err := tx.CreateIssues(ctx, issues, "tester"); err != nil {
return err
}
return sentinel
})
if !errors.Is(err, sentinel) {
t.Fatalf("%s/wisp=%v tx create: %v", mode, wisp, err)
}
restore()
assertCreateInputsRestored(t, mode+"/tx/wisp="+map[bool]string{true: "y", false: "n"}[wisp], before, issues)
}
}
}

View file

@ -0,0 +1,297 @@
package dolt
import (
"context"
"database/sql"
"fmt"
"sort"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/rowid"
"github.com/steveyegge/beads/internal/types"
)
// A create that loses Dolt's commit-time merge to a concurrent writer is
// rolled back by the server and replayed by withRetryTx. These tests prove the
// replay starts from the caller's request: it mints its own ID and derives its
// inline edges and comments from that ID, rather than re-using what the
// rolled-back attempt wrote back into the caller's issue.
// TestConcurrentCounterModeCreatesNeverShareAnID: in counter mode two writers
// that overlap mint the same number, so one of them loses at commit and is
// replayed. A replay that re-used the rolled-back attempt's ID took the
// explicit-ID upsert path and overwrote the winner's issue, and both callers
// were told the same ID.
func TestConcurrentCounterModeCreatesNeverShareAnID(t *testing.T) {
base, cleanup := setupConcurrentTestStore(t)
defer cleanup()
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
if err := base.SetConfig(ctx, "issue_id_mode", "counter"); err != nil {
t.Fatalf("set counter mode: %v", err)
}
const writers, rounds = 4, 3
stores := []*DoltStore{base}
for len(stores) < writers {
stores = append(stores, openPeerStore(t, ctx, base))
}
var mu sync.Mutex
titleByID := map[string]string{}
errs := make(chan error, writers*rounds*3)
record := func(issue *types.Issue) {
mu.Lock()
defer mu.Unlock()
if prev, dup := titleByID[issue.ID]; dup {
errs <- fmt.Errorf("id %s handed to both %q and %q", issue.ID, prev, issue.Title)
return
}
titleByID[issue.ID] = issue.Title
}
newIssue := func(title string) *types.Issue {
return &types.Issue{Title: title, Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
}
var wg sync.WaitGroup
start := make(chan struct{})
for w, s := range stores {
wg.Add(1)
go func() {
defer wg.Done()
<-start
for r := range rounds {
single := newIssue(fmt.Sprintf("single %d-%d", w, r))
if err := s.CreateIssue(ctx, single, "writer"); err != nil {
errs <- fmt.Errorf("create %q: %w", single.Title, err)
} else {
record(single)
}
batch := []*types.Issue{newIssue(fmt.Sprintf("batch %d-%d-a", w, r)), newIssue(fmt.Sprintf("batch %d-%d-b", w, r))}
if err := s.CreateIssues(ctx, batch, "writer"); err != nil {
errs <- fmt.Errorf("create batch %d-%d: %w", w, r, err)
} else {
record(batch[0])
record(batch[1])
}
}
}()
}
close(start)
wg.Wait()
close(errs)
for err := range errs {
t.Error(err)
}
for id, title := range titleByID {
got, err := base.GetIssue(ctx, id)
if err != nil {
t.Errorf("read %s: %v", id, err)
continue
}
if got.Title != title {
t.Errorf("issue %s holds %q, want %q: a replayed create overwrote it", id, got.Title, title)
}
}
if want := writers * rounds * 3; len(titleByID) != want && !t.Failed() {
t.Fatalf("distinct issues = %d, want %d", len(titleByID), want)
}
}
// replayIDModes are the two ID modes a replayed create must restart in: in
// counter mode the competitor mints the very number the loser's rolled-back
// attempt had; in hash mode the replay mints a fresh ID, and the events journal
// makes the two overlapping writers collide.
var replayIDModes = []struct {
name string
counterMode bool
}{
{name: "counter mode", counterMode: true},
{name: "hash mode", counterMode: false},
}
func setUpReplayIDMode(t *testing.T, ctx context.Context, base *DoltStore, counterMode bool) {
t.Helper()
if counterMode {
if err := base.SetConfig(ctx, "issue_id_mode", "counter"); err != nil {
t.Fatalf("set counter mode: %v", err)
}
return
}
base.SetEventsJournalEnabled(true)
}
// TestReplayedBatchCreateKeepsInlineEdgeOnItsOwnIssue holds a batch create
// open (through the skipped-edge callback it runs inside its transaction) until
// a competitor has committed, so the batch loses at commit and is replayed. Its
// inline edge named no source, so the create filled the source in from the
// minted ID: the replay must fill it in again from its own ID.
func TestReplayedBatchCreateKeepsInlineEdgeOnItsOwnIssue(t *testing.T) {
for _, mode := range replayIDModes {
t.Run(mode.name, func(t *testing.T) {
base, cleanup := setupConcurrentTestStore(t)
defer cleanup()
ctx, cancel := concurrentTestContext(t)
defer cancel()
setUpReplayIDMode(t, ctx, base, mode.counterMode)
peer := openPeerStore(t, ctx, base)
peer.SetEventsJournalEnabled(!mode.counterMode)
if err := base.CreateIssue(ctx, &types.Issue{ID: "test-anchor", Title: "anchor", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}, "seed"); err != nil {
t.Fatalf("seed anchor: %v", err)
}
prepared, release := make(chan struct{}), make(chan struct{})
var attempts atomic.Int32
opts := storage.BatchCreateOptions{
// Runs inside the create transaction, after the anchor edge is
// written: hold the first attempt open until the competitor
// has committed.
OnSkippedDependency: func(_, _, _ string) {
if attempts.Add(1) == 1 {
close(prepared)
<-release
}
},
}
loser := &types.Issue{
Title: "loser", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
Dependencies: []*types.Dependency{
{DependsOnID: "test-anchor", Type: types.DepBlocks},
{DependsOnID: "test-missing", Type: types.DepRelated},
},
}
done := make(chan error, 1)
go func() { done <- base.CreateIssuesWithFullOptions(ctx, []*types.Issue{loser}, "loser", opts) }()
waitForTransactionPrepared(t, ctx, prepared, "loser")
competitor := &types.Issue{Title: "competitor", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := peer.CreateIssue(ctx, competitor, "competitor"); err != nil {
close(release)
t.Fatalf("competitor create: %v", err)
}
close(release)
if err := <-done; err != nil {
t.Fatalf("loser create was not replayed to success: %v", err)
}
if n := attempts.Load(); n < 2 {
t.Fatalf("loser create ran %d attempt(s), want a replay: the competitor's commit did not conflict", n)
}
assertCompetitorIntact(t, ctx, base, loser, competitor)
if got := edgeSourcesInto(t, ctx, base, "test-anchor"); len(got) != 1 || got[0] != loser.ID {
t.Fatalf("edges into test-anchor come from %v, want exactly [%s] (the loser's own issue); competitor is %s", got, loser.ID, competitor.ID)
}
})
}
}
// TestReplayedCreateDerivesInlineCommentIDFromItsOwnIssue holds a single
// create's COMMIT at a proxy until a competitor has committed, so the create
// loses and is replayed. Its inline comment carried no ID, so the create
// derived one from the minted issue ID: the replay must derive it again from
// its own.
func TestReplayedCreateDerivesInlineCommentIDFromItsOwnIssue(t *testing.T) {
for _, mode := range replayIDModes {
t.Run(mode.name, func(t *testing.T) {
base, cleanup := setupConcurrentTestStore(t)
defer cleanup()
ctx, cancel := concurrentTestContext(t)
defer cancel()
setUpReplayIDMode(t, ctx, base, mode.counterMode)
proxy := startSQLWireProxy(t)
via := openStoreOnPort(t, ctx, base, proxy.port())
via.SetEventsJournalEnabled(!mode.counterMode)
loser := &types.Issue{
Title: "loser", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask,
Comments: []*types.Comment{{Author: "loser", Text: "filed with the issue"}},
}
mark := proxy.mark()
held, release := proxy.holdNext(isCommitStatement)
done := make(chan error, 1)
go func() { done <- via.CreateIssue(ctx, loser, "loser") }()
select {
case <-held:
case err := <-done:
t.Fatalf("loser create finished before its COMMIT was held: %v", err)
case <-ctx.Done():
t.Fatalf("loser create never reached COMMIT: %v", ctx.Err())
}
competitor := &types.Issue{Title: "competitor", Status: types.StatusOpen, Priority: 2, IssueType: types.TypeTask}
if err := base.CreateIssue(ctx, competitor, "competitor"); err != nil {
release()
t.Fatalf("competitor create: %v", err)
}
release()
if err := <-done; err != nil {
t.Fatalf("loser create was not replayed to success: %v", err)
}
insertsComment := func(q string) bool { return strings.HasPrefix(q, "INSERT INTO comments") }
if n := proxy.statementsSince(mark, insertsComment); n != 2 {
t.Fatalf("loser create wrote its comment in %d attempt(s), want 2 (one lost to the competitor, one replay)", n)
}
assertCompetitorIntact(t, ctx, base, loser, competitor)
assertCommentDerivedFromOwnIssue(t, ctx, base, loser.ID)
})
}
}
func assertCompetitorIntact(t *testing.T, ctx context.Context, base *DoltStore, loser, competitor *types.Issue) {
t.Helper()
if loser.ID == competitor.ID {
t.Fatalf("loser and competitor were both handed %s", loser.ID)
}
got, err := base.GetIssue(ctx, competitor.ID)
if err != nil {
t.Fatalf("read competitor %s: %v", competitor.ID, err)
}
if got.Title != "competitor" {
t.Fatalf("competitor %s holds %q: the replayed create overwrote it", competitor.ID, got.Title)
}
}
func edgeSourcesInto(t *testing.T, ctx context.Context, s *DoltStore, target string) []string {
t.Helper()
rows, err := s.db.QueryContext(ctx, "SELECT issue_id FROM dependencies WHERE depends_on_issue_id = ?", target)
if err != nil {
t.Fatalf("query edges into %s: %v", target, err)
}
defer rows.Close()
var sources []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
t.Fatalf("scan edge source: %v", err)
}
sources = append(sources, id)
}
if err := rows.Err(); err != nil {
t.Fatalf("read edges into %s: %v", target, err)
}
sort.Strings(sources)
return sources
}
// assertCommentDerivedFromOwnIssue checks issueID's one comment carries the ID
// derived from issueID itself, not one left over from a rolled-back attempt
// that derived it from that attempt's issue ID.
func assertCommentDerivedFromOwnIssue(t *testing.T, ctx context.Context, s *DoltStore, issueID string) {
t.Helper()
comments, err := s.GetIssueComments(ctx, issueID)
if err != nil {
t.Fatalf("read comments of %s: %v", issueID, err)
}
if len(comments) != 1 {
t.Fatalf("%s has %d comments, want 1", issueID, len(comments))
}
c := comments[0]
want := rowid.New("comments", 0, rowid.Digest([]sql.NullString{
{String: issueID, Valid: true}, {String: c.Author, Valid: true},
{String: c.Text, Valid: true}, {String: issueops.FormatAuxTime(c.CreatedAt), Valid: true},
}))
if c.ID != want {
t.Fatalf("comment on %s has id %s, want %s (derived from its own issue): the replay re-used the rolled-back attempt's comment id", issueID, c.ID, want)
}
}

View file

@ -40,7 +40,14 @@ func (s *DoltStore) createIssue(ctx context.Context, issue *types.Issue, actor s
}
var result issueops.CreateIssueResult
// withRetryTx replays the body after a lost commit-time conflict, and the
// create writes its minted ID (and what derives from it) back into the
// caller's issue: each attempt restarts from the caller's request, or the
// replay would take the rolled-back ID as the caller's own and upsert over
// the winner's row.
restore := issueops.SnapshotCreateInputs([]*types.Issue{issue})
if err := s.withRetryTx(ctx, func(tx *sql.Tx) error {
restore()
// SkipPrefixValidation matches legacy behavior: single-issue path does
// not validate prefixes for explicit IDs.
bc, err := issueops.NewBatchContext(ctx, tx, storage.BatchCreateOptions{
@ -113,14 +120,19 @@ func (s *DoltStore) createIssuesWithFullOptions(ctx context.Context, issues []*t
issue.Ephemeral = true
}
}
restore := issueops.SnapshotCreateInputs(issues)
return s.withRetryTx(ctx, func(tx *sql.Tx) error {
restore()
_, err := issueops.CreateIssuesInTxWithResult(ctx, tx, issues, actor, opts)
return err
})
}
// Each attempt restarts from the caller's request; see createIssue.
restore := issueops.SnapshotCreateInputs(issues)
var result issueops.CreateIssuesResult
if err := s.withRetryTx(ctx, func(tx *sql.Tx) error {
restore()
var err error
result, err = issueops.CreateIssuesInTxWithResult(ctx, tx, issues, actor, opts)
return err

View file

@ -0,0 +1,33 @@
package dolt
import (
"context"
"testing"
)
// openPeerStore opens a second store on store's database, with a connection
// pool of its own as a second bd process would have.
func openPeerStore(t *testing.T, ctx context.Context, store *DoltStore) *DoltStore {
t.Helper()
return openStoreOnPort(t, ctx, store, testServerPort)
}
// openStoreOnPort opens a store on store's database through whatever listens
// on port: the test server itself, or a test proxy in front of it.
func openStoreOnPort(t *testing.T, ctx context.Context, store *DoltStore, port int) *DoltStore {
t.Helper()
peer, err := New(ctx, &Config{
Path: t.TempDir(),
CommitterName: "test",
CommitterEmail: "test@example.com",
ServerHost: "127.0.0.1",
ServerPort: port,
Database: store.database,
MaxOpenConns: 2,
})
if err != nil {
t.Fatalf("open store on %s through port %d: %v", store.database, port, err)
}
t.Cleanup(func() { _ = peer.Close() })
return peer
}

View file

@ -0,0 +1,205 @@
package dolt
import (
"fmt"
"io"
"net"
"strings"
"sync"
"sync/atomic"
"testing"
)
// comQuery is the MySQL command byte of a text-protocol statement. Every
// statement a store sends travels as one: its DSN interpolates parameters.
const comQuery = 0x03
// sqlWireProxy forwards MySQL wire traffic between stores under test and the
// test Dolt server. It records every statement it forwards and can trap the
// next one that matches a predicate, either holding it until the test releases
// it (to order two writers' commits deterministically) or forwarding it and
// cutting the client off before the server's answer arrives (a response lost
// after the server acted on the statement). Production code needs no test hook.
type sqlWireProxy struct {
ln net.Listener
target string
mu sync.Mutex
trap *sqlWireTrap
held []*sqlWireTrap
queries []string
conns []net.Conn
}
type sqlWireTrap struct {
match func(query string) bool
drop bool
sprung chan struct{}
release chan struct{}
released sync.Once
}
func (trap *sqlWireTrap) open() { trap.released.Do(func() { close(trap.release) }) }
func startSQLWireProxy(t *testing.T) *sqlWireProxy {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("proxy listen: %v", err)
}
p := &sqlWireProxy{ln: ln, target: fmt.Sprintf("127.0.0.1:%d", testServerPort)}
go func() {
for {
client, err := ln.Accept()
if err != nil {
return
}
go p.handle(client)
}
}()
t.Cleanup(p.close)
return p
}
func (p *sqlWireProxy) port() int { return p.ln.Addr().(*net.TCPAddr).Port }
// holdNext traps the next statement matching match and keeps it from the
// server until release is called. sprung is closed once it is caught.
func (p *sqlWireProxy) holdNext(match func(query string) bool) (sprung <-chan struct{}, release func()) {
trap := p.arm(match, false)
return trap.sprung, trap.open
}
// dropResponseToNext traps the next statement matching match: the server runs
// it, and the client's connection is cut instead of receiving the answer.
func (p *sqlWireProxy) dropResponseToNext(match func(query string) bool) (sprung <-chan struct{}) {
return p.arm(match, true).sprung
}
func (p *sqlWireProxy) arm(match func(query string) bool, drop bool) *sqlWireTrap {
trap := &sqlWireTrap{match: match, drop: drop, sprung: make(chan struct{}), release: make(chan struct{})}
p.mu.Lock()
defer p.mu.Unlock()
if p.trap != nil {
p.trap.open()
}
p.trap = trap
return trap
}
// mark returns a position in the statement log for statementsSince.
func (p *sqlWireProxy) mark() int {
p.mu.Lock()
defer p.mu.Unlock()
return len(p.queries)
}
// statementsSince counts the statements forwarded after mark that match.
func (p *sqlWireProxy) statementsSince(mark int, match func(query string) bool) int {
p.mu.Lock()
defer p.mu.Unlock()
n := 0
for _, q := range p.queries[mark:] {
if match(q) {
n++
}
}
return n
}
// intercept logs query and hands back the armed trap when it catches query.
func (p *sqlWireProxy) intercept(query string) *sqlWireTrap {
p.mu.Lock()
defer p.mu.Unlock()
p.queries = append(p.queries, query)
trap := p.trap
if trap == nil || !trap.match(query) {
return nil
}
p.trap = nil
if !trap.drop {
p.held = append(p.held, trap)
}
close(trap.sprung)
return trap
}
func (p *sqlWireProxy) close() {
_ = p.ln.Close()
p.mu.Lock()
defer p.mu.Unlock()
if p.trap != nil {
p.trap.open()
}
for _, trap := range p.held {
trap.open()
}
for _, c := range p.conns {
_ = c.Close()
}
}
func (p *sqlWireProxy) handle(client net.Conn) {
server, err := net.Dial("tcp", p.target)
if err != nil {
_ = client.Close()
return
}
p.mu.Lock()
p.conns = append(p.conns, client, server)
p.mu.Unlock()
var once sync.Once
closeBoth := func() { once.Do(func() { _ = client.Close(); _ = server.Close() }) }
var dropping atomic.Bool
go func() {
buf := make([]byte, 64*1024)
for {
n, err := server.Read(buf)
if n > 0 {
if dropping.Load() {
closeBoth()
return
}
if _, werr := client.Write(buf[:n]); werr != nil {
closeBoth()
return
}
}
if err != nil {
closeBoth()
return
}
}
}()
hdr := make([]byte, 4)
for {
if _, err := io.ReadFull(client, hdr); err != nil {
closeBoth()
return
}
n := int(hdr[0]) | int(hdr[1])<<8 | int(hdr[2])<<16
packet := make([]byte, 4+n)
copy(packet, hdr)
if _, err := io.ReadFull(client, packet[4:]); err != nil {
closeBoth()
return
}
// A command starts a new exchange (sequence id 0); the handshake's
// packets do not.
if hdr[3] == 0 && n > 1 && packet[4] == comQuery {
if trap := p.intercept(strings.TrimSpace(string(packet[5:]))); trap != nil {
if trap.drop {
dropping.Store(true)
} else {
<-trap.release
}
}
}
if _, err := server.Write(packet); err != nil {
closeBoth()
return
}
}
}
func isCommitStatement(query string) bool { return strings.EqualFold(query, "COMMIT") }

View file

@ -36,6 +36,7 @@ go_library(
"create_dep_batch.go",
"create_fastpath_testhook.go",
"create_only_guard.go",
"create_replay.go",
"cycle_report.go",
"cycles.go",
"delete.go",
@ -145,6 +146,7 @@ go_test(
"create_batch_cache_test.go",
"create_only_errors_test.go",
"create_only_guard_test.go",
"create_replay_test.go",
"create_test.go",
"custom_types_parity_test.go",
"cycle_report_test.go",

View file

@ -0,0 +1,67 @@
package issueops
import "github.com/steveyegge/beads/internal/types"
// SnapshotCreateInputs records what the create path writes back into the
// caller's issues and returns a func that puts it back, so a create that a
// transaction retry replays starts every attempt from the caller's request
// instead of from what a rolled-back attempt minted.
//
// The create path writes back three things:
//
// - the Issue struct itself: the minted ID (assignCreateIssueIDInTx) and the
// defaulted timestamps and content hash (PrepareIssueForInsert);
// - through each inline dependency, the owning issue's ID as the edge's
// source when the edge named none (PersistDependenciesWithOptionsResult);
// - through each inline comment, the content-derived ID when the comment
// carried none (PersistComments).
//
// Restoring only the struct would leave the last two pointing at the
// rolled-back attempt's ID. The dependency and comment objects stay the
// caller's: only the fields the create path writes are reset. A new write into
// caller-owned state on the create path needs its restore here.
func SnapshotCreateInputs(issues []*types.Issue) (restore func()) {
type depSource struct {
dep *types.Dependency
issueID string
}
type commentID struct {
comment *types.Comment
id string
}
type issueState struct {
issue *types.Issue
fields types.Issue
deps []depSource
comments []commentID
}
saved := make([]issueState, 0, len(issues))
for _, issue := range issues {
if issue == nil {
continue
}
state := issueState{issue: issue, fields: *issue}
for _, dep := range issue.Dependencies {
if dep != nil {
state.deps = append(state.deps, depSource{dep: dep, issueID: dep.IssueID})
}
}
for _, comment := range issue.Comments {
if comment != nil {
state.comments = append(state.comments, commentID{comment: comment, id: comment.ID})
}
}
saved = append(saved, state)
}
return func() {
for _, state := range saved {
*state.issue = state.fields
for _, d := range state.deps {
d.dep.IssueID = d.issueID
}
for _, c := range state.comments {
c.comment.ID = c.id
}
}
}
}

View file

@ -0,0 +1,70 @@
package issueops
import (
"testing"
"time"
"github.com/steveyegge/beads/internal/types"
)
// TestSnapshotCreateInputsRestoresWhatTheCreatePathWritesBack covers every
// write the create path makes into a caller's issues, including the two it
// makes through pointers the Issue struct only holds: the minted ID copied into
// each inline dependency that named no source, and the derived ID stamped onto
// each inline comment that had none. A struct copy restores neither.
func TestSnapshotCreateInputsRestoresWhatTheCreatePathWritesBack(t *testing.T) {
presetEdge := &types.Dependency{IssueID: "test-explicit", DependsOnID: "test-other", Type: types.DepRelated}
inlineEdge := &types.Dependency{DependsOnID: "test-anchor", Type: types.DepBlocks}
inlineComment := &types.Comment{Author: "a", Text: "first"}
importedComment := &types.Comment{ID: "c-imported", Author: "b", Text: "second"}
issue := &types.Issue{
Title: "loser",
Labels: []string{"area:storage"},
Dependencies: []*types.Dependency{inlineEdge, nil, presetEdge},
Comments: []*types.Comment{inlineComment, importedComment, nil},
}
second := &types.Issue{Title: "second"}
restore := SnapshotCreateInputs([]*types.Issue{issue, nil, second})
// What one create attempt leaves behind.
issue.ID = "test-1"
issue.CreatedAt = time.Now()
issue.UpdatedAt = issue.CreatedAt
issue.ContentHash = "attempt-1-hash"
inlineEdge.IssueID = issue.ID
inlineComment.ID = "c-attempt-1"
second.ID = "test-2"
restore()
if issue.ID != "" || !issue.CreatedAt.IsZero() || !issue.UpdatedAt.IsZero() || issue.ContentHash != "" {
t.Fatalf("issue after restore = id %q, created %v, updated %v, hash %q; want the caller's empty values",
issue.ID, issue.CreatedAt, issue.UpdatedAt, issue.ContentHash)
}
if second.ID != "" {
t.Fatalf("second issue id after restore = %q, want empty", second.ID)
}
if inlineEdge.IssueID != "" {
t.Fatalf("inline edge source after restore = %q, want empty so the replay derives it from the new id", inlineEdge.IssueID)
}
if presetEdge.IssueID != "test-explicit" {
t.Fatalf("explicit edge source after restore = %q, want the caller's %q", presetEdge.IssueID, "test-explicit")
}
if inlineComment.ID != "" {
t.Fatalf("inline comment id after restore = %q, want empty so the replay derives it again", inlineComment.ID)
}
if importedComment.ID != "c-imported" {
t.Fatalf("imported comment id after restore = %q, want the caller's %q", importedComment.ID, "c-imported")
}
if issue.Dependencies[0] != inlineEdge || issue.Comments[0] != inlineComment {
t.Fatal("restore replaced the caller's dependency or comment objects instead of resetting them")
}
// Every replay starts from the same request.
issue.ID = "test-3"
inlineEdge.IssueID = issue.ID
restore()
if issue.ID != "" || inlineEdge.IssueID != "" {
t.Fatalf("second restore left id %q and edge source %q", issue.ID, inlineEdge.IssueID)
}
}