* fix(dolt): honor dolt.auto-commit=batch/off on the proxied-server write path (#4995)
In proxied-server mode every write minted a Dolt history commit whatever
dolt.auto-commit said. Two causes:
- cmd/bd/main.go never applied issueOpsContext(rootCtx) in the
proxiedServerMode branch of the root pre-run, so the context carried no
deferred-commit marker.
- internal/storage/uow/doltserver_tx.go: doltServerTx.Commit ignored
issueops.VersionCommitDeferred(ctx) and ran CALL DOLT_COMMIT('-Am', ?)
whenever a message was supplied.
Apply issueOpsContext to rootCtx in the proxied setup, and have
doltServerTx.Commit blank its message under a deferred context so the existing
plain-COMMIT branch persists the write to the working set without advancing
Dolt history. dolt.auto-commit=on is unchanged.
This commit alone strands batch/off writes in the working set; the explicit
flush point (`bd dolt commit`) follows in the later commits of this series.
(Originally recorded as "witness: salvage uncommitted work (ocb-ani)"; the
change itself is unmodified.)
* test(uow,cmd/bd): add regression tests for proxied dolt.auto-commit batch/off (#4995)
Add focused regression tests verifying that proxied dolt.auto-commit batch/off
defer Dolt version commits:
- cmd/bd: TestIssueOpsContextDefersByModeInProxiedServerMode verifies that
issueOpsContext defers version commits in proxied-server mode when
dolt.auto-commit is batch or off.
- internal/storage/uow: TestDoltServerTxCommitDefersDoltCommitUnderDeferredContext
and TestDoltServerTxRunTxWithDeferredContextSkipsDoltCommit verify that
doltServerTx.Commit blanks the commit message under a deferred context,
bypassing DOLT_COMMIT and status checks, and executing plain COMMIT to
persist writes into the working set without advancing Dolt history.
* fix(dolt): add the proxied flush point dolt.auto-commit=batch/off defers to (#4995)
The deferral half of this PR makes dolt.auto-commit=batch/off actually take
effect on the proxied-server write path. On its own that turns batch/off into
"never commit": proxied mode returns from the root pre-run before newDoltStore
runs, so getStore() is nil and `bd dolt commit` has no store to commit
through. Writes pile up in the sql-server working set where `dolt log`,
`dolt push` and backups never see them.
Add the missing half. cmd/bd/dolt_commit_proxied_server.go is the proxied dual
of `bd dolt commit`: it takes the uow.MaintenanceProvider route (RunNonTx) and
runs CALL DOLT_COMMIT('-Am', ?) on a pinned connection, gated on
issueops.HasPendingChanges. It deliberately does not go through
doltServerTx.Commit, which blanks its own message under the deferred context;
routing the flush through the deferral it exists to discharge would make the
flush a no-op. doltCommitCmd dispatches to it on usesProxiedServer(); the
message is built before the branch so both routes name the same default, and
the "Committed." / "Nothing to commit." contract is unchanged.
--dolt-auto-commit's help no longer claims proxied-server routes are
unaffected: it names the proxied flush point and keeps the SIGTERM/SIGHUP
flush marked embedded-only. docs/reference/configuration.md says the same.
`bd vc commit`'s proxied refusal now points at `bd dolt commit`.
Tests:
- cmd/bd/dolt_commit_proxied_integration_test.go: three batch writes leave
dolt_log unchanged and stay readable, `bd dolt commit` advances it by
exactly one, a second `bd dolt commit` reports "Nothing to commit." with
dolt_log unchanged, and one write under --dolt-auto-commit on advances it
by one. On the PR's original base it failed at the `bd dolt commit` step
with "no store available". On current main the front door refuses the
command before that code is reached, so this test is only green together
with the next commit, which moves `dolt commit` to supported in the proxy
capability registry.
- TestIssueOpsContextDefersByModeInProxiedServerMode renamed to
TestIssueOpsContextDefersByModePolicyTable and the proxiedServerMode
assignment dropped: issueOpsContext branches only on writesCommitNow(),
never on the storage mode, so the old name over-promised.
- doltserver_tx_test.go: "bd: update bd-4995" -> "bd: update GH#4995", which
no longer reads as a bead id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(proxy): move `dolt commit` from refused to supported in the capability registry
Main's proxy capability registry (added by #6269, 24a9da6cc) declares
`dolt commit` direct-only: it has a "proxy.dolt_commit.unsupported" refusal
in proxyMaintenanceRefusals (cmd/bd/proxy_capability.go) and is classed
HistoryDirectOnly in historyCapabilityMatrix (cmd/bd/history_capability.go).
The front door therefore refuses `bd dolt commit` in proxied-server mode
before dispatch, so the proxied dual added earlier in this series is
unreachable, and dolt.auto-commit=batch/off has no way to flush.
the history/VC front-door contract, and neither the registry, its tests nor
the docs explain why `dolt commit` was listed. Nothing on main implemented
`dolt commit` for the proxied route at the time, which is consistent with the
entry being a statement of what existed rather than a design decision.
This flips a capability main had declared unsupported, deliberately: the
proxied route now has an implementation (runDoltCommitProxiedServer), and
batch/off need it as their flush point (GH#4995).
Declared the same way as the other proxied-supported command,
`dolt remote remove`: HistoryProxySupported in the history matrix, and no row
in proxyMaintenanceRefusals.
Tests follow: history_capability_test.go moves `dolt commit` from the
direct-only assertions and refusal contract to the proxy-supported ones
(including the nested front-door pass-through), and
history_remote_frontdoor_matrix_test.go drops the refusal row and adds a
`dolt commit` pass-through check to TestHistoryRemoteSupportedFrontDoorParity.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(proxy): exempt explicit commit points from the proxied deferral (#4995)
Maintainer review fixup for PR #6499, adopt-pr iterations 1-2.
Major (Contract & Interface Fidelity). The deferral is applied once, to
rootCtx, so it blanketed BOTH commit classes the direct route distinguishes by
which wrapper a verb picks: policy-honoring writes (transactHonoringAutoCommit,
per-verb issueOpsContext) and explicit commit points (transact, which passes
the caller's message whatever the policy says). Under dolt.auto-commit=batch a
proxied `bd batch -m "release batch"` therefore persisted the rows, discarded
the message and minted nothing, while the identical command and config on the
direct route committed it — and batch_proxied_server.go's own contract comment
rejects exactly that ("would persist these writes while silently bypassing
Dolt history"), as does batch.go's "the contract is the same on both backends".
issueops.WithImmediateVersionCommit clears the marker and cmd/bd's
explicitCommitPointContext applies it in the duals whose direct twin calls
transact. Scope is the intersection of the transact call sites with
proxyPermittedPaths: batch, mol bond/pour/squash, mol wisp create, and the
WISP half of mol burn (cook and migrate issues also call transact but are
refused in proxied mode, so they have no dual to exempt). The review sized the
radius at batch + 4 mol verbs; mol wisp create is the same class — its direct
twin reaches cloneSubgraph's transact through spawnMoleculeWithOptions — so it
is exempted with them.
Membership is per commit site, not per command (iteration 2). `bd mol burn` is
two classes on the direct route: wisps go through burnWisps -> transact, while
persistent molecules go through deleteBatch -> issueOpsContext ->
dolt/deleter.go, which returns before DOLT_COMMIT when the context defers.
Exempting the whole command minted one Dolt commit per persistent molecule
under batch/off — on a destructive verb, where the identical direct-route
command defers them all to `bd dolt commit`. runMolBurnProxiedServer now
applies the exemption to the wisp RunTxResult only, and the flag help,
configuration.md and the helper's own radius comment say so.
Minors:
- The flush passes '--author' like DoltStorage.CommitAll, so the one commit
batch mode keeps is attributed to the bd actor rather than the SQL session
user (without it dolt_log reads `root <root@localhost>`).
- runDoltCommitProxiedServer calls the shared runProxiedNonTx instead of
re-implementing its provider checks, so `bd --json dolt commit` keeps the
JSON error envelope; `dolt commit`'s RunE no longer re-wraps an
already-reported failure, and runProxiedNonTx's message gains the provider
%T the open-coded version had.
- The flag help and configuration.md drop two false claims: that the deferral
covers "every write" in proxied mode (the bd serve plane over the same
provider never touches rootCtx, so daemon writes still commit per write) and
that the SIGTERM/SIGHUP flush is "embedded-only" (flushBatchCommitOnShutdown
gates on an open store, which direct SQL-server mode has; only proxied mode
lacks one). Both surfaces now name the explicit-commit-point exemption.
- Both user-facing enumerations of that exemption are now derived from the
explicitCommitPointContext call sites rather than restated (iteration 2):
they had omitted mol wisp create, listed mol burn wholesale, and implied a
message-less `bd batch` defers when batch.go synthesizes a default message
and the dual exempts unconditionally, so every proxied batch commits.
- doltserver_tx.go's ephemeral-form comment names its second entry point —
RunTx/RunTxResult under a deferred context — where its
ONLY-dolt_ignored-state rationale stops applying.
Not done: regenerating docs/CLI_REFERENCE.md. Its --dolt-auto-commit drift is
base-preexisting, and the generated CLI docs are built from the release tag
pinned in docs/cli-docs.pin, not from this source, so touching them converts
the drift gate's inherited-drift warn-pass into a failure.
Tests. The uow blanking table gains the immediate-commit arm (tagless, so it
runs in every lane); the proxied integration test gains an explicit-batch-commit
step with a deferred plain create as its control, plus the flush attribution
assertions. Mutation-proved: reverting explicitCommitPointContext to a no-op
reds the new step (dolt_log 80 -> 80, no commit minted), and dropping
'--author' reds the attribution assertion with committer=root.
Iteration 2 adds the WIRING pin those left open — the mechanism was covered,
but deleting `ctx = explicitCommitPointContext(ctx)` from a dual was green in
every lane, which is why the mol burn over-application above was invisible.
dolt_autocommit_exemption_wiring_test.go is tagless and needs no server: a
uow.UnitOfWorkProvider that records the context RunTxResult hands NewUOW and
then refuses observes exactly the context each dual gives its transaction.
Mutation-proved per arm: neutering the exemption at batch/bond/squash reds
those three rows and only those; dropping it from the wisp RunTxResult reds
the wisp arm alone; and restoring the command-wide reassignment reds the
persistent arm alone, reproducing the defect this iteration fixes.
Validation: go build ./..., go vet (cgo and CGO_ENABLED=0), gofmt clean;
internal/storage/issueops; internal/storage/uow -run TestDoltServerTx; the
cmd/bd registry / history / policy / batch / mol / wisp / errors battery; and
the gated real-server lane BEADS_TEST_PROXIED_SERVER=1 green for both
TestProxiedServerBatchDefersThenDoltCommitAdvancesHeadOnce (41s) and
TestProxiedServerMolBurn (33s, all four subtests including the mixed
wisp+persistent burn this iteration changed).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
234 lines
9.1 KiB
Go
234 lines
9.1 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"strconv"
|
|
"strings"
|
|
|
|
"github.com/steveyegge/beads/internal/storage/domain"
|
|
"github.com/steveyegge/beads/internal/storage/uow"
|
|
"github.com/steveyegge/beads/internal/types"
|
|
"github.com/steveyegge/beads/internal/workapi"
|
|
)
|
|
|
|
// runBatchProxiedServer executes an already-parsed batch script in ONE
|
|
// proxied-server unit of work.
|
|
//
|
|
// The two modes divide exactly where the command's contract does. Parsing,
|
|
// validation, the dry-run echo, the empty-input no-op, the default commit
|
|
// message and every byte of output stay in batch.go and are shared: a batch
|
|
// script means the same thing whichever backend runs it, and a script rejected
|
|
// by the parser is rejected before either backend is reached. What is per-mode
|
|
// is the transaction primitive and the verbs the ops dispatch against —
|
|
// storage.Transaction there, the unit of work's use cases here.
|
|
//
|
|
// uow.RunTx is the analog of the classic `transact`: one transaction, one
|
|
// DOLT_COMMIT carrying the caller's message, and rollback of the WHOLE batch on
|
|
// the first failing line (the line is named in the error exactly as the classic
|
|
// path names it). RunTx additionally redoes the entire batch in a fresh unit of
|
|
// work when Dolt reports a serialization failure — the batch is re-executed
|
|
// against the winner's committed rows, never re-committed on a session the
|
|
// server already rolled back.
|
|
//
|
|
// NOT RunTxEphemeral: a batch writes versioned tables, and the ephemeral form
|
|
// commits without a Dolt commit (its whole point for the leases table), which
|
|
// would persist these writes while silently bypassing Dolt history.
|
|
func runBatchProxiedServer(ctx context.Context, ops []batchOp, commitMsg string) ([]batchOpResult, error) {
|
|
if uowProvider == nil {
|
|
return nil, HandleErrorRespectJSON("proxied-server UOW provider not initialized")
|
|
}
|
|
// Explicit commit point, like `transact` on the direct route: the commit
|
|
// carries the caller's message whatever dolt.auto-commit says (GH#4995).
|
|
ctx = explicitCommitPointContext(ctx)
|
|
return uow.RunTxResult(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) ([]batchOpResult, string, error) {
|
|
// Built per attempt, not per call: a retried batch must report the
|
|
// results of the attempt that actually committed.
|
|
results := make([]batchOpResult, 0, len(ops))
|
|
for _, op := range ops {
|
|
res, rerr := runBatchOpUOW(ctx, uw, op)
|
|
if rerr != nil {
|
|
return nil, "", fmt.Errorf("line %d (%s): %w", op.line, op.raw, rerr)
|
|
}
|
|
results = append(results, res)
|
|
}
|
|
return results, commitMsg, nil
|
|
})
|
|
}
|
|
|
|
// runBatchOpUOW dispatches one parsed op against the unit of work, the proxied
|
|
// twin of runBatchOp's dispatch against storage.Transaction. Same grammar, same
|
|
// arity checks, same error strings, same per-op result — only the verb changes.
|
|
//
|
|
// Plane routing is explicit here because it has to be: the classic transaction
|
|
// methods route an id to the issues or wisps tables themselves, while the
|
|
// domain use cases expose the two planes as separate verbs. Every op that names
|
|
// an existing id resolves it through workapi.GetIssueOrWisp first (both planes,
|
|
// this transaction's own snapshot) and then calls the matching verb, so a batch
|
|
// line that touches a wisp touches the wisp tables on both backends.
|
|
func runBatchOpUOW(ctx context.Context, uw uow.UnitOfWork, op batchOp) (batchOpResult, error) {
|
|
actorName := getActor()
|
|
result := batchOpResult{Line: op.line, Op: op.cmd}
|
|
issues := uw.IssueUseCase()
|
|
|
|
switch op.cmd {
|
|
case "close":
|
|
if len(op.args) < 1 {
|
|
return result, fmt.Errorf("close requires <id>")
|
|
}
|
|
id := op.args[0]
|
|
reason := "Closed"
|
|
if len(op.args) > 1 {
|
|
reason = strings.Join(op.args[1:], " ")
|
|
}
|
|
resolved, isWisp, err := batchResolveOpTarget(ctx, uw, id)
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
// Deliberately the UNCHECKED close, matching the classic batch's
|
|
// issueops.CloseIssueInTx: `close <id>` in a batch does not apply close
|
|
// policy at all (the asymmetry with `update status=closed` is documented
|
|
// in the command's help and is contract, not oversight).
|
|
params := domain.CloseIssueParams{Reason: reason}
|
|
if isWisp {
|
|
_, err = issues.CloseWisp(ctx, resolved, params, actorName)
|
|
} else {
|
|
_, err = issues.CloseIssue(ctx, resolved, params, actorName)
|
|
}
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
result.Target = id
|
|
return result, nil
|
|
|
|
case "update":
|
|
if len(op.args) < 2 {
|
|
return result, fmt.Errorf("update requires <id> and at least one key=value")
|
|
}
|
|
id := op.args[0]
|
|
updates, err := parseUpdateKVs(op.args[1:])
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
resolved, isWisp, err := batchResolveOpTarget(ctx, uw, id)
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
if isWisp {
|
|
err = issues.UpdateWisp(ctx, resolved, updates, actorName)
|
|
} else {
|
|
err = issues.UpdateIssue(ctx, resolved, updates, actorName)
|
|
}
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
result.Target = id
|
|
return result, nil
|
|
|
|
case "create":
|
|
if len(op.args) < 3 {
|
|
return result, fmt.Errorf("create requires <type> <priority> <title>")
|
|
}
|
|
issueType := types.IssueType(op.args[0])
|
|
if strings.TrimSpace(op.args[0]) == "" {
|
|
return result, fmt.Errorf("create: type cannot be empty")
|
|
}
|
|
priority, err := strconv.Atoi(op.args[1])
|
|
if err != nil {
|
|
return result, fmt.Errorf("create: invalid priority %q: %w", op.args[1], err)
|
|
}
|
|
title := strings.Join(op.args[2:], " ")
|
|
if strings.TrimSpace(title) == "" {
|
|
return result, fmt.Errorf("create: title cannot be empty")
|
|
}
|
|
issue := &types.Issue{
|
|
Title: title,
|
|
IssueType: issueType,
|
|
Status: types.StatusOpen,
|
|
Priority: priority,
|
|
}
|
|
created, err := issues.CreateIssue(ctx, domain.CreateIssueParams{Issue: issue}, actorName)
|
|
if err != nil {
|
|
return result, err
|
|
}
|
|
// The classic path reports the id the store minted into the caller's
|
|
// struct; the use case hands the same row back as a result.
|
|
if created.Issue != nil {
|
|
result.Target = created.Issue.ID
|
|
} else {
|
|
result.Target = issue.ID
|
|
}
|
|
return result, nil
|
|
|
|
case "dep.add":
|
|
if len(op.args) < 2 {
|
|
return result, fmt.Errorf("dep add requires <from-id> <to-id>")
|
|
}
|
|
from, to := op.args[0], op.args[1]
|
|
depType := "blocks"
|
|
if len(op.args) >= 3 {
|
|
depType = op.args[2]
|
|
}
|
|
dt := types.DependencyType(depType)
|
|
if !dt.IsValid() {
|
|
return result, fmt.Errorf("dep add: invalid dependency type %q", depType)
|
|
}
|
|
dep := &types.Dependency{
|
|
IssueID: from,
|
|
DependsOnID: to,
|
|
Type: dt,
|
|
}
|
|
// AddDependencies (not the single-edge AddDependency) because it lands
|
|
// each edge in the plane its own SOURCE lives in, which is the routing
|
|
// the classic transaction does from isActiveWisp. One edge is still one
|
|
// edge, and the per-edge cycle probe runs.
|
|
//
|
|
// KNOWN DIVERGENCE, in the HISTORY only: every domain edge verb records
|
|
// a dependency_added event, while the classic batch reaches
|
|
// storage.Transaction.AddDependency, whose zero DependencyAddOptions
|
|
// leaves EmitEvent false — the "structural, quiet" setting meant for
|
|
// create-with-deps and reparenting, not for a verb a user typed. So a
|
|
// batched `dep add` shows up in `bd history` here and does not there.
|
|
// The edge itself, its plane, its type and the cycle verdict are
|
|
// identical; `bd dep add` at the CLI emits the event on BOTH backends,
|
|
// which is why the classic batch's silence reads as the outlier.
|
|
// Silencing it here would need a quiet variant on the use case that
|
|
// nothing else wants, so this port takes the event and names it rather
|
|
// than growing the domain surface to reproduce an inconsistency.
|
|
if _, err := uw.DependencyUseCase().AddDependencies(ctx, []*types.Dependency{dep}, actorName, domain.BulkAddDepsOpts{}); err != nil {
|
|
return result, err
|
|
}
|
|
result.Target = fmt.Sprintf("%s->%s", from, to)
|
|
return result, nil
|
|
|
|
case "dep.remove":
|
|
if len(op.args) < 2 {
|
|
return result, fmt.Errorf("dep remove requires <from-id> <to-id>")
|
|
}
|
|
from, to := op.args[0], op.args[1]
|
|
// BySource for the same reason dep.add uses the bulk verb: the edge is
|
|
// removed from the plane its source lives in. A missing edge is not an
|
|
// error on either backend — the removal is idempotent, and a batch that
|
|
// re-runs must not roll back on an edge already gone. The
|
|
// dependency_removed event carries the same divergence, and the same
|
|
// reasoning, as the added one above.
|
|
if _, err := uw.DependencyUseCase().RemoveDependencyBySource(ctx, from, to, actorName); err != nil {
|
|
return result, err
|
|
}
|
|
result.Target = fmt.Sprintf("%s->%s", from, to)
|
|
return result, nil
|
|
}
|
|
return result, fmt.Errorf("internal: unhandled batch op %q", op.cmd)
|
|
}
|
|
|
|
// batchResolveOpTarget resolves an id an op names to the row it is about and
|
|
// the plane that row lives in, inside this transaction. A missing id is an
|
|
// error, which rolls the whole batch back — the same verdict the classic path
|
|
// reaches when its store call cannot find the row.
|
|
func batchResolveOpTarget(ctx context.Context, uw uow.UnitOfWork, id string) (string, bool, error) {
|
|
issue, isWisp, err := workapi.GetIssueOrWisp(ctx, workapi.NewUOWDetailSource(uw), id)
|
|
if err != nil {
|
|
return "", false, err
|
|
}
|
|
return issue.ID, isWisp, nil
|
|
}
|