Beads 1.2 remediation: migration-consent gate — bd never migrates an existing database without explicit consent (#5764)
* feat(schema): never migrate an existing database without explicit consent (bd-qhn8y) Beads 1.2 release remediation. 1.2.0/1.2.1 auto-applied migrations 54-65 at store open on ANY command, including reads; older binaries then refuse the migrated database. MigrateUp now refuses pending main-sequence migrations on an existing database before any write (dolt_ignore seeding included) unless the operator consented: bd migrate / bd migrate schema (preview flags withhold), bd migrate --force, or BD_ALLOW_MIGRATE=1 (BD_ALLOW_REMOTE_MIGRATE honored as an alias). Fresh init and at-latest databases are unaffected; the #4259 remote gate keeps all its protections. Also: freeze shipped migrations (hash-pin test, main 0001-0065 + ignored 0001-0024), retract v1.2.0/v1.2.1 in go.mod, agent-executable un-migrate runbook (docs/recovery/unmigrate-schema-v65.md), and the upgrade smoke + historical migration harnesses now model the consent contract and assert the refusal carries guidance. No version bump here by design: cutting the release stays with the release owner. Rebased onto main800d43eb8d: the go.mod retract hunk is dropped - main already retracts v1.2.0/v1.2.1 (and v1.1.1) since #5782. The uow consent grant moves into the TestMain main added in #6441 (a second TestMain no longer compiles), and the new files are listed in the BUILD.bazel targets main now carries (gazelle). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(schema): cloned-workspace bootstrap carries creation-consent (bd-qhn8y) The consent gate treated a workspace `bd init --remote` / `bd bootstrap` just cloned into existence like a pre-existing database, refusing the smart gate's provably-safe first-mover migration and stranding init in the generic open-error path (bypassing the bd-4mpy7 finalize+guidance ergonomics). Cloning IS creating: those paths now record consent explicitly, and the #4259 remote-migrate gate still applies to them unchanged (consent is not its designated-migrator override). Fixes CI TestEmbeddedInit/remote_behind_schema_gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(migration): consent first in main's wisp lanes too (bd-qhn8y) The rebase onto800d43eb8dbrought in #6049's wisp-plane lanes, which postdate this PR's harness changes. Each one makes an ordinary candidate command the first touch on an old-schema database, and the consent gate refuses it: - historical-dolt-upgrade-test.sh run_wisp_plane_upgrade opens with `bd list` before any migrate: "candidate could not open the seeded wisp workspace" for v1.0.1, v1.1.0, v1.1.2 and v1.2.2 (the Migration Test Harness dolt-runtime shard). - run_dirty_tracked_wisp_scenario (v1.1.2): the consent refusal stops the open before MigrateUp reaches the #5816 dirty tables, so the lane fails "without naming them". - upgrade-smoke-test.sh "Wisp plane survives upgrade" upgrades with a bare cand_init, so list/ready/blocked/update all hit the refusal (4 failures each for v1.3.0 and v1.3.1, both in the PR smoke set). Use the PR's own pattern: the explicit `bd migrate schema` is the first candidate touch (cand_upgrade in the smoke script). Base800d43eb8dpasses these lanes unchanged; with this fixup the full 14-version historical corpus and all five PR smoke versions pass. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(schema): close consent-gate review findings (bd-qhn8y) Review fixes for the migration-consent gate (adopt-pr review, attempts 1 and 2). Majors: - #4566 deadlock rebuilt one check earlier: on a database that is both behind and dirty, the consent refusal fired before the dirty-table guard, whose remedy (`bd dolt commit`) opens without consent. The working-set-reconcile opens now tolerate MigrateConsentError: embedded (toleratesConsentRefusal, WSR intent only) and server mode (LenientOpen's warnLenientOpenRefusal) print WorkingSetWarning and commit at the current schema; strict, read-only and RemoteSync opens still refuse. Covered by an embedded test (a refused open writes nothing: dolt_status, dolt_ignore and schema_migrations unchanged) and a server-mode integration subtest. - Bare `bd migrate` consented: isMigrateConsentCommand now matches only the schema verb, so `bd migrate` (metadata reconcile, including --update-repo-id, --inspect and --dry-run) no longer consents to a one-way migration; `bd migrate schema` and --force do. TestLocalMigrateConsentScope pins the scope through the root pre-run. The dirty-table warnings name `bd migrate schema` accordingly. - Runnable JSON hint: the JSON refusal follows the #4259 AgentDirective convention, a non-runnable hint plus a migrate_consent object (versions, pending, severity, human_decision_required, options) whose commands are retargeted with --global for the shared global database. The text refusal prints the same --global hint under `bd --global`. - Freeze floor: the hash-pinned freeze now covers what v1.3.x shipped, main 0001-0066 and ignored 0001-0026; the CHANGELOG puts new migrations in 0070+ / 0028+. - Fresh-bootstrap heal: MigrateUpWithLock with heal authority takes the consent-free migrate path, so a retried bd init whose first pass left a non-zero cursor converges; other lock callers keep the gate. Minors: the uow team-server guards are repinned on `bd migrate schema`; the un-migrate runbook is linked from the recovery index and cross-linked with the 1.2.1 repair-forward doc; the CHANGELOG BREAKING entry matches the shipped contract; harness comment grammar. Docs: the user docs and README named bare `bd migrate` as the step that applies schema migrations; they now say `bd migrate schema`, which also exists in the pinned docs release (v1.2.2) and in v1.3.x. The `bd migrate schema` help no longer claims migrations run automatically on store open. The Go remedy that sends a remote-gate stop to bare `bd migrate` for its migrate-or-adopt options is unchanged (pinned by TestNoticeSharedMigrateRefusalPerDecision). Upgrade smoke (attempt 2): the consent-gate assertion is graded only when the old binary left a database. Releases before v1.0 have no `init --non-interactive`, so prev_init creates nothing there and the candidate's `bd list` fails with "no beads database found", which has no consent guidance. Main's Bazel smoke lane (#7331) runs the newest 30 releases, 20 of them pre-1.0, under the required ci-gate; all 20 went red on the merge. The 10 v1.x releases still take the refusal branch. Shard manifest (pre-approval CI): the new embedded test TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers is listed in the Bazel 15-shard block of embedded-storage-test-shards.txt (incremental --write; shard 15). Main's ci lane runs TestStorageEmbeddedShardManifestGeneratorNotStale in //scripts:scripts_test, which fails on any unlisted top-level embeddeddolt test. Not addressed (optional): env-only consent docs for embedders, the unparseable BD_ALLOW_REMOTE_MIGRATE alias hint, and the harness `|| true` on `cand migrate schema`. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com> Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
This commit is contained in:
parent
75d9dabf83
commit
70f9e2f8a3
40 changed files with 1710 additions and 71 deletions
|
|
@ -529,6 +529,7 @@
|
|||
15 15 TestEmbeddedDependencyEditorRefusesSelfDependencyWithTheProbeSkipped
|
||||
15 15 TestEmbeddedDependencyEditorRepeatsWithinOneRequestCollapse
|
||||
15 15 TestEmbeddedIssueOperationsUpdateLabelPatchOrdering
|
||||
15 15 TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers
|
||||
15 15 TestEmbeddedMigrateSelfHealsAfterMidPassFault_4566
|
||||
15 15 TestEmbeddedMoveIssuePersistenceEphemeralNoHistorySmoke
|
||||
15 15 TestEmbeddedOpenForRemoteSync_DataBehind
|
||||
|
|
|
|||
40
CHANGELOG.md
40
CHANGELOG.md
|
|
@ -131,6 +131,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
`scripts/check-versions.sh` and `bd preflight` list the three that work: with
|
||||
`cmd/bd/version.go` already at the release version, that re-run rewrites only
|
||||
the `.githooks` markers and `uv.lock` and leaves any other drifted file as it was.
|
||||
- **BREAKING (fail-closed): bd no longer migrates an existing database without
|
||||
explicit consent** (Beads 1.2 release remediation). The 1.2.0/1.2.1 releases
|
||||
applied migrations 54–65 at store open on ANY command — including pure reads
|
||||
like `bd list` — with no prompt, and older binaries then refuse the migrated
|
||||
database (schema-skew guard). A command that would apply pending
|
||||
main-sequence migrations to an existing database at store open (in embedded
|
||||
mode that includes reads) now refuses with guidance instead, and the refused
|
||||
open performs ZERO writes (dolt_ignore seeding and clone-local migrations
|
||||
included). Server-mode read commands open without running migrations, so
|
||||
they keep reading at the current schema. `bd dolt commit` and `bd vc commit`
|
||||
warn and commit at the current schema without migrating, so the dirty-table
|
||||
recovery (commit, then `bd migrate schema`) still works on a database that
|
||||
is both behind and dirty. Consent is any of: `bd migrate schema`, the verb
|
||||
that names the migration (a preview flag withholds it); `--force` on
|
||||
`bd migrate` or `bd migrate schema`; or `BD_ALLOW_MIGRATE=1` for scripts/CI
|
||||
(`BD_ALLOW_REMOTE_MIGRATE=1` is honored as an alias). Bare `bd migrate`,
|
||||
including `--update-repo-id`, `--inspect` and `--dry-run`, does not consent.
|
||||
Fresh databases (`bd init`) still migrate to latest — creating a database is
|
||||
consent for its schema, kept across the retry of an interrupted server
|
||||
bootstrap — and the same creation principle covers a workspace the
|
||||
invocation just cloned into existence (`bd init --remote`, `bd bootstrap`
|
||||
sync), so the #4259 behind-remote flows keep their exact behavior. A
|
||||
database already at this binary's latest version never consults the gate;
|
||||
one left at v65 by 1.2.x or at v66 by 1.3.x still has migrations pending
|
||||
under this release, so it asks once. The remote-backed case keeps all of the
|
||||
#4259 remote-migrate gate's additional protections (smart gate, adopt
|
||||
fast-forward, fork-skew analysis) unchanged. JSON consumers: the refusal
|
||||
renders as a structured error whose `error` states the refusal and whose
|
||||
`hint` is a non-runnable operator directive (neither names a migration
|
||||
command or a consent variable), plus a `migrate_consent` object
|
||||
(`current_version`, `required_version`, `pending`, `severity: "blocking"`,
|
||||
`human_decision_required: true`, and `options`: `migrate` runs
|
||||
`bd migrate schema`, or `bd migrate schema --global` under `--global`;
|
||||
`keep` has no command).
|
||||
- **Shipped migrations are frozen by CI** (same remediation): the content hash
|
||||
of every released up-migration (main 0001–0066, ignored 0001–0026, through
|
||||
the 1.3.x releases) is pinned by `TestShippedMigrationsAreFrozen`. Any
|
||||
amendment to a released migration must land as a new migration (0070+, or
|
||||
ignored 0028+, as of this change) so installed schemas and freshly-migrated
|
||||
ones cannot fork.
|
||||
|
||||
### Fixed
|
||||
- **PRs based on `hotfix/**` branches now run full CI, not just
|
||||
|
|
|
|||
|
|
@ -109,8 +109,8 @@ version: sync remote-backed databases with your current `bd`, back up with
|
|||
`bd export --all`, upgrade the binary, then run `bd info --whats-new`,
|
||||
`bd hooks install`, and `bd version`. If the upgrade crosses a schema
|
||||
migration on a remote-backed database, exactly one designated clone runs
|
||||
`bd migrate` and `bd dolt push`; other clones install the new binary
|
||||
and run `bd bootstrap`. See the full
|
||||
`bd migrate schema` and `bd dolt push`; other clones install the new
|
||||
binary and run `bd bootstrap`. See the full
|
||||
[upgrade guide](https://beads.gascity.com/getting-started/upgrading)
|
||||
or [docs/getting-started/installation.md](docs/getting-started/installation.md#updating-bd).
|
||||
|
||||
|
|
|
|||
|
|
@ -213,6 +213,7 @@ go_library(
|
|||
"metadata_flag.go",
|
||||
"metrics.go",
|
||||
"migrate.go",
|
||||
"migrate_consent.go",
|
||||
"migrate_dolt_mode.go",
|
||||
"migrate_hooks.go",
|
||||
"migrate_hooks_apply.go",
|
||||
|
|
@ -935,6 +936,7 @@ go_test(
|
|||
"metadata_ready_test.go",
|
||||
"metadataless_server_workspace_test.go",
|
||||
"metrics_test.go",
|
||||
"migrate_consent_test.go",
|
||||
"migrate_dolt_mode_frontdoor_integration_test.go",
|
||||
"migrate_dolt_mode_test.go",
|
||||
"migrate_embedded_test.go",
|
||||
|
|
|
|||
|
|
@ -1246,6 +1246,15 @@ func executeSyncAction(ctx context.Context, plan BootstrapPlan, cfg *configfile.
|
|||
return err
|
||||
}
|
||||
|
||||
// Creation-consent for the migration-consent gate
|
||||
// (schema/migrate_consent.go): this sync action just CLONED the database
|
||||
// fresh, so the operator's explicit bootstrap is consent for its schema —
|
||||
// the same principle that exempts a fresh `bd init`. The #4259
|
||||
// remote-migrate gate below is unaffected (consent is not its
|
||||
// designated-migrator override), so the behind-remote refusal keeps its
|
||||
// exact bd-6dnrw.31 behavior.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
|
||||
// Open and close the store to ensure dolt_ignore'd wisp tables are
|
||||
// created in the working set. Clone does not include these tables
|
||||
// (they are never committed), so they must be recreated after clone.
|
||||
|
|
|
|||
|
|
@ -1350,6 +1350,15 @@ Non-interactive mode (--non-interactive or BD_NON_INTERACTIVE=1):
|
|||
syncFromRemote = false
|
||||
} else {
|
||||
bootstrappedFromRemote = true
|
||||
// Creation-consent for the migration-consent gate
|
||||
// (schema/migrate_consent.go): this invocation just CLONED the
|
||||
// database into a workspace that did not exist before, so the
|
||||
// operator's `bd init --remote` is consent for its schema —
|
||||
// the same principle that exempts a fresh `bd init`. The
|
||||
// #4259 remote-migrate gate is unaffected (consent is not its
|
||||
// designated-migrator override), so the smart-gate /
|
||||
// designated-migrator flows keep their exact behavior.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
if !quiet {
|
||||
fmt.Printf(" %s Bootstrapped from remote: %s\n", ui.RenderPass("✓"), syncURL)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -368,7 +368,8 @@ func isForcedMigrate(cmd *cobra.Command) bool {
|
|||
// schema.SharedConsentCommandForced), so retargeting the bare form would hand
|
||||
// the operator a global-scoped command that still cannot succeed. This mirrors
|
||||
// the retarget in handleRemoteMigrateGateJSON. A nil error keeps the
|
||||
// pre-existing bare-verb wording.
|
||||
// pre-existing bare-verb wording, which is also the verb the
|
||||
// migration-consent refusal prescribes, so that refusal passes nil.
|
||||
func printGlobalDatabaseConsentHint(w io.Writer, e *schema.RemoteMigrateGateError) {
|
||||
if !globalFlag {
|
||||
return
|
||||
|
|
@ -415,13 +416,28 @@ func renderTypedOpenError(err error) bool {
|
|||
}
|
||||
return true
|
||||
}
|
||||
// The migration-consent gate blocks silent in-place migration of
|
||||
// ANY existing database (1.2 release remediation) and tells the
|
||||
// operator how to consent.
|
||||
var consentErr *schema.MigrateConsentError
|
||||
if errors.As(err, &consentErr) {
|
||||
if jsonOutput {
|
||||
handleMigrateConsentJSON(consentErr)
|
||||
} else {
|
||||
fmt.Fprint(os.Stderr, consentErr.UserMessage())
|
||||
printGlobalDatabaseConsentHint(os.Stderr, nil)
|
||||
}
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// isSchemaMigrateVerb reports whether cmd is `bd migrate schema` — the one
|
||||
// invocation in which the operator asked for a schema migration by name. That
|
||||
// request is the consent the shared-store gate wants for a database with no
|
||||
// remote (#5920); see schema.SetSharedMigrateConsent.
|
||||
// remote (#5920), and the consent the migration-consent gate wants for any
|
||||
// existing database; see schema.SetSharedMigrateConsent and
|
||||
// schema.SetLocalMigrateConsent.
|
||||
//
|
||||
// Deliberately just this one command, not the `bd migrate` tree. Bare
|
||||
// `bd migrate` reconciles version/repo-id/clone-id metadata and never applies
|
||||
|
|
@ -436,6 +452,17 @@ func isSchemaMigrateVerb(cmd *cobra.Command) bool {
|
|||
return cmd == migrateSchemaCmd
|
||||
}
|
||||
|
||||
// isMigrateConsentCommand reports whether invoking cmd is consent to apply
|
||||
// pending schema migrations to an existing database (migrate_consent.go).
|
||||
// That is `bd migrate schema` alone, the verb that names the migration: bare
|
||||
// `bd migrate` does not consent, for the reasons isSchemaMigrateVerb gives,
|
||||
// and --force on either migrate command consents separately, through the gate
|
||||
// override. A preview withholds the consent, as it does the shared-store one:
|
||||
// a --dry-run/--inspect must not consent to the work it only inspects.
|
||||
func isMigrateConsentCommand(cmd *cobra.Command) bool {
|
||||
return isSchemaMigrateVerb(cmd) && !isPreviewCommand(cmd)
|
||||
}
|
||||
|
||||
// forcedMigratePreviewFlag returns the name of a preview flag (--dry-run,
|
||||
// --inspect) that conflicts with --force on a forced migrate invocation, or ""
|
||||
// when there is no conflict. The combination must be rejected BEFORE the store
|
||||
|
|
@ -1751,6 +1778,12 @@ var rootCmd = &cobra.Command{
|
|||
// discipline as the --force override above.
|
||||
schema.SetSharedMigrateConsent(isSchemaMigrateVerb(cmd) && !previewMode)
|
||||
|
||||
// The same verb carries migration consent on its own
|
||||
// (migrate_consent.go): every other command refuses to apply pending
|
||||
// schema migrations to an existing database. Same unconditional
|
||||
// set-or-clear discipline as the gate override above.
|
||||
schema.SetLocalMigrateConsent(isMigrateConsentCommand(cmd))
|
||||
|
||||
// Auto-migrate database on version bump (bd-jgxi).
|
||||
// Runs for ALL non-preview commands (including read-only ones) because
|
||||
// the migration opens its own store connection, writes the version
|
||||
|
|
|
|||
|
|
@ -880,9 +880,15 @@ var migrateSchemaCmd = &cobra.Command{
|
|||
Short: "Apply pending schema migrations (idempotent)",
|
||||
Long: `Apply pending schema migrations idempotently.
|
||||
|
||||
Schema migrations also run automatically on store open, so this subcommand
|
||||
is typically a no-op. It exists to make migration explicit and observable
|
||||
in CI, release gates, and recovery scenarios.
|
||||
bd does not apply pending schema migrations to an existing database without
|
||||
the operator's consent, and running this command is that consent
|
||||
(BD_ALLOW_MIGRATE=1 is the scripted form). On a remote-backed database the
|
||||
remote-migrate gate still applies: --force confirms you are the single
|
||||
designated migrator. Migrating is one-way: bd releases built for an older
|
||||
schema refuse the migrated database.
|
||||
|
||||
On an up-to-date database this is a no-op, so it is safe to run in CI,
|
||||
release gates, and recovery scenarios.
|
||||
|
||||
Example:
|
||||
bd migrate schema
|
||||
|
|
|
|||
56
cmd/bd/migrate_consent.go
Normal file
56
cmd/bd/migrate_consent.go
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
)
|
||||
|
||||
// handleMigrateConsentJSON renders the migration-consent refusal as a
|
||||
// structured JSON error block for agent consumption.
|
||||
//
|
||||
// It follows handleRemoteMigrateGateJSON's convention, for the same reason:
|
||||
// migrating is one-way, so neither "error" nor the top-level "hint" carries a
|
||||
// runnable remedy. "error" states the refusal, "hint" is the non-runnable
|
||||
// operator-decision directive, and the migrate command lives only inside
|
||||
// migrate_consent.options[migrate], gated on its precondition and annotated
|
||||
// with its risk. Handing an agent `bd migrate schema` (or the consent
|
||||
// environment variable) as "the fix" lets it migrate a database that an older
|
||||
// bd still has to open; the env var stays out of this payload entirely.
|
||||
func handleMigrateConsentJSON(e *schema.MigrateConsentError) {
|
||||
outer := buildJSONError(e.Refusal(), e.AgentDirective())
|
||||
if m, ok := outer.(map[string]interface{}); ok {
|
||||
opts := make([]map[string]interface{}, 0, len(e.Options()))
|
||||
for _, o := range e.Options() {
|
||||
commands := make([]string, len(o.Commands))
|
||||
for i, c := range o.Commands {
|
||||
// Under --global the refused open aimed at beads_global, so the
|
||||
// project-scoped verb would migrate the WRONG database and leave
|
||||
// the refusal in place. Same retarget as the text path
|
||||
// (printGlobalDatabaseConsentHint) and the shared-store gate.
|
||||
if globalFlag && c == schema.SharedConsentCommand {
|
||||
c = schema.SharedConsentCommandGlobal
|
||||
}
|
||||
commands[i] = c
|
||||
}
|
||||
opts = append(opts, map[string]interface{}{
|
||||
"id": o.ID,
|
||||
"when": o.When,
|
||||
"commands": commands,
|
||||
"risk": o.Risk,
|
||||
})
|
||||
}
|
||||
m["migrate_consent"] = map[string]interface{}{
|
||||
"current_version": e.CurrentVersion,
|
||||
"required_version": e.LatestVersion,
|
||||
"pending": e.Pending,
|
||||
"severity": "blocking",
|
||||
"human_decision_required": true,
|
||||
"options": opts,
|
||||
}
|
||||
}
|
||||
encoder := json.NewEncoder(os.Stderr)
|
||||
encoder.SetIndent("", " ")
|
||||
_ = encoder.Encode(outer)
|
||||
}
|
||||
297
cmd/bd/migrate_consent_test.go
Normal file
297
cmd/bd/migrate_consent_test.go
Normal file
|
|
@ -0,0 +1,297 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
sqlmock "github.com/DATA-DOG/go-sqlmock"
|
||||
"github.com/spf13/pflag"
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
)
|
||||
|
||||
// renderMigrateConsentJSON routes a wrapped consent refusal through
|
||||
// renderTypedOpenError in --json mode, the way a refused store open reaches
|
||||
// it, and returns the decoded payload and the raw text.
|
||||
func renderMigrateConsentJSON(t *testing.T, refusal *schema.MigrateConsentError) (map[string]interface{}, string) {
|
||||
t.Helper()
|
||||
pinJSONOutput(t, true)
|
||||
t.Setenv("BD_JSON_ENVELOPE", "")
|
||||
|
||||
var handled bool
|
||||
out := captureNoticeStderr(t, func() {
|
||||
handled = renderTypedOpenError(fmt.Errorf("failed to open database: %w", refusal))
|
||||
})
|
||||
if !handled {
|
||||
t.Fatal("renderTypedOpenError did not handle a wrapped MigrateConsentError")
|
||||
}
|
||||
var parsed map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(out), &parsed); err != nil {
|
||||
t.Fatalf("json.Unmarshal stderr: %v\nstderr was: %s", err, out)
|
||||
}
|
||||
return parsed, out
|
||||
}
|
||||
|
||||
// migrateConsentOptions returns migrate_consent.options keyed by id.
|
||||
func migrateConsentOptions(t *testing.T, parsed map[string]interface{}) map[string]map[string]interface{} {
|
||||
t.Helper()
|
||||
obj, ok := parsed["migrate_consent"].(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("migrate_consent key missing or wrong type: %T", parsed["migrate_consent"])
|
||||
}
|
||||
rawOpts, ok := obj["options"].([]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("migrate_consent.options missing or wrong type: %T", obj["options"])
|
||||
}
|
||||
byID := map[string]map[string]interface{}{}
|
||||
for _, raw := range rawOpts {
|
||||
o, ok := raw.(map[string]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("option has wrong type: %T", raw)
|
||||
}
|
||||
id, _ := o["id"].(string)
|
||||
byID[id] = o
|
||||
}
|
||||
return byID
|
||||
}
|
||||
|
||||
func optionCommands(t *testing.T, o map[string]interface{}) []string {
|
||||
t.Helper()
|
||||
raw, ok := o["commands"].([]interface{})
|
||||
if !ok {
|
||||
t.Fatalf("option %v: commands missing or not an array: %T", o["id"], o["commands"])
|
||||
}
|
||||
commands := make([]string, 0, len(raw))
|
||||
for _, c := range raw {
|
||||
s, _ := c.(string)
|
||||
commands = append(commands, s)
|
||||
}
|
||||
return commands
|
||||
}
|
||||
|
||||
// TestHandleMigrateConsentJSON_Shape pins the agent-facing refusal to the
|
||||
// remote-migrate gate's convention (#4259). Migrating is one-way, so the
|
||||
// payload must not hand an agent a runnable migration as "the fix": the error
|
||||
// names the refusal, the hint is the non-runnable directive, and the migrate
|
||||
// command appears exactly once, inside the operator-gated migrate option. The
|
||||
// consent environment variable must not appear at all.
|
||||
func TestHandleMigrateConsentJSON_Shape(t *testing.T) {
|
||||
orig := globalFlag
|
||||
globalFlag = false
|
||||
t.Cleanup(func() { globalFlag = orig })
|
||||
|
||||
refusal := &schema.MigrateConsentError{CurrentVersion: 65, LatestVersion: 69, Pending: 4}
|
||||
parsed, raw := renderMigrateConsentJSON(t, refusal)
|
||||
|
||||
if got := parsed["error"]; got != refusal.Refusal() {
|
||||
t.Errorf("error = %v, want the remedy-free refusal %q", got, refusal.Refusal())
|
||||
}
|
||||
if got := parsed["hint"]; got != refusal.AgentDirective() {
|
||||
t.Errorf("hint = %v, want the directive %q", got, refusal.AgentDirective())
|
||||
}
|
||||
for _, field := range []string{"error", "hint"} {
|
||||
s, _ := parsed[field].(string)
|
||||
if strings.Contains(s, schema.SharedConsentCommand) {
|
||||
t.Errorf("%s must not carry the runnable migration %q: %q", field, schema.SharedConsentCommand, s)
|
||||
}
|
||||
}
|
||||
if strings.Contains(raw, "BD_ALLOW") {
|
||||
t.Errorf("the payload must not name a consent environment variable:\n%s", raw)
|
||||
}
|
||||
if n := strings.Count(raw, schema.SharedConsentCommand); n != 1 {
|
||||
t.Errorf("%q appears %d times, want exactly once (inside options[migrate]):\n%s",
|
||||
schema.SharedConsentCommand, n, raw)
|
||||
}
|
||||
|
||||
obj := parsed["migrate_consent"].(map[string]interface{})
|
||||
for key, want := range map[string]float64{"current_version": 65, "required_version": 69, "pending": 4} {
|
||||
if got, ok := obj[key].(float64); !ok || got != want {
|
||||
t.Errorf("%s = %v, want %v", key, obj[key], want)
|
||||
}
|
||||
}
|
||||
if got := obj["severity"]; got != "blocking" {
|
||||
t.Errorf("severity = %v, want \"blocking\"", got)
|
||||
}
|
||||
if got := obj["human_decision_required"]; got != true {
|
||||
t.Errorf("human_decision_required = %v, want true", got)
|
||||
}
|
||||
|
||||
opts := migrateConsentOptions(t, parsed)
|
||||
if len(opts) != 2 || opts["migrate"] == nil || opts["keep"] == nil {
|
||||
t.Fatalf("options = %v, want exactly migrate and keep", opts)
|
||||
}
|
||||
if got := optionCommands(t, opts["migrate"]); len(got) != 1 || got[0] != schema.SharedConsentCommand {
|
||||
t.Errorf("options[migrate].commands = %v, want [%q]", got, schema.SharedConsentCommand)
|
||||
}
|
||||
if got := optionCommands(t, opts["keep"]); len(got) != 0 {
|
||||
t.Errorf("options[keep].commands = %v, want none", got)
|
||||
}
|
||||
if risk, _ := opts["migrate"]["risk"].(string); !strings.Contains(risk, "one-way") {
|
||||
t.Errorf("options[migrate].risk must name the one-way risk: %q", risk)
|
||||
}
|
||||
for id, o := range opts {
|
||||
if when, _ := o["when"].(string); when == "" {
|
||||
t.Errorf("options[%s].when is empty; every option is gated on its precondition", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestHandleMigrateConsentJSON_GlobalRetarget pins the --global retarget: the
|
||||
// refused open aimed at beads_global, so the project-scoped verb would migrate
|
||||
// the wrong database and leave the refusal in place. The command lives only
|
||||
// inside the option, so that is where the retarget has to land.
|
||||
func TestHandleMigrateConsentJSON_GlobalRetarget(t *testing.T) {
|
||||
orig := globalFlag
|
||||
globalFlag = true
|
||||
t.Cleanup(func() { globalFlag = orig })
|
||||
|
||||
parsed, raw := renderMigrateConsentJSON(t, &schema.MigrateConsentError{CurrentVersion: 65, LatestVersion: 69, Pending: 4})
|
||||
|
||||
opts := migrateConsentOptions(t, parsed)
|
||||
if got := optionCommands(t, opts["migrate"]); len(got) != 1 || got[0] != schema.SharedConsentCommandGlobal {
|
||||
t.Errorf("options[migrate].commands = %v, want [%q]", got, schema.SharedConsentCommandGlobal)
|
||||
}
|
||||
if strings.Contains(raw, `"`+schema.SharedConsentCommand+`"`) {
|
||||
t.Errorf("--global payload still names the project-scoped %q:\n%s", schema.SharedConsentCommand, raw)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRenderTypedOpenErrorMigrateConsentText covers the human path: the full
|
||||
// UserMessage, plus the --global verb when the invocation targeted the global
|
||||
// database, since the message itself cannot know which database was refused.
|
||||
func TestRenderTypedOpenErrorMigrateConsentText(t *testing.T) {
|
||||
pinJSONOutput(t, false)
|
||||
orig := globalFlag
|
||||
t.Cleanup(func() { globalFlag = orig })
|
||||
|
||||
refusal := &schema.MigrateConsentError{CurrentVersion: 65, LatestVersion: 69, Pending: 4}
|
||||
wrapped := fmt.Errorf("failed to open database: %w", refusal)
|
||||
|
||||
render := func() string {
|
||||
t.Helper()
|
||||
var handled bool
|
||||
out := captureNoticeStderr(t, func() { handled = renderTypedOpenError(wrapped) })
|
||||
if !handled {
|
||||
t.Fatal("renderTypedOpenError did not handle a wrapped MigrateConsentError")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
globalFlag = false
|
||||
if out := render(); out != refusal.UserMessage() {
|
||||
t.Errorf("without --global the refusal is the UserMessage alone, got:\n%s", out)
|
||||
}
|
||||
|
||||
globalFlag = true
|
||||
out := render()
|
||||
if !strings.HasPrefix(out, refusal.UserMessage()) {
|
||||
t.Errorf("under --global the UserMessage must still lead, got:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, schema.SharedConsentCommandGlobal) {
|
||||
t.Errorf("under --global the refusal must name %q, got:\n%s", schema.SharedConsentCommandGlobal, out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLocalMigrateConsentScope pins which command lines consent to migrating
|
||||
// an existing database. Only the verb that names the migration does, plus
|
||||
// --force on either migrate command. Bare `bd migrate` reconciles metadata and
|
||||
// never needed the gate open, so letting it consent made
|
||||
// `bd migrate --update-repo-id` (repo-fingerprint surgery) a one-way schema
|
||||
// migration as a side effect.
|
||||
//
|
||||
// Each line is resolved through the same cobra lookup and fed to the same
|
||||
// predicates as the root pre-run (isForcedMigrate, isMigrateConsentCommand),
|
||||
// then the real gate runs against an existing database one migration behind.
|
||||
func TestLocalMigrateConsentScope(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
cmdline string
|
||||
consents bool
|
||||
}{
|
||||
{cmdline: "bd migrate schema", consents: true},
|
||||
{cmdline: "bd migrate schema --force", consents: true},
|
||||
{cmdline: "bd migrate --force", consents: true},
|
||||
{cmdline: "bd migrate", consents: false},
|
||||
{cmdline: "bd migrate --update-repo-id", consents: false},
|
||||
{cmdline: "bd migrate --inspect", consents: false},
|
||||
{cmdline: "bd migrate --dry-run", consents: false},
|
||||
{cmdline: "bd list", consents: false},
|
||||
} {
|
||||
t.Run(tt.cmdline, func(t *testing.T) {
|
||||
err := runLocalConsentGate(t, tt.cmdline, tt.consents)
|
||||
if tt.consents {
|
||||
// Programmatic consent short-circuits the gate before its version
|
||||
// reads, and those reads would fail here with "migrate consent:"
|
||||
// (the mock expects nothing). Whatever MigrateUp hit past the gate
|
||||
// is not this test's concern.
|
||||
if schema.IsMigrateConsentError(err) || (err != nil && strings.Contains(err.Error(), "migrate consent:")) {
|
||||
t.Fatalf("%q must consent, but the gate ran: %v", tt.cmdline, err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !schema.IsMigrateConsentError(err) {
|
||||
t.Fatalf("%q must not consent; want the consent refusal, got %T: %v", tt.cmdline, err, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// runLocalConsentGate replays the root pre-run's two consent lines for
|
||||
// cmdline and runs MigrateUp. A consenting line must not reach the gate's
|
||||
// reads, so its mock expects nothing; a refusing one gets the gate's reads
|
||||
// for an existing database one migration behind.
|
||||
func runLocalConsentGate(t *testing.T, cmdline string, consents bool) error {
|
||||
t.Helper()
|
||||
t.Setenv(schema.AllowMigrateEnv, "")
|
||||
t.Setenv(schema.AllowRemoteMigrateEnv, "")
|
||||
|
||||
args := strings.Fields(strings.TrimPrefix(cmdline, "bd "))
|
||||
target, rest, err := rootCmd.Find(args)
|
||||
if err != nil {
|
||||
t.Fatalf("%q does not resolve to a command: %v", cmdline, err)
|
||||
}
|
||||
if err := target.ParseFlags(rest); err != nil {
|
||||
t.Fatalf("%q: flags do not parse: %v", cmdline, err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
// Package-level cobra singletons: a parsed flag would outlive this
|
||||
// test and silently consent for the next one.
|
||||
target.Flags().VisitAll(func(f *pflag.Flag) {
|
||||
if !f.Changed {
|
||||
return
|
||||
}
|
||||
_ = f.Value.Set(f.DefValue)
|
||||
f.Changed = false
|
||||
})
|
||||
schema.SetForceAllowRemoteMigrate(false)
|
||||
schema.SetLocalMigrateConsent(false)
|
||||
})
|
||||
|
||||
// The two lines the root pre-run runs, verbatim in effect.
|
||||
schema.SetForceAllowRemoteMigrate(isForcedMigrate(target))
|
||||
schema.SetLocalMigrateConsent(isMigrateConsentCommand(target))
|
||||
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
if !consents {
|
||||
for range 2 {
|
||||
mock.ExpectQuery(`SELECT COUNT\(\*\) FROM information_schema\.tables`).
|
||||
WithArgs("schema_migrations").
|
||||
WillReturnRows(sqlmock.NewRows([]string{"count"}).AddRow(1))
|
||||
mock.ExpectQuery(`SELECT COALESCE\(MAX\(version\), 0\) FROM schema_migrations`).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"version"}).AddRow(schema.LatestVersion() - 1))
|
||||
}
|
||||
}
|
||||
|
||||
_, gateErr := schema.MigrateUp(context.Background(), db)
|
||||
if !consents {
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatalf("unmet expectations for %q: %v", cmdline, err)
|
||||
}
|
||||
}
|
||||
return gateErr
|
||||
}
|
||||
|
|
@ -102,7 +102,8 @@ func newDoltStore(ctx context.Context, cfg *dolt.Config) (s storage.DoltStorage,
|
|||
// Working-set-reconcile commands (bd dolt commit, bd vc commit) must
|
||||
// not be bricked by a pending-migration dirty-table refusal: that
|
||||
// refusal's documented recovery is exactly the commit these commands
|
||||
// run, so failing the open here would deadlock (#4566). The server
|
||||
// run, so failing the open here would deadlock (#4566). The same goes
|
||||
// for the migration-consent refusal that precedes it. The server
|
||||
// arm above honors the same cfg.LenientOpen inside dolt.New; this
|
||||
// branch is the embedded half of one policy, not the whole of it.
|
||||
return embeddeddolt.OpenForWorkingSetReconcile(ctx, cfg.BeadsDir, cfg.Database, "main")
|
||||
|
|
|
|||
|
|
@ -276,6 +276,7 @@
|
|||
"recovery/circular-dependencies",
|
||||
"recovery/sync-failures",
|
||||
"recovery/history-squash",
|
||||
"recovery/unmigrate-schema-v65",
|
||||
"recovery/uninstalling"
|
||||
]
|
||||
},
|
||||
|
|
|
|||
|
|
@ -448,7 +448,7 @@ Upgrade checklist:
|
|||
`bd version`
|
||||
5. If crossing a schema migration on a remote-backed database, only the
|
||||
designated migrator runs:
|
||||
`bd migrate`
|
||||
`bd migrate schema`
|
||||
`bd dolt push`
|
||||
|
||||
Other clones should install the new binary and run `bd bootstrap`, not
|
||||
|
|
|
|||
|
|
@ -312,7 +312,7 @@ In server mode, data is managed by the external `dolt sql-server`.
|
|||
|
||||
## Migrating databases
|
||||
|
||||
After upgrading bd, use `bd migrate` to check for and migrate old database files:
|
||||
After upgrading bd, check for pending database migrations and apply them:
|
||||
|
||||
```bash
|
||||
# Inspect migration plan (AI agents)
|
||||
|
|
@ -324,8 +324,8 @@ bd info --schema --json
|
|||
# Preview migration changes
|
||||
bd migrate --dry-run
|
||||
|
||||
# Migrate old databases to beads.db
|
||||
bd migrate
|
||||
# Apply pending schema migrations
|
||||
bd migrate schema
|
||||
|
||||
# Migrate and clean up old files
|
||||
bd migrate --yes
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ bd info --whats-new
|
|||
`bd version`
|
||||
5. If crossing a schema migration on a remote-backed database, only the
|
||||
designated migrator runs:
|
||||
`bd migrate`
|
||||
`bd migrate schema`
|
||||
`bd dolt push`
|
||||
|
||||
Other clones must not migrate independently. After the migrator pushes, each
|
||||
|
|
@ -139,7 +139,7 @@ bd migrate --inspect --json
|
|||
bd migrate --dry-run
|
||||
|
||||
# Apply migrations
|
||||
bd migrate
|
||||
bd migrate schema
|
||||
|
||||
# Migrate and clean up old files
|
||||
bd migrate --yes
|
||||
|
|
@ -307,7 +307,7 @@ also copy the `.beads` directory (or `dolt backup` in server mode) while no
|
|||
bd dolt push # 1. CURRENT binary: publish all local work
|
||||
bd export --all -o .beads/backup/pre-migrate.jsonl # 2. backup (see above)
|
||||
# 3. install the new binary (see Upgrading above)
|
||||
bd migrate # 4. migrate as the designated migrator
|
||||
bd migrate schema # 4. migrate as the designated migrator
|
||||
bd dolt push # 5. publish the migrated schema
|
||||
bd version # 6. confirm the new version is active
|
||||
```
|
||||
|
|
@ -317,10 +317,10 @@ migrating here as the designated migrator, adopting the remote's already-migrate
|
|||
database, or recovering a fork — and asks for an explicit operator decision.
|
||||
Follow the guidance it prints.
|
||||
|
||||
For scripted or CI upgrades where nobody reads the output, run `bd migrate` as
|
||||
an explicit step in exactly one job, never in all of them. If the gate blocks a
|
||||
run it prints both the available options and the scripted override that fits
|
||||
the situation.
|
||||
For scripted or CI upgrades where nobody reads the output, run
|
||||
`bd migrate schema` as an explicit step in exactly one job, never in all of
|
||||
them. If the gate blocks a run it prints both the available options and the
|
||||
scripted override that fits the situation.
|
||||
|
||||
**Multiple clones sharing one remote:**
|
||||
|
||||
|
|
@ -332,7 +332,7 @@ bd dolt pull
|
|||
bd export --all -o .beads/backup/pre-migrate.jsonl
|
||||
|
||||
# 2. Designated migrator ONLY: install the new binary, then migrate and publish.
|
||||
bd migrate
|
||||
bd migrate schema
|
||||
bd dolt push
|
||||
|
||||
# 3. Every OTHER clone, after the migrator has pushed: pull the migrated
|
||||
|
|
@ -538,9 +538,11 @@ bridge below accepts SQLite sources only.
|
|||
|
||||
### `.beads/embeddeddolt/`: direct upgrade
|
||||
|
||||
Upgrade the binary and run:
|
||||
Upgrade the binary, apply the pending schema migrations, and update the
|
||||
database metadata:
|
||||
|
||||
```bash
|
||||
bd migrate schema
|
||||
bd migrate
|
||||
```
|
||||
|
||||
|
|
@ -589,8 +591,8 @@ metadata is missing or leaves `dolt_mode` blank/`embedded`; it does not override
|
|||
an explicit server selection with a local root. Compatibility admission cannot
|
||||
prove that a workspace is modern. If you know it was created by v0.55.4 through
|
||||
v0.62.0, use this explicit bridge even when its witness was lost or damaged.
|
||||
Otherwise, follow the normal `bd migrate --dry-run` and `bd migrate` flow for
|
||||
an admitted server workspace.
|
||||
Otherwise, follow the normal `bd migrate --dry-run` and `bd migrate schema`
|
||||
flow for an admitted server workspace.
|
||||
|
||||
### One `.beads/*.db` file: sealed SQLite bridge
|
||||
|
||||
|
|
@ -653,7 +655,7 @@ bd hooks install
|
|||
|
||||
```bash
|
||||
bd migrate --dry-run
|
||||
bd migrate
|
||||
bd migrate schema
|
||||
```
|
||||
|
||||
### Recovery after upgrade
|
||||
|
|
|
|||
|
|
@ -118,3 +118,7 @@ database directory, install v1.2.2, then `bd import backup.jsonl`.
|
|||
Caveats: issues deleted after the upgrade come back (import cannot
|
||||
re-delete), and audit events recorded while on v1.2.1 are lost. Most users
|
||||
should prefer the cursor rollback above.
|
||||
|
||||
To rebuild a 1.1.2-compatible workspace from a full export instead, keeping
|
||||
the migrated original aside, see
|
||||
[Un-migrate from Schema v65](/recovery/unmigrate-schema-v65).
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ This section provides step-by-step recovery procedures for common Beads issues.
|
|||
| Issue | Symptoms | Runbook |
|
||||
|-------|----------|---------|
|
||||
| Schema Version Mismatch | `bd` refuses with `schema version mismatch: database is at vNN, binary knows up to vNN` | [Accidental v1.2.1 Release](/recovery/accidental-1-2-1-release) |
|
||||
| Going Back from v65 | A database bd 1.2.x migrated to v65 has to work with bd 1.1.2 again, rebuilt from a full export | [Un-migrate from Schema v65](/recovery/unmigrate-schema-v65) |
|
||||
| Init Safety Refusals | `bd init` or `bd dolt` refuses with a pattern code like `pk-fork-refused` | [Recovery Playbooks](/recovery/init-safety) |
|
||||
| Database Corruption | Database errors, missing data | [Database Corruption](/recovery/database-corruption) |
|
||||
| Merge Conflicts | Dolt conflicts during sync | [Merge Conflicts](/recovery/merge-conflicts) |
|
||||
|
|
|
|||
91
docs/recovery/unmigrate-schema-v65.md
Normal file
91
docs/recovery/unmigrate-schema-v65.md
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
---
|
||||
title: Un-migrate from Schema v65
|
||||
description: Move a database that bd 1.2.x auto-migrated back to a 1.1.2-compatible workspace, keeping all data
|
||||
---
|
||||
|
||||
The 1.2.0/1.2.1 releases migrated existing databases to schema v65
|
||||
automatically, on any command — including reads. If your database was
|
||||
migrated and you want to go back to bd 1.1.2 (whose binaries refuse a v65
|
||||
database with `schema version mismatch: database is at v65, binary knows up
|
||||
to v53`), this runbook rebuilds a 1.1.2-compatible workspace from a full
|
||||
export. It is written to be executable by an agent as-is.
|
||||
|
||||
There is no in-place down-migration. The procedure is export → set aside →
|
||||
fresh init → import. If you only need the 1.1 line to open the database
|
||||
again, the cursor rollback in
|
||||
[Accidental v1.2.1 Release](/recovery/accidental-1-2-1-release) is shorter
|
||||
and keeps the Dolt history in place.
|
||||
|
||||
**What survives** (verified): issues with statuses and close reasons,
|
||||
dependencies, comments, notes, labels, memories, and wisps (with
|
||||
`--all`).
|
||||
**What does not**: Dolt commit history and the events/telemetry audit
|
||||
tables. Both remain readable in the set-aside `.beads.v65.bak` directory —
|
||||
nothing is deleted.
|
||||
|
||||
<Warning>
|
||||
Run every step with the binaries it names: the EXPORT runs under the 1.2.x
|
||||
binary that matches the database; the INIT and IMPORT run under the 1.1.2
|
||||
binary you are returning to. Mixing them up fails closed (schema-skew guard),
|
||||
but re-running the whole sequence from the top is the only supported retry.
|
||||
</Warning>
|
||||
|
||||
## Preconditions
|
||||
|
||||
1. **Stop anything that writes to this workspace** — supervisors, watchers,
|
||||
hooks on a timer.
|
||||
2. **Server mode: stop the dolt sql-server** serving this database
|
||||
(`bd dolt stop`, or stop the process that owns it). For a SHARED server
|
||||
database, coordinate with the other clients first — after this procedure
|
||||
they must re-clone or run it too; do not un-migrate a shared database
|
||||
unilaterally.
|
||||
3. Install (or locate) a **bd 1.1.2 binary** alongside the current one. GitHub
|
||||
releases: tag `v1.1.2`. Keep both paths handy; the steps name which one to
|
||||
use.
|
||||
4. Run from the directory that CONTAINS `.beads/`.
|
||||
|
||||
## Procedure
|
||||
|
||||
```bash
|
||||
# 1. Full export, with the 1.2.x binary the database currently matches.
|
||||
bd export --all -o snapshot.jsonl
|
||||
# Sanity: the file is non-empty and its line count is plausible for
|
||||
# your workspace.
|
||||
wc -l snapshot.jsonl
|
||||
|
||||
# 2. Set the migrated workspace aside. Salvage-first: rename, never delete.
|
||||
mv .beads .beads.v65.bak
|
||||
|
||||
# 3. Fresh init under the 1.1.2 binary (creates a v53-schema workspace).
|
||||
# Use the same issue prefix as before.
|
||||
bd-1.1.2 init --prefix <your-prefix>
|
||||
|
||||
# 4. Import the snapshot under the 1.1.2 binary.
|
||||
bd-1.1.2 import snapshot.jsonl
|
||||
```
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# Counts match your expectations (compare against the export):
|
||||
bd-1.1.2 list -n 0 --status all | tail -3
|
||||
# A known issue round-trips with its comments/labels/deps:
|
||||
bd-1.1.2 show <some-known-id>
|
||||
# Memories survived:
|
||||
bd-1.1.2 memories | head
|
||||
```
|
||||
|
||||
If anything is missing, the migrated original is intact in `.beads.v65.bak`:
|
||||
restore it with `mv .beads .beads.failed && mv .beads.v65.bak .beads` and use
|
||||
a 1.2.x binary with it while you retry.
|
||||
|
||||
## Afterwards
|
||||
|
||||
- Keep `.beads.v65.bak` until you are confident; it holds the Dolt history
|
||||
and audit tables the import does not carry.
|
||||
- Pin your install to 1.1.2 until a release whose default does not migrate
|
||||
your database is available; see the release notes of the version that sent
|
||||
you here.
|
||||
- If this workspace had a Dolt remote, the remote still holds the MIGRATED
|
||||
history. Do not push the rebuilt workspace over it without deciding which
|
||||
side is canonical; a fresh remote is the simple safe choice.
|
||||
|
|
@ -122,6 +122,71 @@ func TestServerModeLenientOpenDirtyTableGate(t *testing.T) {
|
|||
}
|
||||
})
|
||||
|
||||
// The migration-consent refusal fires before the dirty guard, so on a
|
||||
// database that is both behind and dirty the unconsented commit the
|
||||
// guard prescribes would rebuild this deadlock one check earlier.
|
||||
t.Run("behind and dirty database without consent recovers through a lenient open", func(t *testing.T) {
|
||||
const database = "lenient_consent"
|
||||
admin := prepareLenientOpenDatabaseAtV51(t, ctx, state.Port, database)
|
||||
defer admin.Close()
|
||||
|
||||
if _, err := admin.ExecContext(ctx,
|
||||
"INSERT INTO issues (id, title, description, design, acceptance_criteria, notes) VALUES (?, ?, '', '', '', '')",
|
||||
database+"-1", "uncommitted issue",
|
||||
); err != nil {
|
||||
t.Fatalf("dirty issues table: %v", err)
|
||||
}
|
||||
|
||||
schema.SetLocalMigrateConsent(false)
|
||||
t.Cleanup(func() { schema.SetLocalMigrateConsent(true) })
|
||||
t.Setenv(schema.AllowMigrateEnv, "")
|
||||
t.Setenv(schema.AllowRemoteMigrateEnv, "")
|
||||
|
||||
refuses := func(name string, cfg *Config) {
|
||||
t.Helper()
|
||||
s, err := New(ctx, cfg)
|
||||
if err == nil {
|
||||
s.Close()
|
||||
t.Fatalf("New (%s) = nil, want *schema.MigrateConsentError", name)
|
||||
}
|
||||
if !schema.IsMigrateConsentError(err) {
|
||||
t.Fatalf("New (%s) error = %T (%v), want *schema.MigrateConsentError", name, err, err)
|
||||
}
|
||||
}
|
||||
refuses("plain writable open", lenientOpenConfig(beadsDir, state.Port, database))
|
||||
syncCfg := lenientOpenConfig(beadsDir, state.Port, database)
|
||||
syncCfg.RemoteSyncOpen = true
|
||||
refuses("RemoteSyncOpen", syncCfg)
|
||||
|
||||
lenientCfg := lenientOpenConfig(beadsDir, state.Port, database)
|
||||
lenientCfg.LenientOpen = true
|
||||
store, err := New(ctx, lenientCfg)
|
||||
if err != nil {
|
||||
t.Fatalf("New (LenientOpen) = %v, want it to continue past the consent refusal", err)
|
||||
}
|
||||
defer store.Close()
|
||||
if got := lenientOpenSchemaVersion(t, ctx, store.db); got != 51 {
|
||||
t.Fatalf("schema version after lenient open = %d, want 51 (no consent, so no migration)", got)
|
||||
}
|
||||
if err := store.Commit(ctx, "checkpoint before migration"); err != nil {
|
||||
t.Fatalf("Commit: %v", err)
|
||||
}
|
||||
if got := lenientOpenDirtyCount(t, ctx, store.db); got != 0 {
|
||||
t.Fatalf("dolt_status rows after commit = %d, want 0", got)
|
||||
}
|
||||
store.Close()
|
||||
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
migrated, err := New(ctx, lenientOpenConfig(beadsDir, state.Port, database))
|
||||
if err != nil {
|
||||
t.Fatalf("New (consented, post-commit): %v", err)
|
||||
}
|
||||
defer migrated.Close()
|
||||
if got := lenientOpenSchemaVersion(t, ctx, migrated.db); got != schema.LatestVersion() {
|
||||
t.Fatalf("schema version after consented open = %d, want latest %d", got, schema.LatestVersion())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("clean database still migrates through a lenient open", func(t *testing.T) {
|
||||
const database = "lenient_clean"
|
||||
admin := prepareLenientOpenDatabaseAtV51(t, ctx, state.Port, database)
|
||||
|
|
|
|||
|
|
@ -392,14 +392,15 @@ type Config struct {
|
|||
// would make it eligible to sweep.
|
||||
ClassifiedRead bool
|
||||
|
||||
// LenientOpen opens the store leniently: a migration gate refusal (#4259)
|
||||
// or a dirty-working-set refusal (#4566) skips the migration instead of
|
||||
// failing the open. Set for working-set-reconcile commands (bd dolt
|
||||
// commit, bd vc commit; #4566), whose entire purpose is to clear the
|
||||
// working set that the migration would otherwise refuse to touch.
|
||||
// Honored in embedded and server mode alike. Migrations still RUN on a
|
||||
// lenient open — only those two refusals are tolerated — so a lenient
|
||||
// open of a clean database converges normally.
|
||||
// LenientOpen opens the store leniently: a migration gate refusal (#4259),
|
||||
// a dirty-working-set refusal (#4566) or a migration-consent refusal
|
||||
// skips the migration instead of failing the open. Set for
|
||||
// working-set-reconcile commands (bd dolt commit, bd vc commit; #4566),
|
||||
// whose entire purpose is to clear the working set that the migration
|
||||
// would otherwise refuse to touch. Honored in embedded and server mode
|
||||
// alike. Migrations still RUN on a lenient open — only those refusals are
|
||||
// tolerated — so a lenient open of a clean database with consent to
|
||||
// migrate converges normally.
|
||||
LenientOpen bool
|
||||
|
||||
// RemoteSyncOpen is LenientOpen's narrow sibling for the #6575
|
||||
|
|
@ -2261,8 +2262,9 @@ func newServerMode(ctx context.Context, cfg *Config) (*DoltStore, error) {
|
|||
return nil, fmt.Errorf("failed to initialize schema: %w", err)
|
||||
}
|
||||
// A tolerated refusal still reports what the aborted pass
|
||||
// applied (0 for both guards today, since each refuses before
|
||||
// migrating), so the rebuild below stays correct either way.
|
||||
// applied (0 for every tolerated refusal today, since each
|
||||
// refuses before migrating), so the rebuild below stays correct
|
||||
// either way.
|
||||
}
|
||||
// initSchema runs migrations over a separate pool (openMigrationDB).
|
||||
// The Ping above already pinned a connection in store.db to the
|
||||
|
|
@ -2300,14 +2302,15 @@ func newServerMode(ctx context.Context, cfg *Config) (*DoltStore, error) {
|
|||
// warnLenientOpenRefusal reports whether a lenient open (Config.LenientOpen)
|
||||
// may continue past err instead of failing, warning on stderr when it may.
|
||||
//
|
||||
// Server mode reaches the same two pending-migration refusals embedded mode
|
||||
// Server mode reaches the same pending-migration refusals embedded mode
|
||||
// relaxes for this intent (embeddeddolt's openWorkingSetReconcile): the #4566
|
||||
// dirty-table guard, whose documented recovery IS the commit these opens exist
|
||||
// to run, and the #4259 remote-migrate gate, a coordination stop with no
|
||||
// business blocking a commit of the local working set. Against an external
|
||||
// server the operator cannot sidestep either by deleting a local database, so
|
||||
// leaving them fatal here left the refusals with no in-band recovery at all
|
||||
// (#5781).
|
||||
// to run; the migration-consent refusal, which fires before that guard and so
|
||||
// would rebuild the same deadlock on a database both behind and dirty; and
|
||||
// the #4259 remote-migrate gate, a coordination stop with no business blocking
|
||||
// a commit of the local working set. Against an external server the operator
|
||||
// cannot sidestep any of them by deleting a local database, so leaving them
|
||||
// fatal here left the refusals with no in-band recovery at all (#5781).
|
||||
//
|
||||
// Every other migration failure still fails the open, and the schema-skew and
|
||||
// identity guards run before this point either way: lenient relaxes migration,
|
||||
|
|
@ -2318,10 +2321,15 @@ func warnLenientOpenRefusal(err error) bool {
|
|||
fmt.Fprintf(os.Stderr,
|
||||
"Warning: %v\n"+
|
||||
" Committing the working set at the current schema; when it completes,\n"+
|
||||
" re-run 'bd migrate'.\n",
|
||||
" re-run 'bd migrate schema'.\n",
|
||||
dirtyErr)
|
||||
return true
|
||||
}
|
||||
var consentErr *schema.MigrateConsentError
|
||||
if errors.As(err, &consentErr) {
|
||||
fmt.Fprint(os.Stderr, consentErr.WorkingSetWarning())
|
||||
return true
|
||||
}
|
||||
var gateErr *schema.RemoteMigrateGateError
|
||||
if errors.As(err, &gateErr) {
|
||||
fmt.Fprintf(os.Stderr,
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import (
|
|||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/doltserver"
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
"github.com/steveyegge/beads/internal/testutil"
|
||||
"github.com/steveyegge/beads/internal/testutil/credentialcmd"
|
||||
)
|
||||
|
|
@ -57,6 +58,11 @@ func TestMain(m *testing.M) {
|
|||
}
|
||||
|
||||
func testMainInner(m *testing.M) int {
|
||||
// Package-wide migration consent: these tests exercise the machinery that
|
||||
// runs BELOW the consent gate (migrate_consent.go), exactly as production
|
||||
// does once an operator has consented. Gate-behavior tests manage the
|
||||
// consent state themselves.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
os.Setenv("BEADS_TEST_MODE", "1")
|
||||
// BEADS_TEST_PDEATHSIG=1 protects TestMultiProcessSchemaInit_DoltVerify
|
||||
// (initschema_multiprocess_test.go), which calls doltserver.Start
|
||||
|
|
|
|||
|
|
@ -176,6 +176,7 @@ go_test(
|
|||
"merge_blockers_test.go",
|
||||
"merge_strategy_test.go",
|
||||
"metadata_cas_contract_test.go",
|
||||
"migrate_consent_gate_test.go",
|
||||
"migrate_dep_rekey_dedup_test.go",
|
||||
"migrate_events_flip_test.go",
|
||||
"migrate_frozen_guard_convergence_test.go",
|
||||
|
|
|
|||
|
|
@ -48,8 +48,10 @@ func OpenForReadOnlyCommand(ctx context.Context, beadsDir, database, branch stri
|
|||
// is to reconcile the Dolt working set (bd dolt commit, bd vc commit) must be
|
||||
// able to open the store even when pending migrations touch dirty tables -
|
||||
// otherwise the commit that would clear the dirty state and unblock the
|
||||
// migration can never run. The returned store is otherwise a normal writable
|
||||
// store.
|
||||
// migration can never run. A schema.MigrateConsentError, which fires first on
|
||||
// a database that is also behind, is skipped the same way
|
||||
// (toleratesConsentRefusal). The returned store is otherwise a normal
|
||||
// writable store.
|
||||
func OpenForWorkingSetReconcile(ctx context.Context, beadsDir, database, branch string) (*EmbeddedDoltStore, error) {
|
||||
return openCached(ctx, beadsDir, database, branch, openWorkingSetReconcile)
|
||||
}
|
||||
|
|
|
|||
190
internal/storage/embeddeddolt/migrate_consent_gate_test.go
Normal file
190
internal/storage/embeddeddolt/migrate_consent_gate_test.go
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
//go:build cgo
|
||||
|
||||
package embeddeddolt_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
"github.com/steveyegge/beads/internal/types"
|
||||
)
|
||||
|
||||
// TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers covers
|
||||
// the #4566 deadlock the migration-consent gate would otherwise rebuild one
|
||||
// check earlier. On a database that is both behind and dirty, `bd migrate
|
||||
// schema` consents but is refused by the dirty-table guard, whose remedy is
|
||||
// `bd dolt commit`; that commit opens the store without consent, and MigrateUp
|
||||
// checks consent before the dirty guard. The working-set-reconcile open must
|
||||
// warn and commit at the current schema, strict and read-only opens must keep
|
||||
// refusing, and the consented migration must then converge.
|
||||
func TestEmbeddedMigrateConsent_BehindAndDirtyWorkingSetReconcileRecovers(t *testing.T) {
|
||||
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
||||
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt tests")
|
||||
}
|
||||
|
||||
ctx := t.Context()
|
||||
beadsDir := filepath.Join(t.TempDir(), ".beads")
|
||||
dataDir := filepath.Join(beadsDir, "embeddeddolt")
|
||||
|
||||
// Build the fixture under the package-wide consent (TestMain): create
|
||||
// and commit the database, leave `issues` dirty, and regress the cursor
|
||||
// to v51 (absolute, for the reasons dirty_tables_gate_test.go gives).
|
||||
store, err := embeddeddolt.Open(ctx, beadsDir, "testdb", "main")
|
||||
if err != nil {
|
||||
t.Fatalf("Open: %v", err)
|
||||
}
|
||||
if err := store.SetConfig(ctx, "issue_prefix", "testdb"); err != nil {
|
||||
store.Close()
|
||||
t.Fatalf("SetConfig(issue_prefix): %v", err)
|
||||
}
|
||||
if err := store.Commit(ctx, "bd init"); err != nil {
|
||||
store.Close()
|
||||
t.Fatalf("Commit (init): %v", err)
|
||||
}
|
||||
issue := &types.Issue{
|
||||
ID: "testdb-1",
|
||||
Title: "dirty working set issue",
|
||||
Status: types.StatusOpen,
|
||||
Priority: 1,
|
||||
IssueType: types.TypeTask,
|
||||
}
|
||||
if err := store.CreateIssue(ctx, issue, "tester"); err != nil {
|
||||
store.Close()
|
||||
t.Fatalf("CreateIssue: %v", err)
|
||||
}
|
||||
db, cleanup, err := embeddeddolt.OpenSQL(ctx, dataDir, "testdb", "main")
|
||||
if err != nil {
|
||||
store.Close()
|
||||
t.Fatalf("OpenSQL: %v", err)
|
||||
}
|
||||
const regressedVersion = 51
|
||||
if _, err := db.ExecContext(ctx,
|
||||
"DELETE FROM schema_migrations WHERE version > ?", regressedVersion); err != nil {
|
||||
t.Fatalf("regress schema_migrations: %v", err)
|
||||
}
|
||||
_ = cleanup()
|
||||
store.Close()
|
||||
|
||||
// From here on no consent source is set: this is the `bd dolt commit`
|
||||
// an operator runs on the dirty guard's advice.
|
||||
schema.SetLocalMigrateConsent(false)
|
||||
schema.SetForceAllowRemoteMigrate(false)
|
||||
t.Cleanup(func() { schema.SetLocalMigrateConsent(true) })
|
||||
t.Setenv(schema.AllowMigrateEnv, "")
|
||||
t.Setenv(schema.AllowRemoteMigrateEnv, "")
|
||||
|
||||
type opener func(context.Context, string, string, string) (*embeddeddolt.EmbeddedDoltStore, error)
|
||||
refuses := func(name string, open opener) {
|
||||
t.Helper()
|
||||
s, err := open(ctx, beadsDir, "testdb", "main")
|
||||
if err == nil {
|
||||
s.Close()
|
||||
t.Fatalf("%s = nil, want *schema.MigrateConsentError on a behind database without consent", name)
|
||||
}
|
||||
if !schema.IsMigrateConsentError(err) {
|
||||
t.Fatalf("%s error = %T (%v), want *schema.MigrateConsentError (consent is checked before the dirty guard)", name, err, err)
|
||||
}
|
||||
}
|
||||
before := refusalSnapshot(t, dataDir, "testdb")
|
||||
refuses("Open", embeddeddolt.Open)
|
||||
refuses("OpenForReadOnlyCommand", embeddeddolt.OpenForReadOnlyCommand)
|
||||
if after := refusalSnapshot(t, dataDir, "testdb"); after != before {
|
||||
t.Fatalf("a refused open wrote to the database:\nbefore:\n%s\nafter:\n%s", before, after)
|
||||
}
|
||||
|
||||
var reconcileStore *embeddeddolt.EmbeddedDoltStore
|
||||
var openErr error
|
||||
warning := captureStderr(t, func() {
|
||||
reconcileStore, openErr = embeddeddolt.OpenForWorkingSetReconcile(ctx, beadsDir, "testdb", "main")
|
||||
})
|
||||
if openErr != nil {
|
||||
t.Fatalf("OpenForWorkingSetReconcile = %v, want it to warn and continue past the consent refusal", openErr)
|
||||
}
|
||||
if want := "Working-set reconcile command: continuing on schema v51 without"; !strings.Contains(warning, want) {
|
||||
reconcileStore.Close()
|
||||
t.Fatalf("warning is missing %q; got:\n%s", want, warning)
|
||||
}
|
||||
if current, err := schemaVersion(ctx, dataDir, "testdb"); err != nil {
|
||||
reconcileStore.Close()
|
||||
t.Fatalf("read schema version: %v", err)
|
||||
} else if current != regressedVersion {
|
||||
reconcileStore.Close()
|
||||
t.Fatalf("schema version = %d, want %d (no consent, so no migration)", current, regressedVersion)
|
||||
}
|
||||
if err := reconcileStore.Commit(ctx, "checkpoint"); err != nil {
|
||||
reconcileStore.Close()
|
||||
t.Fatalf("Commit: %v", err)
|
||||
}
|
||||
reconcileStore.Close()
|
||||
|
||||
// The commit consented to nothing.
|
||||
refuses("Open (after commit)", embeddeddolt.Open)
|
||||
|
||||
// The consent `bd migrate schema` records now converges: the dirty guard
|
||||
// that refused it has nothing left to refuse.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
migrated, err := embeddeddolt.Open(ctx, beadsDir, "testdb", "main")
|
||||
if err != nil {
|
||||
t.Fatalf("Open (consented, post-commit): %v", err)
|
||||
}
|
||||
defer migrated.Close()
|
||||
if current, err := schemaVersion(ctx, dataDir, "testdb"); err != nil {
|
||||
t.Fatalf("read schema version: %v", err)
|
||||
} else if current != schema.LatestVersion() {
|
||||
t.Fatalf("schema version after consented reopen = %d, want latest %d", current, schema.LatestVersion())
|
||||
}
|
||||
}
|
||||
|
||||
// refusalSnapshot renders what a refused open must leave untouched: the
|
||||
// working-set status, the dolt_ignore patterns MigrateUp seeds, and the
|
||||
// schema cursor.
|
||||
func refusalSnapshot(t *testing.T, dataDir, database string) string {
|
||||
t.Helper()
|
||||
ctx := t.Context()
|
||||
db, cleanup, err := embeddeddolt.OpenSQL(ctx, dataDir, database, "main")
|
||||
if err != nil {
|
||||
t.Fatalf("OpenSQL: %v", err)
|
||||
}
|
||||
defer func() { _ = cleanup() }()
|
||||
var b strings.Builder
|
||||
for _, q := range []string{
|
||||
"SELECT table_name, staged, status FROM dolt_status ORDER BY table_name, staged",
|
||||
"SELECT pattern, ignored FROM dolt_ignore ORDER BY pattern",
|
||||
"SELECT version FROM schema_migrations ORDER BY version",
|
||||
} {
|
||||
rows, err := db.QueryContext(ctx, q)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", q, err)
|
||||
}
|
||||
cols, err := rows.Columns()
|
||||
if err != nil {
|
||||
rows.Close()
|
||||
t.Fatalf("%s: columns: %v", q, err)
|
||||
}
|
||||
fmt.Fprintf(&b, "%s\n", q)
|
||||
for rows.Next() {
|
||||
vals := make([]any, len(cols))
|
||||
for i := range vals {
|
||||
vals[i] = new(any)
|
||||
}
|
||||
if err := rows.Scan(vals...); err != nil {
|
||||
rows.Close()
|
||||
t.Fatalf("%s: scan: %v", q, err)
|
||||
}
|
||||
for _, v := range vals {
|
||||
fmt.Fprintf(&b, " %v", *(v.(*any)))
|
||||
}
|
||||
b.WriteString("\n")
|
||||
}
|
||||
if err := rows.Close(); err != nil {
|
||||
t.Fatalf("%s: close: %v", q, err)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
|
@ -98,8 +98,9 @@ func lenientGateWarningBody(gateErr *schema.RemoteMigrateGateError) string {
|
|||
// open on any pending-migration refusal; openReadOnlyCommand and
|
||||
// openWorkingSetReconcile relax both the #4259 remote-migrate gate refusal and
|
||||
// the #4566 dirty-table refusal, each with its own warning text (see
|
||||
// initSchema); openRemoteSync relaxes exactly one gate refusal and nothing
|
||||
// else.
|
||||
// initSchema), and openWorkingSetReconcile alone also relaxes the
|
||||
// migration-consent refusal (toleratesConsentRefusal); openRemoteSync relaxes
|
||||
// exactly one gate refusal and nothing else.
|
||||
type openIntent int
|
||||
|
||||
const (
|
||||
|
|
@ -115,7 +116,9 @@ const (
|
|||
// commands (bd dolt commit, bd vc commit): their entire purpose is to
|
||||
// clear the dirty working set that a migration would otherwise refuse to
|
||||
// touch, so failing the open here would deadlock the documented recovery
|
||||
// (#4566). Used by OpenForWorkingSetReconcile.
|
||||
// (#4566). For the same reason it relaxes the migration-consent refusal,
|
||||
// which fires before the dirty-table guard. Used by
|
||||
// OpenForWorkingSetReconcile.
|
||||
openWorkingSetReconcile
|
||||
// openRemoteSync is the same shape of deadlock break as
|
||||
// openWorkingSetReconcile, for the #6575 data-behind gate refusal: that
|
||||
|
|
@ -240,6 +243,27 @@ func (s *EmbeddedDoltStore) toleratesMigrationRefusal() bool {
|
|||
return s.intent == openReadOnlyCommand || s.intent == openWorkingSetReconcile
|
||||
}
|
||||
|
||||
// toleratesConsentRefusal reports whether this open's intent may warn and
|
||||
// continue on the current schema past the migration-consent refusal
|
||||
// (schema.MigrateConsentError) instead of failing the open.
|
||||
//
|
||||
// Only openWorkingSetReconcile may. MigrateUp checks consent before the #4566
|
||||
// dirty-table guard, so on a database that is both behind and dirty a strict
|
||||
// commit open rebuilds the #4566 deadlock one check earlier: `bd migrate
|
||||
// schema` is refused as dirty and names `bd dolt commit`, which is refused
|
||||
// for consent. The commit needs no migration, and MigrateUp refuses before
|
||||
// any write, so it runs on the current schema.
|
||||
//
|
||||
// openReadOnlyCommand stays strict on purpose. The remote-migrate gate lets
|
||||
// reads ride through because its migrate-or-adopt decision is cross-clone
|
||||
// coordination that takes a while to settle; this decision is one local
|
||||
// command either way (`bd migrate schema`, or keep the matching release), and
|
||||
// the refusal is where the operator learns it is needed. openRemoteSync stays
|
||||
// strict for the reason toleratesMigrationRefusal gives.
|
||||
func (s *EmbeddedDoltStore) toleratesConsentRefusal() bool {
|
||||
return s.intent == openWorkingSetReconcile
|
||||
}
|
||||
|
||||
// errClosed is returned when a method is called after Close.
|
||||
var errClosed = errors.New("embeddeddolt: store is closed")
|
||||
|
||||
|
|
@ -637,6 +661,11 @@ func (s *EmbeddedDoltStore) initSchema(ctx context.Context) error {
|
|||
// Embedded mode relies on the dolthub/driver/v2's local file/concurrency
|
||||
// controls; schema.MigrateUpWithLock requires a sql-server session lock.
|
||||
if _, err := schema.MigrateUp(ctx, conn); err != nil {
|
||||
var consentErr *schema.MigrateConsentError
|
||||
if s.toleratesConsentRefusal() && errors.As(err, &consentErr) {
|
||||
fmt.Fprint(os.Stderr, consentErr.WorkingSetWarning())
|
||||
return nil
|
||||
}
|
||||
var dirtyErr *schema.DirtyTablesError
|
||||
if s.toleratesMigrationRefusal() && errors.As(err, &dirtyErr) {
|
||||
// The guard exists to keep dirty user data from being entangled
|
||||
|
|
@ -651,13 +680,13 @@ func (s *EmbeddedDoltStore) initSchema(ctx context.Context) error {
|
|||
fmt.Fprintf(os.Stderr,
|
||||
"Warning: %v\n"+
|
||||
" Committing the working set at the current schema; when it completes,\n"+
|
||||
" re-run 'bd migrate'.\n",
|
||||
" re-run 'bd migrate schema'.\n",
|
||||
dirtyErr)
|
||||
default: // openReadOnlyCommand
|
||||
fmt.Fprintf(os.Stderr,
|
||||
"Warning: %v\n"+
|
||||
" Continuing without migrating. Run 'bd dolt commit' to commit the\n"+
|
||||
" working set at the current schema, then re-run 'bd migrate'.\n",
|
||||
" working set at the current schema, then re-run 'bd migrate schema'.\n",
|
||||
dirtyErr)
|
||||
}
|
||||
return nil
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ import (
|
|||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
"github.com/steveyegge/beads/internal/types"
|
||||
)
|
||||
|
||||
|
|
@ -49,6 +50,12 @@ func TestMain(m *testing.M) {
|
|||
}
|
||||
embeddedDoltFixtureRoot = root
|
||||
|
||||
// Package-wide migration consent: these tests exercise the machinery that
|
||||
// runs BELOW the consent gate (migrate_consent.go), exactly as production
|
||||
// does once an operator has consented. Gate-behavior tests manage the
|
||||
// consent state themselves.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
|
||||
code := m.Run()
|
||||
if err := os.RemoveAll(root); err != nil && code == 0 {
|
||||
fmt.Fprintln(os.Stderr, "remove embedded Dolt test fixture root:", err)
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ go_library(
|
|||
"dirty_tables_error.go",
|
||||
"ignored_cursor_untrack.go",
|
||||
"lock.go",
|
||||
"migrate_consent.go",
|
||||
"migration_content_hashes.go",
|
||||
"migration_repairs.go",
|
||||
"remote_migrate_gate.go",
|
||||
|
|
@ -146,10 +147,12 @@ go_test(
|
|||
"dolt_ignore_seed_test.go",
|
||||
"ignored_cursor_untrack_test.go",
|
||||
"lock_test.go",
|
||||
"migrate_consent_test.go",
|
||||
"migration_0067_versioned_beads_test.go",
|
||||
"migration_0068_add_attribution_status_test.go",
|
||||
"migration_0069_datetime_precision_test.go",
|
||||
"migration_body_drain_test.go",
|
||||
"migration_freeze_test.go",
|
||||
"migration_watchdog_test.go",
|
||||
"progress_test.go",
|
||||
"remote_migrate_gate_test.go",
|
||||
|
|
|
|||
|
|
@ -326,7 +326,14 @@ func MigrateUpWithLock(ctx context.Context, conn *sql.Conn, databaseName string,
|
|||
}
|
||||
}
|
||||
|
||||
applied, err = MigrateUp(ctx, conn)
|
||||
// The same authority is consent for MigrateUp's own gate, which is that
|
||||
// version-only test: an init that never sets local consent would otherwise
|
||||
// be refused the retry of its own interrupted bootstrap.
|
||||
migrate := MigrateUp
|
||||
if o.freshBootstrapHeal != nil {
|
||||
migrate = migrateUpConsented
|
||||
}
|
||||
applied, err = migrate(ctx, conn)
|
||||
var dirtyErr *DirtyTablesError
|
||||
if err != nil && o.freshBootstrapHeal != nil && errors.As(err, &dirtyErr) {
|
||||
// Authorization is checked after the dirty guard fires and while the
|
||||
|
|
@ -347,7 +354,7 @@ func MigrateUpWithLock(ctx context.Context, conn *sql.Conn, databaseName string,
|
|||
if resetErr := drainFreshBootstrapReset(ctx, conn); resetErr != nil {
|
||||
return applied, errors.Join(err, fmt.Errorf("schema: fresh-bootstrap reset: %w", resetErr))
|
||||
}
|
||||
applied, err = MigrateUp(ctx, conn)
|
||||
applied, err = migrate(ctx, conn)
|
||||
}
|
||||
return applied, err
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1013,6 +1013,124 @@ func TestMigrateUpWithLockFreshBootstrapHealCapabilityIsOneShot(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// expectMigrateConsentReads mocks checkMigrateConsent's reads when no consent
|
||||
// is held: the main cursor once for the fresh-database exemption, and again
|
||||
// for the pending count.
|
||||
func expectMigrateConsentReads(mock sqlmock.Sqlmock, cursor int) {
|
||||
for range 2 {
|
||||
expectCursorProbe(mock, "schema_migrations", true)
|
||||
expectScalar(mock, "SELECT COALESCE(MAX(version), 0) FROM schema_migrations", "version", cursor)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrateUpWithLockFreshBootstrapHealIsMigrateConsent pins that heal
|
||||
// authority is consent for MigrateUp's own gate, not just for the caller's
|
||||
// migration gate. A retried server bootstrap finds the non-zero cursor its
|
||||
// first pass left behind, and server-mode `bd init` records no local consent,
|
||||
// so a version-only consent check would refuse the init the database it just
|
||||
// created. Both migration passes are covered; sqlmock's ordered expectations
|
||||
// fail either case on a consent read.
|
||||
func TestMigrateUpWithLockFreshBootstrapHealIsMigrateConsent(t *testing.T) {
|
||||
cases := map[string]func(t *testing.T, mock sqlmock.Sqlmock){
|
||||
"partially migrated": func(t *testing.T, mock sqlmock.Sqlmock) {
|
||||
expectOnePendingMigration(t, mock)
|
||||
},
|
||||
"partially migrated with dirty debris": func(t *testing.T, mock sqlmock.Sqlmock) {
|
||||
expectDirtyGuardRefusal(t, mock)
|
||||
expectFreshBootstrapIdentityMatch(mock)
|
||||
mock.ExpectQuery(regexp.QuoteMeta("CALL DOLT_RESET('--hard')")).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"status"}))
|
||||
expectOnePendingMigration(t, mock)
|
||||
},
|
||||
}
|
||||
for name, expectPasses := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
failOnSwallowedAdvisory(t)
|
||||
resetConsentState(t)
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("create sql mock: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
ctx := context.Background()
|
||||
conn, err := db.Conn(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("pin mock connection: %v", err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
lockName := MigrationLockName("testdb")
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT GET_LOCK(?, ?)")).
|
||||
WithArgs(lockName, migrationLockAcquireTimeoutSeconds).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"locked"}).AddRow(1))
|
||||
expectPasses(t, mock)
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT RELEASE_LOCK(?)")).
|
||||
WithArgs(lockName).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"released"}).AddRow(1))
|
||||
|
||||
applied, err := MigrateUpWithLock(ctx, conn, "testdb",
|
||||
WithFreshBootstrapHeal(testFreshBootstrapHealCapability(), testBootstrapEndpoint))
|
||||
if err != nil {
|
||||
t.Fatalf("MigrateUpWithLock() error = %v", err)
|
||||
}
|
||||
if applied != 1 {
|
||||
t.Fatalf("MigrateUpWithLock() applied = %d, want 1", applied)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatalf("unmet SQL expectations: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrateUpWithLockRefusesUnconsentedMigration is the control for the
|
||||
// test above: the same behind database, opened without heal authority or
|
||||
// consent, is refused under the lock before any write, and the lock is still
|
||||
// released.
|
||||
func TestMigrateUpWithLockRefusesUnconsentedMigration(t *testing.T) {
|
||||
failOnSwallowedAdvisory(t)
|
||||
resetConsentState(t)
|
||||
db, mock, err := sqlmock.New()
|
||||
if err != nil {
|
||||
t.Fatalf("create sql mock: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
ctx := context.Background()
|
||||
conn, err := db.Conn(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("pin mock connection: %v", err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
lockName := MigrationLockName("testdb")
|
||||
expectConvergedFastPathMiss(mock, "testdb")
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT GET_LOCK(?, ?)")).
|
||||
WithArgs(lockName, migrationLockAcquireTimeoutSeconds).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"locked"}).AddRow(1))
|
||||
expectMigrateConsentReads(mock, LatestVersion()-1)
|
||||
mock.ExpectQuery(regexp.QuoteMeta("SELECT RELEASE_LOCK(?)")).
|
||||
WithArgs(lockName).
|
||||
WillReturnRows(sqlmock.NewRows([]string{"released"}).AddRow(1))
|
||||
|
||||
applied, err := MigrateUpWithLock(ctx, conn, "testdb", WithDatabaseSelector(testDatabaseSelector))
|
||||
if applied != 0 {
|
||||
t.Fatalf("MigrateUpWithLock() applied = %d, want 0", applied)
|
||||
}
|
||||
var consentErr *MigrateConsentError
|
||||
if !errors.As(err, &consentErr) {
|
||||
t.Fatalf("MigrateUpWithLock() error = %v, want *MigrateConsentError", err)
|
||||
}
|
||||
if consentErr.CurrentVersion != LatestVersion()-1 || consentErr.Pending != 1 {
|
||||
t.Fatalf("refusal = v%d with %d pending, want v%d with 1 pending",
|
||||
consentErr.CurrentVersion, consentErr.Pending, LatestVersion()-1)
|
||||
}
|
||||
if err := mock.ExpectationsWereMet(); err != nil {
|
||||
t.Fatalf("unmet SQL expectations: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrateUpWithLockMigrationGate pins where WithMigrationGate's callback
|
||||
// sits: after the lock and after locked preparation (so the gate reads a
|
||||
// prepared, selected database), before MigrateUp (so a refusal applies no
|
||||
|
|
|
|||
235
internal/storage/schema/migrate_consent.go
Normal file
235
internal/storage/schema/migrate_consent.go
Normal file
|
|
@ -0,0 +1,235 @@
|
|||
package schema
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// Migration consent (gastownhall Beads 1.2 release remediation).
|
||||
//
|
||||
// bd used to auto-apply pending schema migrations the first time a new binary
|
||||
// opened an existing database — on ANY command, including reads. The 1.2.x
|
||||
// releases turned that into a trap: users upgraded by a package manager had
|
||||
// their databases migrated in place with no prompt, and older binaries then
|
||||
// refuse the migrated database (schema-skew guard). The remote-migrate gate
|
||||
// (#4259) already refuses the silent migration for remote-backed databases;
|
||||
// this gate extends the same contract to EVERY existing database: bd never
|
||||
// initiates a schema migration without explicit operator consent.
|
||||
//
|
||||
// Consent is any of:
|
||||
// - `bd migrate schema` (the verb that names the migration; the root
|
||||
// command records the intent via SetLocalMigrateConsent before the store
|
||||
// opens — bare `bd migrate` reconciles metadata and does not consent),
|
||||
// - --force on `bd migrate` or `bd migrate schema` (the remote-gate
|
||||
// override, a strict superset), or
|
||||
// - BD_ALLOW_MIGRATE=1 (scripted/CI form; BD_ALLOW_REMOTE_MIGRATE=1 is
|
||||
// honored as an alias for continuity with the remote gate's hatch).
|
||||
//
|
||||
// Fresh databases (no schema_migrations table, version 0) are exempt: creating
|
||||
// a database is consent for its schema. A server bootstrap holding
|
||||
// fresh-bootstrap heal authority keeps that consent across a retry whose
|
||||
// interrupted first pass left a non-zero cursor (MigrateUpWithLock). The same
|
||||
// creation principle covers a workspace this invocation just CLONED into
|
||||
// existence (`bd init --remote`, `bd bootstrap` sync) — those paths record the
|
||||
// consent explicitly, and the #4259 remote-migrate gate still applies to them
|
||||
// unchanged. Databases already at this binary's latest version never consult
|
||||
// the gate. The ignored (clone-local, dolt_ignored) migration sequence is
|
||||
// deliberately NOT gated on its own: it only materializes per-clone
|
||||
// bookkeeping tables and runs on every fresh clone — but when the MAIN
|
||||
// sequence is pending and unconsented, MigrateUp refuses before ANY write,
|
||||
// ignored sequence and dolt_ignore seeding included.
|
||||
//
|
||||
// A refusal fails the open, except for the working-set-reconcile commands
|
||||
// (bd dolt commit, bd vc commit), which commit at the current schema without
|
||||
// migrating: the refusal comes before the #4566 dirty-table guard, whose
|
||||
// documented recovery is that commit (WorkingSetWarning).
|
||||
|
||||
// AllowMigrateEnv, when set to a boolean true ("1", "true", ...), consents to
|
||||
// applying pending schema migrations to an existing database. It is consulted
|
||||
// only when migrations are actually pending, so exporting it permanently does
|
||||
// not warn on every store open.
|
||||
const AllowMigrateEnv = "BD_ALLOW_MIGRATE"
|
||||
|
||||
// localMigrateConsent is the programmatic consent recorded by
|
||||
// `bd migrate schema` and by the paths that just cloned the database into
|
||||
// existence. Process-local by design, like forceAllowRemoteMigrate: it cannot
|
||||
// leak into child processes.
|
||||
var localMigrateConsent bool
|
||||
|
||||
// SetLocalMigrateConsent records (or clears) the in-process consent to apply
|
||||
// pending schema migrations to an existing database. Set by the root command
|
||||
// when the invoked command is `bd migrate schema`, before both
|
||||
// autoMigrateOnVersionBump and the main store open. External test packages may
|
||||
// reset it to false after each test case.
|
||||
func SetLocalMigrateConsent(v bool) { localMigrateConsent = v }
|
||||
|
||||
// MigrateConsentError is returned when bd would apply pending schema
|
||||
// migrations to an existing database without operator consent.
|
||||
type MigrateConsentError struct {
|
||||
CurrentVersion int
|
||||
LatestVersion int
|
||||
Pending int
|
||||
// UnrecognizedEnv carries a BD_ALLOW_MIGRATE value that was set but not
|
||||
// understood (only boolean values consent), so a typo'd hatch fails with
|
||||
// a hint instead of silently staying locked (0054-gate precedent,
|
||||
// bd-6dnrw.34).
|
||||
UnrecognizedEnv string
|
||||
}
|
||||
|
||||
func (e *MigrateConsentError) Error() string {
|
||||
return fmt.Sprintf("%s — run `bd migrate schema`, or keep using a bd release that matches schema v%d",
|
||||
e.Refusal(), e.CurrentVersion)
|
||||
}
|
||||
|
||||
// Refusal states what was refused without prescribing a remedy, for surfaces
|
||||
// that carry the remedy separately and must not hand an agent a runnable
|
||||
// migration (cmd/bd's JSON refusal, whose remedies live in Options).
|
||||
func (e *MigrateConsentError) Refusal() string {
|
||||
return fmt.Sprintf(
|
||||
"database schema is at v%d; this bd requires v%d (%d pending migration(s)), and bd no longer migrates a database without explicit consent",
|
||||
e.CurrentVersion, e.LatestVersion, e.Pending)
|
||||
}
|
||||
|
||||
// AgentDirective is the non-runnable instruction surfaced to agents in place
|
||||
// of a ready-to-run migrate command, following the remote-migrate gate's
|
||||
// convention (RemoteMigrateGateError.AgentDirective). Migrating is one-way,
|
||||
// and whether anything still has to open this database with an older bd is
|
||||
// not observable from this process, so the decision goes to the operator.
|
||||
func (e *MigrateConsentError) AgentDirective() string {
|
||||
return fmt.Sprintf(
|
||||
"Operator decision required: this database is at schema v%d and this bd requires v%d. Migrating is ONE-WAY — "+
|
||||
"bd releases built for an older schema refuse the migrated database, and whether one still has to open it "+
|
||||
"is not observable from this process. Do NOT auto-run the migration or set a consent environment variable "+
|
||||
"to get past this refusal: surface migrate_consent.options to the operator and let them choose.",
|
||||
e.CurrentVersion, e.LatestVersion)
|
||||
}
|
||||
|
||||
// Options returns the two mutually exclusive paths, migrate or keep the
|
||||
// current schema, each gated on its precondition and annotated with its risk.
|
||||
// The migrate command is reachable only through the operator's decision,
|
||||
// never as a top-level hint. Its verb is the shared-store gate's consent verb
|
||||
// too, so cmd/bd retargets both with the same --global rewrite.
|
||||
func (e *MigrateConsentError) Options() []GateOption {
|
||||
return []GateOption{
|
||||
{
|
||||
ID: "migrate",
|
||||
When: "the operator chooses to upgrade this database, and nothing that still opens it runs a bd release built for an older schema (confirmed with the operator)",
|
||||
Commands: []string{SharedConsentCommand},
|
||||
Risk: fmt.Sprintf("one-way: v%d -> v%d; bd releases built for an older schema refuse the migrated database", e.CurrentVersion, e.LatestVersion),
|
||||
},
|
||||
{
|
||||
ID: "keep",
|
||||
When: fmt.Sprintf("something still has to open this database with a bd release built for schema v%d", e.CurrentVersion),
|
||||
Commands: []string{},
|
||||
Risk: "none to the database; this bd release refuses it until it is migrated, so keep using the release that matches it",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// UserMessage renders the full operator-facing refusal.
|
||||
func (e *MigrateConsentError) UserMessage() string {
|
||||
msg := fmt.Sprintf(
|
||||
"This database is at schema v%d; this bd release uses schema v%d.\n"+
|
||||
"bd no longer upgrades a database automatically (see the release notes).\n"+
|
||||
"\n"+
|
||||
" To upgrade this database (one-way; older bd releases cannot read it):\n"+
|
||||
" bd migrate schema\n"+
|
||||
"\n"+
|
||||
" To keep the current schema, keep using the bd release that created it.\n"+
|
||||
"\n"+
|
||||
" Scripted/CI consent: BD_ALLOW_MIGRATE=1 bd <command>\n",
|
||||
e.CurrentVersion, e.LatestVersion)
|
||||
if e.UnrecognizedEnv != "" {
|
||||
msg += fmt.Sprintf(
|
||||
"\nNote: %s=%q was set but not understood (use a boolean value, e.g. \"1\" or \"true\"), so it did not consent.\n",
|
||||
AllowMigrateEnv, e.UnrecognizedEnv)
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
// WorkingSetWarning renders the stderr warning for a working-set-reconcile
|
||||
// open (bd dolt commit, bd vc commit) that continues past this refusal. The
|
||||
// commit needs no migration, and MigrateUp refused before any write, so it
|
||||
// runs on the current schema; the migration still waits for consent. Both
|
||||
// storage modes print it (embeddeddolt's openWorkingSetReconcile, server
|
||||
// mode's LenientOpen).
|
||||
func (e *MigrateConsentError) WorkingSetWarning() string {
|
||||
return fmt.Sprintf(
|
||||
"Warning: %s.\n"+
|
||||
" Working-set reconcile command: continuing on schema v%d without\n"+
|
||||
" migrating; the commit applies to the working set at the current schema.\n"+
|
||||
" To upgrade this database afterwards (one-way): bd migrate schema\n",
|
||||
e.Refusal(), e.CurrentVersion)
|
||||
}
|
||||
|
||||
// IsMigrateConsentError reports whether err (or any error it wraps) is a
|
||||
// *MigrateConsentError.
|
||||
func IsMigrateConsentError(err error) bool {
|
||||
var e *MigrateConsentError
|
||||
return errors.As(err, &e)
|
||||
}
|
||||
|
||||
// checkMigrateConsent refuses an unconsented migration of an existing
|
||||
// database. Called at the top of MigrateUp, before any write (dolt_ignore
|
||||
// seeding included). Returns nil for a fresh database (version 0), a database
|
||||
// with no pending main-sequence migrations, or when consent was given.
|
||||
func checkMigrateConsent(ctx context.Context, db DBConn) error {
|
||||
if localMigrateConsent || forceAllowRemoteMigrate {
|
||||
return nil // programmatic consent — skip the version reads entirely
|
||||
}
|
||||
current, err := mainSource.currentVersion(ctx, db)
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate consent: read current version: %w", err)
|
||||
}
|
||||
if current == 0 {
|
||||
return nil // fresh database — creating it is consent for its schema
|
||||
}
|
||||
pending, err := mainSource.pendingVersions(ctx, db)
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate consent: read pending versions: %w", err)
|
||||
}
|
||||
return migrateConsentDecision(current, len(pending))
|
||||
}
|
||||
|
||||
// migrateConsentDecision is checkMigrateConsent's pure decision core: nil when
|
||||
// nothing is pending or consent was given (programmatic or env), a typed
|
||||
// *MigrateConsentError otherwise. current is known to be > 0 here except when
|
||||
// callers short-circuit earlier; a zero current is still treated as consent
|
||||
// (fresh database).
|
||||
func migrateConsentDecision(current, pending int) error {
|
||||
if current == 0 || pending == 0 {
|
||||
return nil
|
||||
}
|
||||
if localMigrateConsent || forceAllowRemoteMigrate {
|
||||
return nil
|
||||
}
|
||||
unrecognized := ""
|
||||
for _, env := range []string{AllowMigrateEnv, AllowRemoteMigrateEnv} {
|
||||
v := os.Getenv(env)
|
||||
if v == "" {
|
||||
continue
|
||||
}
|
||||
allowed, perr := strconv.ParseBool(v)
|
||||
if perr != nil {
|
||||
if env == AllowMigrateEnv {
|
||||
unrecognized = v
|
||||
}
|
||||
continue
|
||||
}
|
||||
if allowed {
|
||||
fmt.Fprintf(defaultStderr(),
|
||||
"Applying %d pending schema migration(s) (%s=%s): v%d -> v%d\n",
|
||||
pending, env, v, current, LatestVersion())
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return &MigrateConsentError{
|
||||
CurrentVersion: current,
|
||||
LatestVersion: LatestVersion(),
|
||||
Pending: pending,
|
||||
UnrecognizedEnv: unrecognized,
|
||||
}
|
||||
}
|
||||
164
internal/storage/schema/migrate_consent_test.go
Normal file
164
internal/storage/schema/migrate_consent_test.go
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
package schema
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestMain grants migration consent for the whole package: the MigrateUp
|
||||
// machinery tests exercise what runs BELOW the consent gate, exactly as
|
||||
// production does once an operator has consented. The consent tests below
|
||||
// clear the grant themselves via resetConsentState.
|
||||
func TestMain(m *testing.M) {
|
||||
SetLocalMigrateConsent(true)
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// resetConsentState clears every consent source for one test case (undoing
|
||||
// TestMain's package-wide grant) and restores it afterwards, so consent tests
|
||||
// see a clean slate and machinery tests keep their grant.
|
||||
func resetConsentState(t *testing.T) {
|
||||
t.Helper()
|
||||
SetLocalMigrateConsent(false)
|
||||
SetForceAllowRemoteMigrate(false)
|
||||
t.Cleanup(func() {
|
||||
SetLocalMigrateConsent(true)
|
||||
SetForceAllowRemoteMigrate(false)
|
||||
})
|
||||
t.Setenv(AllowMigrateEnv, "")
|
||||
t.Setenv(AllowRemoteMigrateEnv, "")
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_FreshDB_Allows(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
if err := migrateConsentDecision(0, 12); err != nil {
|
||||
t.Fatalf("decision = %v, want nil for fresh database (current=0)", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_NothingPending_Allows(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
if err := migrateConsentDecision(LatestVersion(), 0); err != nil {
|
||||
t.Fatalf("decision = %v, want nil when nothing is pending", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_PendingNoConsent_Refuses(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
err := migrateConsentDecision(53, 12)
|
||||
var consentErr *MigrateConsentError
|
||||
if !errors.As(err, &consentErr) {
|
||||
t.Fatalf("decision = %v, want *MigrateConsentError", err)
|
||||
}
|
||||
if consentErr.CurrentVersion != 53 || consentErr.Pending != 12 {
|
||||
t.Fatalf("error fields = v%d/%d pending, want v53/12", consentErr.CurrentVersion, consentErr.Pending)
|
||||
}
|
||||
if !IsMigrateConsentError(err) {
|
||||
t.Fatalf("IsMigrateConsentError(err) = false, want true")
|
||||
}
|
||||
msg := consentErr.UserMessage()
|
||||
for _, want := range []string{"bd migrate schema", AllowMigrateEnv} {
|
||||
if !strings.Contains(msg, want) {
|
||||
t.Fatalf("UserMessage missing %q:\n%s", want, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_LocalConsent_Allows(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
SetLocalMigrateConsent(true)
|
||||
if err := migrateConsentDecision(53, 12); err != nil {
|
||||
t.Fatalf("decision = %v, want nil with local consent set", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_ForceOverride_Allows(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
SetForceAllowRemoteMigrate(true)
|
||||
if err := migrateConsentDecision(53, 12); err != nil {
|
||||
t.Fatalf("decision = %v, want nil with the migrate --force override set", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_EnvConsent_Allows(t *testing.T) {
|
||||
for _, env := range []string{AllowMigrateEnv, AllowRemoteMigrateEnv} {
|
||||
for _, v := range []string{"1", "true", "TRUE"} {
|
||||
t.Run(env+"="+v, func(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
t.Setenv(env, v)
|
||||
if err := migrateConsentDecision(53, 12); err != nil {
|
||||
t.Fatalf("decision = %v, want nil with %s=%s", err, env, v)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_EnvFalse_Refuses(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
t.Setenv(AllowMigrateEnv, "0")
|
||||
if !IsMigrateConsentError(migrateConsentDecision(53, 12)) {
|
||||
t.Fatalf("want refusal with %s=0", AllowMigrateEnv)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateConsentDecision_UnparseableEnv_RefusesWithHint(t *testing.T) {
|
||||
resetConsentState(t)
|
||||
t.Setenv(AllowMigrateEnv, "yes-please")
|
||||
err := migrateConsentDecision(53, 12)
|
||||
var consentErr *MigrateConsentError
|
||||
if !errors.As(err, &consentErr) {
|
||||
t.Fatalf("decision = %v, want *MigrateConsentError", err)
|
||||
}
|
||||
if consentErr.UnrecognizedEnv != "yes-please" {
|
||||
t.Fatalf("UnrecognizedEnv = %q, want the unparseable value surfaced", consentErr.UnrecognizedEnv)
|
||||
}
|
||||
if !strings.Contains(consentErr.UserMessage(), "yes-please") {
|
||||
t.Fatalf("UserMessage does not surface the unparseable env value:\n%s", consentErr.UserMessage())
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrateConsentError_AgentSurfacesNameNoRemedy pins the #4259 directive
|
||||
// convention cmd/bd's JSON refusal is built on: Refusal and AgentDirective are
|
||||
// the top-level fields an agent reads, so neither may carry a runnable
|
||||
// migration or a consent env var; the migrate command lives only in the
|
||||
// operator-gated "migrate" option.
|
||||
func TestMigrateConsentError_AgentSurfacesNameNoRemedy(t *testing.T) {
|
||||
e := &MigrateConsentError{CurrentVersion: 65, LatestVersion: 69, Pending: 4}
|
||||
for name, s := range map[string]string{"Refusal": e.Refusal(), "AgentDirective": e.AgentDirective()} {
|
||||
for _, unwanted := range []string{SharedConsentCommand, "BD_ALLOW"} {
|
||||
if strings.Contains(s, unwanted) {
|
||||
t.Errorf("%s carries %q:\n%s", name, unwanted, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
opts := e.Options()
|
||||
if len(opts) != 2 || opts[0].ID != "migrate" || opts[1].ID != "keep" {
|
||||
t.Fatalf("Options = %+v, want exactly migrate then keep", opts)
|
||||
}
|
||||
if len(opts[0].Commands) != 1 || opts[0].Commands[0] != SharedConsentCommand {
|
||||
t.Errorf("migrate option commands = %q, want [%q]", opts[0].Commands, SharedConsentCommand)
|
||||
}
|
||||
if len(opts[1].Commands) != 0 {
|
||||
t.Errorf("keep option commands = %q, want none", opts[1].Commands)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrateConsentError_WorkingSetWarning pins the warning a
|
||||
// working-set-reconcile open prints when it continues past the refusal: it
|
||||
// states the refusal and that the commit runs on the current schema, and it
|
||||
// never offers the consent env var as the way forward.
|
||||
func TestMigrateConsentError_WorkingSetWarning(t *testing.T) {
|
||||
e := &MigrateConsentError{CurrentVersion: 65, LatestVersion: 69, Pending: 4}
|
||||
w := e.WorkingSetWarning()
|
||||
for _, want := range []string{e.Refusal(), "Working-set reconcile command: continuing on schema v65 without"} {
|
||||
if !strings.Contains(w, want) {
|
||||
t.Errorf("WorkingSetWarning missing %q:\n%s", want, w)
|
||||
}
|
||||
}
|
||||
if strings.Contains(w, "BD_ALLOW") {
|
||||
t.Errorf("WorkingSetWarning offers a consent env var:\n%s", w)
|
||||
}
|
||||
}
|
||||
183
internal/storage/schema/migration_freeze_test.go
Normal file
183
internal/storage/schema/migration_freeze_test.go
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
package schema
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"io/fs"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Migration freeze (Beads 1.2 release remediation).
|
||||
//
|
||||
// Migrations 0001–0066 (and ignored/ 0001–0026) are SHIPPED: the 1.2.x
|
||||
// releases published schema v65 to thousands of installs and the 1.3.x
|
||||
// releases added 0066 and ignored/ 0025–0026, so these files are now the
|
||||
// de-facto on-disk contract of every migrated database. Amending a shipped
|
||||
// migration forks installed schemas from freshly-migrated ones — the exact
|
||||
// skew the smart gate's fork-skew detection exists to catch at runtime. This
|
||||
// test catches it in CI instead: the content hash of every shipped
|
||||
// up-migration is pinned; any change must land as a NEW migration, numbered
|
||||
// after the newest file in its directory. When a release ships new
|
||||
// migrations, raise the floors below and pin their hashes.
|
||||
//
|
||||
// If this test fails because you edited a shipped migration: revert the edit
|
||||
// and write a new migration. There is no legitimate reason to update a pinned
|
||||
// hash for an already-released version.
|
||||
|
||||
const (
|
||||
frozenMainThrough = 66
|
||||
frozenIgnoredThrough = 26
|
||||
)
|
||||
|
||||
var frozenMigrationHashes = map[string]string{
|
||||
"0001_create_issues.up.sql": "0c9bc4d97e2c5ee5d276c952333b934cd7d7d18210997d6bfd72d17f0fdb14f0",
|
||||
"0002_create_dependencies.up.sql": "9ca011fa0b24ddf26f689eebf347cc2f80dc90256b099d91b993ec82c4d74da5",
|
||||
"0003_create_labels.up.sql": "1780f0dc941d97b74af53d5603646d05e83d94eee5a02ac68ccf929200deaec0",
|
||||
"0004_create_comments.up.sql": "35ac76b3d0b2d6287cad1329c77c8a0562bb6310ba0ac34ae273d72d4bed0e2a",
|
||||
"0005_create_events.up.sql": "c8cbe4bebbe0f92c8a0ccc2a1a0d11ad442635811c2dab093c34c9341a8499b2",
|
||||
"0006_create_config.up.sql": "435f5ef129dd4999a9b393c98d633038f0e443280a26e63f7d4315bf025dab1f",
|
||||
"0007_create_metadata.up.sql": "f92c4d3dd32cae489142de34c4706900736c91cdc53b0d54e87e71656b828dc9",
|
||||
"0008_create_child_counters.up.sql": "99db9b4b5a785ddd209dff628364c4c1eff549c3ca5ec48a64c604b15ef95bfb",
|
||||
"0009_create_issue_snapshots.up.sql": "269ef83f083766b7f2fd2dfd2814ef36a585ae4aa18bcab0468a713d48595e48",
|
||||
"0010_create_compaction_snapshots.up.sql": "cb34cbe311fd89a17d72067f449a65089f2fc9a7c4b3774d54448db5ebc3d659",
|
||||
"0011_create_repo_mtimes.up.sql": "bde5204466541dddb53a652fe58561c08b6b3c1fa7feb38a0a582d432d76bda6",
|
||||
"0012_create_routes.up.sql": "68e8ef1fc336e8ca6973f6b927b09d3206a7ebd96db9c4cdceab52fb8498c431",
|
||||
"0013_create_issue_counter.up.sql": "7ddb30dabe32a66197de78f62400f41d000cf43d0ba8ba2a3943536504dddcdb",
|
||||
"0014_create_interactions.up.sql": "0e17f1cb9fc5ee5a7c43ca8e02b9fd59814f8fd4372f9ffb17b50c34388614ce",
|
||||
"0015_create_federation_peers.up.sql": "6aa04b3f66566b243c9a4fc09136af5c17f0ffe947a489245e3c451b995a59b9",
|
||||
"0016_default_config.up.sql": "5d556c01404eb06acfb9df2b20fde36a2e91bd8a7d14462acbd3c0ae928ea286",
|
||||
"0017_create_ready_issues_view.up.sql": "4bab7acbe2d8e441f156cc5912f421dcb214b9b08d7958eacc8e3a7318ab5f57",
|
||||
"0018_create_blocked_issues_view.up.sql": "84dc6a3950a5447e6665e3d98b43f11a7631dc4e53bab1be332d135e51c9511f",
|
||||
"0019_wisps_dolt_ignore.up.sql": "3df660feb6b64c84ef74c408fe8f5840a2815936534362ed123122f795a21c26",
|
||||
"0020_create_wisps.up.sql": "bca95662441635454b879bb44ca1e607df9c5efd8fd802be3d53a7371a5ced86",
|
||||
"0021_create_wisp_auxiliary.up.sql": "d19284a2be70a448db7746ae009cb264dc86dac0d9094aae9fe80a713a08ee70",
|
||||
"0022_wisp_dep_type_index.up.sql": "fc1375e64620cd4264516718d8f924d88131881f4c00c51471b7630c11b74a45",
|
||||
"0023_add_no_history_column.up.sql": "d11cd7cb462239e49c0b2f9b93ac73771106d5450ebc3422bb7737748c6f967b",
|
||||
"0024_create_custom_status_type_tables.up.sql": "816da3feeed531ca9787dca8d4b8072dd94b64d573cf4b6751c61f2390c88a68",
|
||||
"0025_update_ready_issues_view.up.sql": "09782d195f22d620ba41802856f9eb6e1a10f68bc5bcc8f3616624f8c28b576a",
|
||||
"0026_update_blocked_issues_view.up.sql": "33daef6b5a07364fc5f59ab59b0f5c677dcf98d9bd0866c7c3bea874f1f01348",
|
||||
"0027_add_started_at.up.sql": "dd3b58a8bbed4c49e855a6002981a285f10b6094938534dd3da9b121155f24d6",
|
||||
"0028_local_state_dolt_ignore.up.sql": "24a816406a6f40c66db98697869d172b2b4366b107788fdd00f44b99abee45c3",
|
||||
"0029_create_local_metadata.up.sql": "b989fc49ab03e06ff6520f150d6958efdbd3f444b69b824f7b4bb283ca6f2b65",
|
||||
"0030_migrate_local_metadata_keys.up.sql": "8d55135274c1c75e722e802b250911cc574d52a4ba74fbb13edde84f30f9b3a4",
|
||||
"0031_wisp_events_created_at_index.up.sql": "2d5cf3a6d4245b48c1b893efcc17b9fa25f1ea807e61290cf54064451fc58d93",
|
||||
"0032_drop_schema_migrations_applied_at.up.sql": "42b86d6b7352f7e158532cc67f4afa142dc2503ce9f780733c06fc9da3478751",
|
||||
"0033_add_wisp_type_column.up.sql": "9256746bb0d6ad9ed9afba0f0501259f0a3599b7d7da507dd7d808a13c688c75",
|
||||
"0034_add_spec_id_column.up.sql": "7add95ead971c57618cfbc52e5cb6771d872785bfc035bde12cd27a76358848e",
|
||||
"0035_migrate_infra_to_wisps.up.sql": "f917fe319540155572f5c51618cbbe20ec8b9b669d5926d68e3a62629c317c5a",
|
||||
"0036_cleanup_autopush_metadata.up.sql": "e55f1870c2bc031c574b495ebca650365a9c8088d7a8968fb86a19cab81267e8",
|
||||
"0037_uuid_primary_keys.up.sql": "7526beadee13aa7d6ee2175e9a543210b07c1a4426c382f251ebd31eebe32745",
|
||||
"0038_drop_hop_columns.up.sql": "002dc637d45840487c35a01ce15cee04872baa45c21f8e20170a4339135a38e7",
|
||||
"0039_drop_child_counters_fk.up.sql": "db65f241d1e82da41f34b960abc912f78b72e7dc1cb6dd2298974308edf8b768",
|
||||
"0040_ignored_tables_also_nonlocal_tables.up.sql": "c7068d67ad3e2b4852549e04d97f25814121de83db919e220bdcb9bdd9b61e5e",
|
||||
"0041_split_dependencies_target.up.sql": "e84ef393576df31550570fd3217579b1880850c0645b7d2b6ad8cf55a18ac1c0",
|
||||
"0042_add_on_update_cascade.up.sql": "9d0fbb3d02955d25442c4a628ac518236068a97ecf164edbc814216bd5a55adc",
|
||||
"0043_drop_dependencies_generated_column.up.sql": "548b8a69ab82179b7ee7ab3cc87ef00b3ae28d5a3a4976f42364f6939e384668",
|
||||
"0044_update_views_drop_depends_on_id.up.sql": "531524854cc0e2361817d1ad94ba9470b0e81e378a5422ffb8bce71f3500adf9",
|
||||
"0045_update_blocked_view_drop_depends_on_id.up.sql": "395dcbf186deedb592d8ed144e0b7953455efa0ccf02076d3b63e5e5d05dc241",
|
||||
"0046_add_is_blocked.up.sql": "4d6a5a5fc5e386f2c538b4f3574b82fb2ba025176997953a0a4fbd2a91cddf68",
|
||||
"0047_recompute_mixed_is_blocked.up.sql": "4aff5a7cc2e1e0250ce4cdb911d010c936681871fb627bceee4b2dc4647dd0a1",
|
||||
"0048_widen_event_value_columns.up.sql": "093c7a41145508879844d16cbe5acfa69baddbb19bf3e05f00bc08720adea657",
|
||||
"0049_longtext_large_content_columns.up.sql": "3359343efb7c28e5803de07e9088670489615d151843d7f23705f3ca8dd6a1e8",
|
||||
"0050_dependencies_deterministic_id.up.sql": "96265ff898753f3bb34549fafa18bbaeb67519705b8cf4e32e67f1058df6841a",
|
||||
"0051_drop_aux_id_defaults.up.sql": "deb741d5ac5fc1e0cd2e60eb47e84f384ed3455b18d3e2f12cb4e935fe461059",
|
||||
"0052_add_date_indexes.up.sql": "60ae32bc26c41da2e743c1172f73930d18482b8a98aea08b9de85f6009f9c01d",
|
||||
"0053_repair_rig_wisps.up.sql": "f13909f0db89f48db694a782d1805eee7ee1b2b2e4dcb3d4b12352333cbcb330",
|
||||
"0054_add_lease_columns.up.sql": "2e51058ba15be9d87415949bb03c2c7c2f0e0743c5ae0d3e819b617d95f61680",
|
||||
"0055_move_leases_to_table.up.sql": "2237ddbe13f5a783c65da3bb1292d9a5d89a5f22fddb0c33d2db57f1492df22c",
|
||||
"0056_add_comments_keyset_index.up.sql": "acc28e9b1aa7272c16c86fcb9fd4a2038ae47729a60b53bb75f3a9df05cd3a82",
|
||||
"0057_events_value_columns_idempotent_longtext.up.sql": "f86e0f77be2e58affe1c485767b8dd450408d782c0413a41948ca0c0b6d72d06",
|
||||
"0058_heal_wisp_dependencies_split_constraints.up.sql": "9c1c6d557a03e71cf6b49123831c9f5b5f910fcea3ca51e982ec4acce04ea6ec",
|
||||
"0059_recompute_null_gate_is_blocked.up.sql": "f0186029a1c87d579b67e0540ec22ad0bd0fc59793b07f7d24a24e122b77c394",
|
||||
"0060_add_storage_class.up.sql": "40419632ba75d92dfcf08e16e77a780f014172a17068f6021937df9086a24588",
|
||||
"0061_content_derived_aux_ids.up.sql": "d2d65b92789f137da8ee586067c862a9c7744921ccac421e1b793f42050b62a2",
|
||||
"0062_events_dolt_ignore.up.sql": "c7de6089997344784a71e7d82ea5e417baaf66320b4533d225a4a6bc3c74222a",
|
||||
"0063_create_provenance_events.up.sql": "1ef173be07ca58d17cb09ba3b3f5ca0c96edfcace8d52ae3adde2fd27e8248bd",
|
||||
"0064_create_events_journal.up.sql": "25ea9ad8e849dc7e32e7c5850631bedc63a5c5da3bdc58ccb2762fd7a6c758fc",
|
||||
"0065_widen_wisp_comments_text.up.sql": "c7341bb977daff92e073db2579e63058cef88d43f1e09b13fa153a772caac2a6",
|
||||
"0066_add_events_journal_actor.up.sql": "a524d42ba0eced9350edbf6402dc2952957ff8f75a896bc8339c4cbc8596c793",
|
||||
}
|
||||
|
||||
var frozenIgnoredMigrationHashes = map[string]string{
|
||||
"0001_create_local_state_tables.up.sql": "a73e9a486124f89e90d50f4597fd1d7ad08e5d0d654459ec87b7520b47f1bb5f",
|
||||
"0002_create_wisp_child_counters.up.sql": "c575c6710169f2e697f3aca60f5f698acc8a64af0edb1e2305b9d377fd845f45",
|
||||
"0003_split_wisp_dependencies_target.up.sql": "f82947deeafe589598e2a1802eddf17712bd322692f924bb5653b0fbd5c46a0d",
|
||||
"0004_add_wisp_aux_fks.up.sql": "12e18a8bdd214d6f96625e78998433ccc9c2d0ef4d1a35307f2653e725c800b9",
|
||||
"0005_drop_wisp_dependencies_generated_column.up.sql": "08780b28ddef6c15f5de558ea08d821b52610f810555739ee1d737779986c607",
|
||||
"0006_add_wisp_is_blocked.up.sql": "4746f10d7040d0b7a9e127437356de87765d12d2d27ee37c9800191e6b619d14",
|
||||
"0007_recompute_wisp_is_blocked.up.sql": "c3c8d92846a54183cc99afc09f779f6f5ba8030c16ce46b3ee750314bf134627",
|
||||
"0008_widen_wisp_event_value_columns.up.sql": "51375854ec081d2e0b1434b5ff8b6d87092b891636214ebb9f1dbef22438cc1a",
|
||||
"0009_aux_row_id_rekey_marker.up.sql": "3d1ee4f9dde3342d110474c8090a30c474edafe2290e4372a3fc4607c1fdc030",
|
||||
"0010_drop_wisp_id_defaults.up.sql": "59d6644667b7e34f9d23261cb694d461ddd4953969e75519d370b0cce8afd0e8",
|
||||
"0011_cleanup_orphaned_child_counters.up.sql": "0efade0ecb620420ee42127d3f67230da108e31f2ff3c4281855a8dc9e187ecd",
|
||||
"0012_create_leases.up.sql": "e128a1301a2bcb967f684b09318aa5cf7ed9adb5f6dd7d412de3d67d1de2b28d",
|
||||
"0013_add_wisp_row_lock.up.sql": "127a4af7012769e5f016b13b198b7a57ff8386a10f8a68ffb9ff204989564d1b",
|
||||
"0014_add_wisp_comments_keyset_index.up.sql": "29ad3654218700481b9541331c714ecbbe96dc81c04dc83fba6bf8d35bed7fa4",
|
||||
"0015_recompute_null_gate_wisp_is_blocked.up.sql": "e761d5c3ceb9610edbb2e1d153e82d235cc7d7f0b87f28a45bec43835e6a4e19",
|
||||
"0016_add_lease_granted_node.up.sql": "31a55421b78d8c90db6cfba5bf8e99e87d6123271575282f761aa077e5d36386",
|
||||
"0017_add_wisps_defer_until_index.up.sql": "1f031aaf2037280e62cbf604748a4157bf25e3a60b3392af7d74835c5059bdd8",
|
||||
"0018_aux_row_id_rekey2_marker.up.sql": "3c2120bf55d15e71a9f96972007c3d914154d17ea37a2b51b44412e3e3df6ef1",
|
||||
"0019_create_events.up.sql": "db072d67a403e60d5bcbe924b84e473adb09f2326d36ede82fcdab17bb1e2a87",
|
||||
"0020_add_wisp_storage_class.up.sql": "9ff476e6c73a3a477aec95143d5cbf180517a9484f647447392576f13181a015",
|
||||
"0021_widen_wisp_large_content_columns.up.sql": "3cfa1e07a3d02eb7f791814d9642a93b9af3bf63728943afa05dcb0e489919c8",
|
||||
"0022_create_events_journal.up.sql": "3d0adcb9f5833b4ba70fb48f88e1043250796feb7173abe000ea4de85eb8b3a5",
|
||||
"0023_repair_events_journal_shape.up.sql": "d0edd83ca4cfbc46bd17799c6497a70c8c011d130d08a00c71daf792ee1dcd51",
|
||||
"0024_widen_wisp_comments_text.up.sql": "69a96330e00643dfeddc4bf6ae9bf3b743a3cfddd86f16dc7b8912758b1b570d",
|
||||
"0025_add_events_journal_actor.up.sql": "c81ca9edc80c25f6ab4f92d05d6ea412a780b108d1ed83745027440e9723dceb",
|
||||
"0026_dep_rekey_dedup_marker.up.sql": "dc5c1b9a74e42184451796068ecba1ca11d8ebd0fe7e8c4bd4900d0cfee30446",
|
||||
}
|
||||
|
||||
func verifyFrozenMigrations(t *testing.T, fsys fs.FS, dir string, frozen map[string]string, frozenThrough int) {
|
||||
t.Helper()
|
||||
entries, err := fs.ReadDir(fsys, dir)
|
||||
if err != nil {
|
||||
t.Fatalf("reading embedded %s: %v", dir, err)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
name := entry.Name()
|
||||
version, err := parseVersion(name)
|
||||
if err != nil {
|
||||
t.Fatalf("unparseable migration filename %q: %v", name, err)
|
||||
}
|
||||
if version > frozenThrough {
|
||||
continue // not yet released — frozen when a release ships it
|
||||
}
|
||||
want, ok := frozen[name]
|
||||
if !ok {
|
||||
t.Errorf("shipped migration %s (v%d <= frozen floor %d) has no pinned hash — a released version was renamed or replaced; restore the original file", name, version, frozenThrough)
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
content, err := fs.ReadFile(fsys, dir+"/"+name)
|
||||
if err != nil {
|
||||
t.Fatalf("reading embedded %s/%s: %v", dir, name, err)
|
||||
}
|
||||
sum := sha256.Sum256(content)
|
||||
if got := hex.EncodeToString(sum[:]); got != want {
|
||||
t.Errorf("shipped migration %s was AMENDED (hash %s, pinned %s) — shipped migrations are frozen; land the change as a new migration instead", name, got, want)
|
||||
}
|
||||
}
|
||||
for name := range frozen {
|
||||
if !seen[name] {
|
||||
t.Errorf("pinned migration %s is missing from the embedded FS — a released version was deleted or renamed", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestShippedMigrationsAreFrozen pins the content of every up-migration that a
|
||||
// release has shipped. See the file comment for the policy.
|
||||
func TestShippedMigrationsAreFrozen(t *testing.T) {
|
||||
verifyFrozenMigrations(t, upMigrations, "migrations", frozenMigrationHashes, frozenMainThrough)
|
||||
}
|
||||
|
||||
// TestShippedIgnoredMigrationsAreFrozen is the ignored-sequence twin. The
|
||||
// ignored tables are clone-local, so an amendment does not fork synced
|
||||
// history — but it DOES make freshly-materialized clones diverge from
|
||||
// long-lived ones, the same class of bug one plane over.
|
||||
func TestShippedIgnoredMigrationsAreFrozen(t *testing.T) {
|
||||
verifyFrozenMigrations(t, upIgnoredMigrations, "migrations/ignored", frozenIgnoredMigrationHashes, frozenIgnoredThrough)
|
||||
}
|
||||
|
|
@ -523,7 +523,8 @@ func (e *RemoteMigrateGateError) AgentDirective() string {
|
|||
}
|
||||
}
|
||||
|
||||
// GateOption is one conditional remediation path for the remote-migrate gate.
|
||||
// GateOption is one conditional remediation path for the remote-migrate gate
|
||||
// (and the migration-consent gate, MigrateConsentError.Options).
|
||||
// It is intentionally conditional (When) and carries its Risk, so an agent
|
||||
// cannot treat any single command as the unconditional fix.
|
||||
type GateOption struct {
|
||||
|
|
|
|||
|
|
@ -269,8 +269,8 @@ type SchemaBehindError struct {
|
|||
}
|
||||
|
||||
func (e *SchemaBehindError) Error() string {
|
||||
return fmt.Sprintf("schema version mismatch: database is at v%d, binary expects v%d, and the read-only open cannot migrate it; run any bd write command in that workspace to migrate, or set BD_IGNORE_SCHEMA_SKEW=1 to read anyway (queries touching newer schema may fail)",
|
||||
e.DBVersion, e.BinaryVersion)
|
||||
return fmt.Sprintf("schema version mismatch: database is at v%d, binary expects v%d; bd does not migrate a database without explicit consent — run `bd migrate schema` in that workspace to migrate, keep using a bd release that matches schema v%d, or set BD_IGNORE_SCHEMA_SKEW=1 to read anyway (queries touching newer schema may fail)",
|
||||
e.DBVersion, e.BinaryVersion, e.DBVersion)
|
||||
}
|
||||
|
||||
// IsSchemaBehindError reports whether err (or any error it wraps) is a
|
||||
|
|
@ -734,6 +734,21 @@ func MigrateUpTo(ctx context.Context, db DBConn, maxVersion int) (int, error) {
|
|||
}
|
||||
|
||||
func MigrateUp(ctx context.Context, db DBConn) (int, error) {
|
||||
// Consent gate first, before ANY write — dolt_ignore seeding included: an
|
||||
// existing database with pending main-sequence migrations must stay
|
||||
// byte-identical when the operator has not consented (see
|
||||
// migrate_consent.go). Fresh and already-current databases pass through.
|
||||
if err := checkMigrateConsent(ctx, db); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return migrateUpConsented(ctx, db)
|
||||
}
|
||||
|
||||
// migrateUpConsented is MigrateUp past its consent gate. It exists for the one
|
||||
// caller whose consent the gate's version test cannot see: an open holding
|
||||
// fresh-bootstrap heal authority, which proves it created the database it is
|
||||
// migrating (see MigrateUpWithLock). Every other caller goes through MigrateUp.
|
||||
func migrateUpConsented(ctx context.Context, db DBConn) (int, error) {
|
||||
// Re-assert the canonical dolt_ignore patterns before anything else, and
|
||||
// in particular before the migrationWorkNeeded short-circuit: a database
|
||||
// whose migration cursors arrived at-latest without executing the seeding
|
||||
|
|
|
|||
|
|
@ -241,7 +241,7 @@ func TestTeamServerMode_Integration(t *testing.T) {
|
|||
require.Error(t, err)
|
||||
assert.Nil(t, p)
|
||||
assert.Contains(t, err.Error(), "bts migrate")
|
||||
assert.NotContains(t, err.Error(), "run any bd write command",
|
||||
assert.NotContains(t, err.Error(), "bd migrate schema",
|
||||
"must not suggest bd-driven migration on a bts-owned schema")
|
||||
|
||||
assert.Equal(t, rolledBack, snapshotMigrations(t, ctx, direct),
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ func checkTeamServerSchema(ctx context.Context, conn schema.DBConn, database str
|
|||
case current < latest:
|
||||
// No BD_IGNORE_SCHEMA_SKEW hatch here: it would let a newer bd write
|
||||
// against an older bts schema. Not SchemaBehindError either: its "run
|
||||
// any bd write command to migrate" advice is wrong for a bts-owned schema.
|
||||
// `bd migrate schema`" advice is wrong for a bts-owned schema.
|
||||
return fmt.Errorf(
|
||||
"uow: database %q is at schema v%d, this bd expects v%d; the schema is managed by beads-team-server — ask your operator to run 'bts migrate', or use a bd built against schema v%d",
|
||||
database, current, latest, current)
|
||||
|
|
|
|||
|
|
@ -108,7 +108,7 @@ func TestCheckTeamServerSchema_Behind_RefusesWithBtsMigrate(t *testing.T) {
|
|||
if !strings.Contains(err.Error(), "bts migrate") {
|
||||
t.Errorf("error %q should point at 'bts migrate'", err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "run any bd write command") {
|
||||
if strings.Contains(err.Error(), "bd migrate schema") {
|
||||
t.Errorf("error %q must not suggest bd-driven migration on a bts-owned schema", err)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import (
|
|||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/doltserver"
|
||||
"github.com/steveyegge/beads/internal/storage/schema"
|
||||
"github.com/steveyegge/beads/internal/testutil"
|
||||
)
|
||||
|
||||
|
|
@ -33,6 +34,12 @@ func TestMain(m *testing.M) {
|
|||
}
|
||||
|
||||
func testMainInner(m *testing.M) int {
|
||||
// Package-wide migration consent: these tests exercise the
|
||||
// provider/bootstrap machinery that runs BELOW the consent gate
|
||||
// (schema/migrate_consent.go), exactly as production does once an operator
|
||||
// has consented.
|
||||
schema.SetLocalMigrateConsent(true)
|
||||
|
||||
// Clear out the roots of earlier runs of this suite whose process is
|
||||
// gone, before claiming one of our own. Roots with no owner marker, and
|
||||
// roots whose owner is still running (a parallel package under
|
||||
|
|
|
|||
|
|
@ -1038,6 +1038,18 @@ run_embedded_dolt_upgrade() {
|
|||
jq -e '.backend == "dolt" and .dolt_mode == "embedded"' "$metadata" >/dev/null ||
|
||||
die "$version: source metadata does not select embedded Dolt"
|
||||
metadata_sha=$(sha256_file "$metadata") || die "$version: could not fingerprint source metadata"
|
||||
# Migration-consent gate (Beads 1.2 remediation): the candidate no longer
|
||||
# migrates an existing database from ordinary commands, so the explicit
|
||||
# schema migration is now the FIRST candidate step — previously it ran
|
||||
# later and was already a no-op because the first candidate open had
|
||||
# silently migrated. Same effective sequence, now stated. Before
|
||||
# consenting, assert the gate actually refuses an ordinary command.
|
||||
if run_in_workspace "$candidate" list >/dev/null 2>"$workspace/consent-refusal.err"; then
|
||||
die "$version: candidate operated an un-migrated database without consent"
|
||||
fi
|
||||
grep -q 'bd migrate schema' "$workspace/consent-refusal.err" ||
|
||||
die "$version: consent refusal did not carry migration guidance"
|
||||
migrate_schema_current "$version" first
|
||||
verify_surviving_fixture "$version" direct
|
||||
if [ "$version" = v1.0.0 ]; then
|
||||
[ "$(run_in_workspace "$candidate" config get status.custom)" = review:active ] || die "$version: candidate did not preserve custom status config"
|
||||
|
|
@ -1056,7 +1068,6 @@ run_embedded_dolt_upgrade() {
|
|||
blocker=$(sed -n '2p' "$workspace/fixture-ids")
|
||||
run_in_workspace "$candidate" show "$blocker" --json > "$workspace/after-first.json"
|
||||
jq -S . "$workspace/after-first.json" > "$workspace/after-first-canonical.json"
|
||||
migrate_schema_current "$version" first
|
||||
verify_idempotent_migration "$version" direct
|
||||
run_in_workspace "$candidate" show "$task" --json --include-comments |
|
||||
jq -S . > "$workspace/direct-after-noops.json"
|
||||
|
|
@ -1692,10 +1703,12 @@ run_wisp_plane_upgrade() {
|
|||
|
||||
# First candidate touch runs the full MigrateUp under the standard per-op
|
||||
# timeout. A dirty-table refusal or a migration error here is the headline
|
||||
# failure class for this lane.
|
||||
# failure class for this lane. The migration-consent gate makes an ordinary
|
||||
# command refuse an un-migrated database, so the explicit schema migration
|
||||
# is that first touch (as in run_embedded_dolt_upgrade); the open follows.
|
||||
migrate_schema_current "$version" wisp-first
|
||||
run_in_workspace "$candidate" list --json -n 0 --all > "$workspace/wisp-first-open.json" ||
|
||||
die "$version: candidate could not open the seeded wisp workspace"
|
||||
migrate_schema_current "$version" wisp-first
|
||||
snapshot_wisp_plane "$version" after
|
||||
|
||||
# Read-only assertions first: the events count must be observed before any
|
||||
|
|
@ -1767,8 +1780,10 @@ run_dirty_tracked_wisp_scenario() {
|
|||
before=$(oracle_count "$version" 'SELECT COUNT(*) FROM wisps')
|
||||
output="$workspace/dirty-tracked-open.out"
|
||||
# run_in_workspace already wraps every call in `timeout`, so a hard-lock
|
||||
# surfaces as a timeout kill rather than hanging the job.
|
||||
if run_in_workspace "$candidate" list --json -n 0 --all > "$output" 2>&1; then
|
||||
# surfaces as a timeout kill rather than hanging the job. The open is the
|
||||
# consented `migrate schema`: an ordinary command now stops at the
|
||||
# migration-consent gate before MigrateUp ever reaches the dirty tables.
|
||||
if run_in_workspace "$candidate" migrate schema > "$output" 2>&1; then
|
||||
printf ' · candidate accepted the dirty tracked wisp tables\n'
|
||||
else
|
||||
grep -Fq 'dirty tables' "$output" ||
|
||||
|
|
|
|||
|
|
@ -223,6 +223,17 @@ cand() { (cd "$WS" && "$CAND_BIN" "$@"); }
|
|||
prev_init() { prev init --quiet --non-interactive --skip-hooks --skip-agents; }
|
||||
cand_init() { cand init --quiet --non-interactive --skip-hooks --skip-agents; }
|
||||
|
||||
# Upgrade an existing workspace to the candidate. Since the migration-consent
|
||||
# gate (Beads 1.2 release remediation) the candidate never migrates an
|
||||
# existing database from ordinary commands — `bd migrate schema` is the
|
||||
# consent step every real upgrade now includes, so the smoke upgrade includes
|
||||
# it too. Tolerates a workspace the old binary could not initialize (the
|
||||
# migrate is a no-op failure there and cand_init creates a fresh database).
|
||||
cand_upgrade() {
|
||||
cand migrate schema >/dev/null 2>&1 || true
|
||||
cand_init 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Create an issue with the previous binary, tolerating missing --silent flag.
|
||||
# Older binaries don't have --silent; we just need to know creation succeeded.
|
||||
# Sets _CREATED_ID to a non-empty value on success.
|
||||
|
|
@ -260,8 +271,27 @@ prev_create --title "Pre-upgrade issue" --type task --priority 1 || true
|
|||
ID1="${_CREATED_ID}"
|
||||
prev_create --title "Another issue" --type bug || true
|
||||
|
||||
# Upgrade: run candidate init (simulates upgrade)
|
||||
cand_init 2>/dev/null || true
|
||||
# Migration-consent gate: before consenting, an ordinary candidate command on
|
||||
# the old-schema database must either work as-is (previous release already
|
||||
# ships the candidate's schema — nothing pending) or refuse WITH the consent
|
||||
# guidance. A failure without the guidance is a regression. Graded only when
|
||||
# the old binary left a database to gate: releases before v1.0 have no
|
||||
# `init --non-interactive`, so prev_init creates nothing there.
|
||||
if ! embedded_db_exists; then
|
||||
pass "Consent gate check skipped (old binary could not initialize a database)"
|
||||
else
|
||||
GATE_OUT=$(cand list 2>&1) && GATE_RC=0 || GATE_RC=$?
|
||||
if [ "$GATE_RC" -eq 0 ]; then
|
||||
pass "Candidate operates database directly (schema already current)"
|
||||
elif echo "$GATE_OUT" | grep -q "bd migrate schema"; then
|
||||
pass "Candidate refuses un-migrated database with consent guidance"
|
||||
else
|
||||
fail "Candidate failed on old database without consent guidance"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Upgrade: consent + candidate init (simulates upgrade)
|
||||
cand_upgrade
|
||||
|
||||
# Verify
|
||||
ROLE=$(git -C "$WS" config --get beads.role 2>/dev/null || echo "MISSING")
|
||||
|
|
@ -307,7 +337,7 @@ prev_init 2>/dev/null || true
|
|||
git -C "$WS" config beads.role contributor
|
||||
|
||||
# Upgrade
|
||||
cand_init 2>/dev/null || true
|
||||
cand_upgrade
|
||||
|
||||
ROLE=$(git -C "$WS" config --get beads.role 2>/dev/null || echo "MISSING")
|
||||
if [ "$ROLE" = "contributor" ]; then
|
||||
|
|
@ -341,7 +371,7 @@ else
|
|||
fi
|
||||
|
||||
# Upgrade with candidate (always runs — verifies candidate defaults to embedded)
|
||||
cand_init 2>/dev/null || true
|
||||
cand_upgrade
|
||||
|
||||
# Verify candidate created an embedded DB
|
||||
if embedded_db_exists; then
|
||||
|
|
@ -411,8 +441,8 @@ if [ -z "${MUT_ID:-}" ]; then
|
|||
else
|
||||
pass "Issue created with previous binary (id: $MUT_ID)"
|
||||
|
||||
# Upgrade: run candidate init
|
||||
cand_init 2>/dev/null || true
|
||||
# Upgrade: consent + candidate init
|
||||
cand_upgrade
|
||||
|
||||
# Mutate using the candidate binary
|
||||
cand update "$MUT_ID" --notes "smoke test mutation" 2>/dev/null || true
|
||||
|
|
@ -479,8 +509,8 @@ if ! $DEP_CREATED; then
|
|||
else
|
||||
pass "Dependency created with previous binary ($BLOCKED_ID depends on $BLOCKER_ID)"
|
||||
|
||||
# Upgrade (runs migrations 0041–0047, including the data-copy).
|
||||
cand_init 2>/dev/null || true
|
||||
# Upgrade (consents to and runs migrations 0041+, including the data-copy).
|
||||
cand_upgrade
|
||||
|
||||
# 1. Blocker queries must not error on the migrated dependency rows.
|
||||
if cand ready >/dev/null 2>&1; then
|
||||
|
|
@ -573,8 +603,8 @@ else
|
|||
pass "Wisp dependency skipped (old binary could not block a wisp on a task)"
|
||||
fi
|
||||
|
||||
# Upgrade.
|
||||
cand_init 2>/dev/null || true
|
||||
# Upgrade: consent + candidate init
|
||||
cand_upgrade
|
||||
|
||||
# 1. The wisp survived the 0054-0066 + ignored 0012-0025 replay. --all
|
||||
# because the default listing hides closed wisps.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue