fix(import): seed issue_prefix from config.yaml before the batch guard needs it (#6624)
* fix(import): seed issue_prefix from config.yaml before the batch guard needs it
NewBatchContext (via ReadConfigPrefix) unconditionally requires an
issue_prefix row in the database's own config table before creating
any issue. bd init writes that row via SetConfig, but a database an
external tool provisioned directly (e.g. an orchestrator that created
the Dolt database via SQL and pointed config.yaml at it without ever
calling bd init) never gets one. Every bd import then failed with
"issue_prefix config is missing", even though config.yaml carried the
prefix and bd where/bd context resolved it correctly.
bd import already had logic to sync config.yaml's prefix into that
table, but only after a successful import — useless when the import
itself can't start without the row. This moves an equivalent, guarded
sync (only fires when the table is empty, never overwrites an existing
prefix) before the import attempt, reading config.yaml directly via
GetStringFromDir rather than the global config singleton, since that
singleton isn't guaranteed to be initialized this early in the import
path.
Added a regression test that reproduces the exact failure with stock
bd (bd init --server, then delete the config-table row and write
config.yaml by hand — no orchestrator involved) and confirms it now
imports successfully.
* fix(import): address review — proxied-mode parity, --global exclusion, GetString
Three issues from @bee-ghosttrack's review of ffb32fb:
1. Proxied-server mode had the identical bug: importer.go's ImportBatch ran
CreateIssuesInTxWithResult (-> NewBatchContext -> ReadConfigPrefix) before
its own SyncIssuePrefix reconciliation, in the same transaction. Seeds the
prefix before create there too, when the table is empty.
2. --global was not excluded from either sync (import.go's existing
post-import sync and this fix's new pre-import seed, plus the proxied
path's SyncIssuePrefix source). config.yaml is per-project; the shared
global store's own prefix must win in --global mode (selectCreateIDPrefix
already established this for `bd create`), or a project's local prefix
silently overwrites the shared database with two dolt commits.
3. GetStringFromDir's rationale didn't hold: config.Initialize() has already
run by main.go's init() and prepareSelectedCommandContext, and
GetStringFromDir skips the config.local.yaml overlay GetString honors —
disagreeing with the sync 40 lines below it. Switched to GetString,
matching import.go:415 and import_proxied_server.go's existing usage.
Smaller fixes: TrimSpace + validatePrefix on the seeded value (a failed
validation now warns instead of writing an invalid prefix); SetConfig/Commit
failures now warn to stderr instead of being silently swallowed.
Test changes:
- Added TestImporterUOW/SeedsPrefixBeforeCreateNeedsIt, exercising the
proxied-mode fix directly at the unit-of-work layer (no CLI/container
overhead) by clearing config-table issue_prefix and confirming
ImportBatch still creates issues. Revert-mutant confirmed: fails with
the identical "issue_prefix config is missing" without the fix.
- Rewrote the E2E test with this repo's server-mode conventions:
skipIfNoDolt, uniqueTestDBName + dropTestDatabase, explicit --server-port
— no longer leaks a database on the shared test server.
- Added TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix (separate
file, cgo-only build tag, using the shared-server harness from
global_identity_integration_test.go): confirms a --global import does not
adopt a project's config.yaml prefix. Revert-mutant confirmed: without the
--global guards, the shared global store's prefix is overwritten by the
local project's ("global" -> "localproj").
go build ./... clean; go vet ./cmd/bd/... (both with and without -tags
"cgo integration") clean; gofmt -l clean on all touched files.
* fix(import): commit the seeded prefix on an empty batch; one resolved prefix
Maintainer review fixup for PR #6624 (iterations 1-2). Addresses the two major
findings and the nearby minors/nits.
MAJOR — the proxied seed was discarded on a batch that landed no rows.
The seed wrote issue_prefix but left ImportBatchResult.PrefixSynced false, so
importBatchCommitMessage produced "" for a batch with Created == 0 and no
memories, RunTxResultWithin skipped uw.Commit on the empty message, and the
deferred closeAttempt ROLLED BACK the seeded row. `bd import` reported success,
the prefix never landed, and a re-run dedup'd again — the state never
converged. Reachable against a missing-prefix database whenever every input row
is dropped by the in-transaction stale guard or the CLI's title dedup. The seed
now reports itself through PrefixSynced, restoring the "the prefix sync runs
even when the batch is otherwise empty" contract (be-llaf) that
import_proxied_server.go's comment promises; the field's doc says so too.
MAJOR — that arm had no coverage in any lane that runs.
TestImporterUOW/SeedsPrefixEvenWhenEveryRowIsRejected pins it: empty stored
prefix + SyncIssuePrefix set + every row stale-rejected => the prefix is
committed, under the sync message, with history moving exactly one entry. It
fails on the pre-fixup tree on all four assertions. The existing
SeedsPrefixBeforeCreateNeedsIt cannot catch this (Created == 1 carries the
commit) and PrefixSyncAloneCommitsUnderTheSyncMessage runs with a non-empty
stored prefix, so its seed never fires. Both are tag-free and container-free,
so unlike the classic-path e2e they execute in ordinary lanes.
Minors and nits:
- Seed errors are returned, not warned. A SetConfig failure here is a
precondition failure for the very next statement: swallowing it surfaced as
the "issue_prefix config is missing" error this seed exists to prevent, and
hid serialization failures from RunTxResult's retry classifier. Drops the
os import.
- Classic path resolves ONE prefix for both its writers (importSyncPrefix;
iteration 2 below extends the same resolver to the proxied path). The
seed used a TrimSpace'd + validatePrefix'd value while the post-import sync
used the raw one, so (a) a prefix the seed refused still reached the database
through the sync, bypassing the guard this PR adds, and (b) a padded
`issue-prefix: " bd"` made the two disagree and the sync overwrote the
trimmed value. TestImportSyncPrefix pins the resolver with no store and no
container; each arm mutation-proved independently. Side effect: the flagged
worst_changed function drops from cyclo 25/cognitive 43 to 22/34.
- Warnings name the whole effect on both paths: an invalid issue-prefix
disables the sync as well as the seed, which base did not do.
- The --global isolation test anchors what its "before" value IS, so the
after == before comparison cannot go vacuous. The anchor is
doltserver.GlobalIssuePrefix, NOT the --prefix passed to init: --prefix names
the project's prefix and `bd init --global` always writes the constant
(init.go), which an unanchored equality had been hiding.
- Comments record two boundaries the diff left unstated: the classic seed
commits before an import that may fail (deferring it is not an option — the
post-import CommitWithConfig is gated on dbPrefix != syncPrefix, which a seed
makes false, so the row would stay uncommitted because store.Commit excludes
config (GH#2455), and a dirty internal config key blocks the next pull); and
a --global import against a global database missing its own
config.issue_prefix stays unseedable by design.
Iteration 2 — three non-gating findings (both majors above stayed fixed; the
fresh synthesis and quality scorecard both say approve):
- The classic seed now runs BEFORE the memory writes. CommitWithConfig is
DOLT_COMMIT -Am, so it stages everything dirty at that instant: with the seed
below the memory loop, a `bd import` carrying memories against a
missing-prefix database whose issue phase then failed left those kv.memory.*
rows PERMANENTLY COMMITTED, under a message asserting the commit is the
prefix seed — where at base the failure left the whole working set for the
operator to discard. Seeding first restores "the only thing this commit can
carry is the prefix". Pure statement reorder: the seed reads only issues,
syncPrefix and the config table, and the ordering is now recorded as
load-bearing at the block so a later edit does not undo it.
- The proxied path CALLS importSyncPrefix() instead of restating its four rules
inline. The two copies were identical rule for rule — same globalFlag guard,
same TrimSpace, same validatePrefix, byte-identical warning — but only the
classic copy was pinned, so the underscore-key fallback ReadConfigPrefix
honors, or hyphen normalization, could have reintroduced ACROSS import modes
exactly the divergence the bullet above closed within one mode. One
implementation now, so TestImportSyncPrefix's six rows govern both routes.
Drops the os, strings and config imports from import_proxied_server.go.
- The uow subtests no longer lean on suite order.
SeedsPrefixEvenWhenEveryRowIsRejected snapshots the sync-message history
count before the batch and asserts +1: the old `n != 0` form could not fail
in a full-suite run, because PrefixSyncAloneCommitsUnderTheSyncMessage has
already committed under that message by then. Mutation-proved — routing an
all-rejected batch to a "0 issues from ..." message reds only the new
assertion ("moved 1 -> 1"), and the 1 in that output is precisely the count
that would have satisfied the old form. SeedsPrefixBeforeCreateNeedsIt also
restores issue_prefix the way its two neighbours do, instead of leaving the
fixture to be restored by the code under test.
The PR description's stale "GetStringFromDir / config.GetString returned empty
in this exact spot" claim and its "single-file production change" scope claim
were corrected on GitHub, and the CI-lane reality for each added test is now
recorded there.
Validation (iteration 1): gofmt clean; go build ./...; go vet on cmd/bd,
internal/storage/uow and issueops; full internal/storage/uow package green
(570s, under a private temp root); focused cmd/bd import tests,
TestImportSyncPrefix, TestEmbeddedImport/prefix_sync{,_no_config_key} and the
--global isolation e2e all green.
Validation (iteration 2): gofmt and `git diff --check` clean; go build
./cmd/... ./internal/storage/uow/...; go vet on cmd/bd and
internal/storage/uow; full TestImporterUOW green in subtest order (9/9) plus
the mutation A/B above; TestImportSyncPrefix green (6/6). Two reds in the wider
`go test -run Import ./cmd/bd` are environmental on this box and reproduce
identically with the fix set stashed: TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix
dies inside `bd init` because the cloned remote is one schema version behind
this build (v66 -> v67), and TestImportBlockedDuringMigrationFreeze cannot
start its server (port 3307 busy).
---------
Co-authored-by: ilasic <ilasic@reversinglabs.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
This commit is contained in:
parent
da6263374d
commit
48b9e904dc
8 changed files with 558 additions and 9 deletions
|
|
@ -356,6 +356,42 @@ func runImportRecordsClassic(ctx context.Context, issues []*types.Issue, memorie
|
|||
return renderImportDryRun(result, len(memories), source, dedupHits)
|
||||
}
|
||||
|
||||
// ONE value for both prefix writers on this path — the seed just below and
|
||||
// the post-import sync at the bottom. Resolved once so they cannot disagree
|
||||
// on trimming or on validation; see importSyncPrefix.
|
||||
syncPrefix := importSyncPrefix()
|
||||
|
||||
// Seed issue_prefix from config.yaml before the config table has one, so
|
||||
// NewBatchContext's ReadConfigPrefix (below, via importIssuesCore) does
|
||||
// not reject an externally-provisioned database that config.yaml already
|
||||
// names a prefix for.
|
||||
//
|
||||
// It runs BEFORE the memory writes below, deliberately: CommitWithConfig
|
||||
// is DOLT_COMMIT -Am, so it stages everything dirty at that instant.
|
||||
// Seeding first is what keeps the prefix the only thing this commit can
|
||||
// carry — otherwise this run's kv.memory.* rows land in a commit named for
|
||||
// the prefix seed, and stay permanently committed when the import below
|
||||
// then fails, instead of being left in the working set for the operator to
|
||||
// discard.
|
||||
//
|
||||
// The commit lands HERE, before an import that may fail, which means a
|
||||
// failed `bd import` can leave the seeded prefix behind. Accepted: the
|
||||
// value is config.yaml's own and is the converged one either way, so the
|
||||
// residue is the state the next successful import would reach. Deferring
|
||||
// it to the post-import CommitWithConfig below is NOT an alternative —
|
||||
// that block only fires when dbPrefix != syncPrefix, which a seed makes
|
||||
// false, so the row would stay uncommitted (store.Commit excludes config,
|
||||
// GH#2455) and a dirty internal config key then blocks the next pull.
|
||||
if len(issues) > 0 && syncPrefix != "" {
|
||||
if dbPrefix, _ := store.GetConfig(ctx, "issue_prefix"); dbPrefix == "" {
|
||||
if setErr := store.SetConfig(ctx, "issue_prefix", syncPrefix); setErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to seed issue_prefix from config.yaml: %v\n", setErr)
|
||||
} else if commitErr := store.CommitWithConfig(ctx, "bd import: seed issue_prefix from config.yaml"); commitErr != nil {
|
||||
fmt.Fprintf(os.Stderr, "warning: failed to commit seeded issue_prefix: %v\n", commitErr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Import memories
|
||||
for _, mem := range memories {
|
||||
storageKey := kvPrefix + memoryPrefix + mem.Key
|
||||
|
|
@ -396,10 +432,13 @@ func runImportRecordsClassic(ctx context.Context, issues []*types.Issue, memorie
|
|||
// for this intentional config update after the issues commit completes.
|
||||
// config.yaml is authoritative here and existing issue IDs are intentionally
|
||||
// left unchanged: this deliberately bypasses the `bd config set issue_prefix`
|
||||
// guard for the import/migration flow and is not a rename.
|
||||
if yamlPrefix := config.GetString("issue-prefix"); yamlPrefix != "" {
|
||||
if dbPrefix, _ := store.GetConfig(ctx, "issue_prefix"); dbPrefix != yamlPrefix {
|
||||
if setErr := store.SetConfig(ctx, "issue_prefix", yamlPrefix); setErr == nil {
|
||||
// guard for the import/migration flow and is not a rename. Reconciles
|
||||
// against the SAME resolved value the seed used, so a padded config.yaml
|
||||
// prefix cannot make this test true forever and overwrite the trimmed
|
||||
// value the seed stored on every import.
|
||||
if syncPrefix != "" {
|
||||
if dbPrefix, _ := store.GetConfig(ctx, "issue_prefix"); dbPrefix != syncPrefix {
|
||||
if setErr := store.SetConfig(ctx, "issue_prefix", syncPrefix); setErr == nil {
|
||||
_ = store.CommitWithConfig(ctx, "bd import: sync issue_prefix from config.yaml")
|
||||
}
|
||||
}
|
||||
|
|
@ -408,6 +447,38 @@ func runImportRecordsClassic(ctx context.Context, issues []*types.Issue, memorie
|
|||
return renderImportOutcome(result, source, dedupHits)
|
||||
}
|
||||
|
||||
// importSyncPrefix resolves the ONE config.yaml issue_prefix value EVERY
|
||||
// import writer uses: the classic path's seed (before the batch needs it) and
|
||||
// its post-import sync (after the batch lands), and the SyncIssuePrefix the
|
||||
// proxied path hands ImportBatch (import_proxied_server.go), which seeds and
|
||||
// syncs inside its one transaction. "" means "reconcile nothing", which is
|
||||
// what --global, an absent value, and an invalid value all resolve to.
|
||||
//
|
||||
// Resolving once is what keeps those writers honest: the seed stored the
|
||||
// TRIMMED value while the sync used to compare the raw one, so a padded
|
||||
// `issue-prefix: " bd"` made them disagree and the sync overwrote the seed's
|
||||
// value; and the sync used to skip validatePrefix entirely, so a prefix the
|
||||
// seed refused still reached the database through it.
|
||||
func importSyncPrefix() string {
|
||||
// --global: config.yaml is a per-project file, and the shared global
|
||||
// store's own prefix must win (selectCreateIDPrefix), not whatever project
|
||||
// happened to import into it.
|
||||
if globalFlag {
|
||||
return ""
|
||||
}
|
||||
yamlPrefix := strings.TrimSpace(config.GetString("issue-prefix"))
|
||||
if yamlPrefix == "" {
|
||||
return ""
|
||||
}
|
||||
if err := validatePrefix(yamlPrefix); err != nil {
|
||||
// Naming both effects: an invalid value disables the sync as well as
|
||||
// the seed, and base did sync an invalid prefix through.
|
||||
fmt.Fprintf(os.Stderr, "warning: ignoring invalid issue-prefix %q from config.yaml (not seeding or syncing issue_prefix): %v\n", yamlPrefix, err)
|
||||
return ""
|
||||
}
|
||||
return yamlPrefix
|
||||
}
|
||||
|
||||
// applyImportDryRunClassification folds a dry-run classification into the
|
||||
// command's JSON result, identically in both modes.
|
||||
func applyImportDryRunClassification(result *importResultJSON, classification *ImportResult) {
|
||||
|
|
|
|||
125
cmd/bd/import_global_prefix_isolation_test.go
Normal file
125
cmd/bd/import_global_prefix_isolation_test.go
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
//go:build cgo
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/steveyegge/beads/internal/doltserver"
|
||||
"github.com/steveyegge/beads/internal/testutil"
|
||||
)
|
||||
|
||||
// TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix pins that `bd
|
||||
// import --global` does not seed or sync issue_prefix from the current
|
||||
// project's config.yaml: the shared global store's own prefix must win in
|
||||
// --global mode (selectCreateIDPrefix), the same rule bd create already
|
||||
// follows. A project's config.yaml leaking into the global store would let
|
||||
// whichever project last ran a global import silently repoint every other
|
||||
// project's global issue IDs. Uses the shared-server harness from
|
||||
// global_identity_integration_test.go — --global requires shared-server mode.
|
||||
func TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("not supported on Windows")
|
||||
}
|
||||
|
||||
bdBinary := buildSharedServerTestBinary(t)
|
||||
|
||||
cp, err := testutil.NewContainerProvider()
|
||||
if err != nil {
|
||||
t.Skipf("skipping: Dolt container not available: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cp.Stop() })
|
||||
|
||||
sharedDir := t.TempDir()
|
||||
if err := cp.WritePortFile(sharedDir); err != nil {
|
||||
t.Fatalf("write port file: %v", err)
|
||||
}
|
||||
|
||||
projectDir := filepath.Join(t.TempDir(), "localproj")
|
||||
if err := os.MkdirAll(projectDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir project dir: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
if err := gitInit(ctx, projectDir); err != nil {
|
||||
t.Fatalf("git init: %v", err)
|
||||
}
|
||||
|
||||
env := []string{
|
||||
"PATH=" + os.Getenv("PATH"),
|
||||
"HOME=" + t.TempDir(),
|
||||
"GOPATH=" + os.Getenv("GOPATH"),
|
||||
"GOROOT=" + os.Getenv("GOROOT"),
|
||||
"BEADS_SHARED_SERVER_DIR=" + sharedDir,
|
||||
"BEADS_DOLT_SHARED_SERVER=1",
|
||||
"BEADS_DOLT_SERVER_PORT=" + strconv.Itoa(cp.Port()),
|
||||
"BEADS_DOLT_AUTO_START=0",
|
||||
"BEADS_TEST_MODE=1",
|
||||
"BD_DISABLE_METRICS=1",
|
||||
"BD_DISABLE_EVENT_FLUSH=1",
|
||||
"GIT_TERMINAL_PROMPT=0",
|
||||
"GIT_ASKPASS=",
|
||||
"SSH_ASKPASS=",
|
||||
"GT_ROOT=",
|
||||
}
|
||||
|
||||
initArgs := []string{"init", "--shared-server", "--global", "--external", "--prefix", "globalinit", "--quiet", "--non-interactive"}
|
||||
if out, err := ssExec(ctx, bdBinary, projectDir, env, initArgs...); err != nil {
|
||||
t.Fatalf("bd %s (global init) failed: %v\noutput:\n%s", strings.Join(initArgs, " "), err, out)
|
||||
}
|
||||
|
||||
// bd init (non-no-db) stores the prefix in the DB, not config.yaml; write
|
||||
// a DIFFERENT project-local prefix into config.yaml directly, the same
|
||||
// way global_identity_integration_test.go exercises the YAML-first
|
||||
// prefix-selection path (selectCreateIDPrefix) that bd-4646 regressed:
|
||||
// this is what a project's config.yaml looks like in practice, and
|
||||
// --global mode must not read it.
|
||||
beadsDir := filepath.Join(projectDir, ".beads")
|
||||
if err := os.WriteFile(filepath.Join(beadsDir, "config.yaml"), []byte("issue-prefix: localproj\n"), 0o644); err != nil {
|
||||
t.Fatalf("write config.yaml issue-prefix: %v", err)
|
||||
}
|
||||
|
||||
globalOut, err := ssExec(ctx, bdBinary, projectDir, env, "config", "get", "issue_prefix", "--global")
|
||||
if err != nil {
|
||||
t.Fatalf("bd config get issue_prefix --global (before import) failed: %v\n%s", err, globalOut)
|
||||
}
|
||||
globalPrefixBefore := strings.TrimSpace(globalOut)
|
||||
// Anchor what `before` IS, so the after == before comparison below cannot
|
||||
// go vacuous: if this read ever stops resolving the global store's row it
|
||||
// returns "" or the "issue_prefix (not set)" line (cmd/bd/config.go), and
|
||||
// an unanchored after == before would then pass while measuring nothing.
|
||||
//
|
||||
// The expected value is doltserver.GlobalIssuePrefix, NOT the --prefix
|
||||
// passed to init above: --prefix names the PROJECT's prefix, and bd init
|
||||
// --global always writes the constant into the shared store (init.go).
|
||||
if globalPrefixBefore != doltserver.GlobalIssuePrefix {
|
||||
t.Fatalf("global issue_prefix before import = %q, want %q — the --global config read is not resolving the global store's row", globalPrefixBefore, doltserver.GlobalIssuePrefix)
|
||||
}
|
||||
|
||||
issue := `{"id":"globaltest-1","title":"Global import test","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-01T00:00:00Z"}`
|
||||
jsonlPath := filepath.Join(projectDir, "global.jsonl")
|
||||
if err := os.WriteFile(jsonlPath, []byte(issue+"\n"), 0644); err != nil {
|
||||
t.Fatalf("failed to write JSONL fixture: %v", err)
|
||||
}
|
||||
|
||||
if out, err := ssExec(ctx, bdBinary, projectDir, env, "import", "--global", "-i", "global.jsonl"); err != nil {
|
||||
t.Fatalf("bd import --global failed: %v\n%s", err, out)
|
||||
}
|
||||
|
||||
afterOut, err := ssExec(ctx, bdBinary, projectDir, env, "config", "get", "issue_prefix", "--global")
|
||||
if err != nil {
|
||||
t.Fatalf("bd config get issue_prefix --global (after import) failed: %v\n%s", err, afterOut)
|
||||
}
|
||||
if got := strings.TrimSpace(afterOut); got != doltserver.GlobalIssuePrefix {
|
||||
t.Fatalf("global issue_prefix changed from %q to %q — a --global import must not adopt the local project's config.yaml prefix (localproj)", globalPrefixBefore, got)
|
||||
}
|
||||
}
|
||||
96
cmd/bd/import_missing_config_prefix_test.go
Normal file
96
cmd/bd/import_missing_config_prefix_test.go
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
//go:build cgo && integration
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestCLI_Import_ServerDatabaseMissingConfigPrefix_E2E pins that `bd import`
|
||||
// succeeds against a server-mode database whose `config` table has no
|
||||
// issue_prefix row, as long as config.yaml carries one.
|
||||
//
|
||||
// This is the state an externally-provisioned database is in: config.yaml
|
||||
// carries issue-prefix (written directly by the provisioner, not via `bd
|
||||
// init`), but the database's own config table has no issue_prefix row (only
|
||||
// `bd init`'s SetConfig call writes one). No orchestrator is involved in
|
||||
// reproducing this — the test simulates it directly by deleting the
|
||||
// config-table row and writing config.yaml by hand, exactly like
|
||||
// dolt_metadata_e2e_test.go simulates a pre-Phase-1 database.
|
||||
//
|
||||
// Before the fix, NewBatchContext (via ReadConfigPrefix) required this row
|
||||
// unconditionally, so `bd import` failed with "issue_prefix config is
|
||||
// missing" even though config.yaml had the prefix and `bd where`/`bd
|
||||
// context` resolved it correctly.
|
||||
func TestCLI_Import_ServerDatabaseMissingConfigPrefix_E2E(t *testing.T) {
|
||||
skipIfNoDolt(t)
|
||||
|
||||
tmpDir := t.TempDir()
|
||||
env := os.Environ()
|
||||
database := uniqueTestDBName(t)
|
||||
t.Cleanup(func() {
|
||||
dropTestDatabase(database, testDoltServerPort)
|
||||
})
|
||||
|
||||
initOut, initErr := runBDExecAllowErrorWithEnv(t, tmpDir, env,
|
||||
"init", "--backend", "dolt", "--server", "--external",
|
||||
"--server-host", "127.0.0.1",
|
||||
"--server-port", fmt.Sprintf("%d", testDoltServerPort),
|
||||
"--database", database,
|
||||
"--prefix", "extdb", "--quiet")
|
||||
if initErr != nil {
|
||||
t.Fatalf("bd init --server failed: %v\n%s", initErr, initOut)
|
||||
}
|
||||
|
||||
// Delete the config-table row bd init just wrote, and write issue-prefix
|
||||
// into config.yaml directly (bd init does not do this on its own — an
|
||||
// external provisioner writing config.yaml by hand, as an orchestrator
|
||||
// would, is what puts a project in this state: config.yaml has the
|
||||
// prefix, the database's own config table does not).
|
||||
sqlOut, sqlErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "sql",
|
||||
"DELETE FROM config WHERE `key` = 'issue_prefix'")
|
||||
if sqlErr != nil {
|
||||
t.Fatalf("bd sql DELETE failed: %v\n%s", sqlErr, sqlOut)
|
||||
}
|
||||
|
||||
configPath := tmpDir + "/.beads/config.yaml"
|
||||
configBytes, err := os.ReadFile(configPath)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to read config.yaml: %v", err)
|
||||
}
|
||||
configBytes = append(configBytes, []byte("\nissue-prefix: extdb\n")...)
|
||||
if err := os.WriteFile(configPath, configBytes, 0644); err != nil {
|
||||
t.Fatalf("failed to write issue-prefix into config.yaml: %v", err)
|
||||
}
|
||||
|
||||
// Confirm the precondition: reads that need the config-table prefix now
|
||||
// fail, matching the reported bug's symptom.
|
||||
precheckOut, precheckErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "create", "--title", "should fail before fix")
|
||||
if precheckErr == nil {
|
||||
t.Fatalf("expected bd create to fail with the config-table row deleted, but it succeeded: %s", precheckOut)
|
||||
}
|
||||
if !strings.Contains(precheckOut, "issue_prefix config is missing") {
|
||||
t.Fatalf("expected 'issue_prefix config is missing', got: %s", precheckOut)
|
||||
}
|
||||
|
||||
issue := `{"id":"extdb-1","title":"Externally provisioned issue","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-01T00:00:00Z"}`
|
||||
if err := os.WriteFile(tmpDir+"/external.jsonl", []byte(issue+"\n"), 0644); err != nil {
|
||||
t.Fatalf("failed to write JSONL fixture: %v", err)
|
||||
}
|
||||
|
||||
importOut, importErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "import", "-i", "external.jsonl")
|
||||
if importErr != nil {
|
||||
t.Fatalf("bd import failed against a server-mode database missing its config-table prefix: %v\nOutput: %s", importErr, importOut)
|
||||
}
|
||||
|
||||
listOut, listErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "list", "--id", "extdb-1", "--json")
|
||||
if listErr != nil {
|
||||
t.Fatalf("bd list failed: %v\n%s", listErr, listOut)
|
||||
}
|
||||
if !strings.Contains(listOut, "extdb-1") {
|
||||
t.Fatalf("expected extdb-1 to be imported, but list output was: %s", listOut)
|
||||
}
|
||||
}
|
||||
|
|
@ -6,7 +6,6 @@ import (
|
|||
"fmt"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/steveyegge/beads/internal/config"
|
||||
"github.com/steveyegge/beads/internal/storage/uow"
|
||||
"github.com/steveyegge/beads/internal/types"
|
||||
publicops "github.com/steveyegge/beads/issueops"
|
||||
|
|
@ -128,17 +127,34 @@ func runImportRecordsProxied(ctx context.Context, issues []*types.Issue, memorie
|
|||
return err
|
||||
}
|
||||
|
||||
// The ONE resolver both import paths share for config.yaml's issue_prefix
|
||||
// (importSyncPrefix, import.go): the --global exclusion, the single trim,
|
||||
// validatePrefix, the warning that names BOTH effects, and "" meaning
|
||||
// "reconcile nothing". Calling it instead of restating its rules here is
|
||||
// what keeps the two modes from drifting — a second hand-maintained copy
|
||||
// of this policy is the very divergence one resolver exists to prevent,
|
||||
// and the resolver's table test then covers this route too.
|
||||
//
|
||||
// Accepted boundary: a --global import against an externally-provisioned
|
||||
// GLOBAL database that is itself missing config.issue_prefix stays
|
||||
// unseedable and still fails with "issue_prefix config is missing" — that
|
||||
// row is `bd init --global`'s to write, and config.yaml cannot speak for
|
||||
// the shared store.
|
||||
syncPrefix := importSyncPrefix()
|
||||
|
||||
// THE BATCH: rows, memories and the issue_prefix reconciliation in one
|
||||
// transaction, one history entry. The prefix sync runs even when the
|
||||
// batch is otherwise empty, exactly as the classic path's post-commit
|
||||
// sync does (be-llaf; config.yaml is authoritative, not a rename).
|
||||
// sync does (be-llaf; config.yaml is authoritative, not a rename) — and
|
||||
// so does the seed, which reports itself through the same PrefixSynced
|
||||
// result so an otherwise-empty batch still commits it.
|
||||
batch, err := importer.ImportBatch(ctx, publicops.ImportBatchRequest{
|
||||
Actor: getActorWithGit(),
|
||||
Issues: issues,
|
||||
Memories: memoryEntries,
|
||||
AllowStale: importAllowStale,
|
||||
SkipPrefixValidation: true,
|
||||
SyncIssuePrefix: config.GetString("issue-prefix"),
|
||||
SyncIssuePrefix: syncPrefix,
|
||||
Source: filepath.Base(source),
|
||||
})
|
||||
if err != nil {
|
||||
|
|
|
|||
89
cmd/bd/import_sync_prefix_test.go
Normal file
89
cmd/bd/import_sync_prefix_test.go
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/config"
|
||||
)
|
||||
|
||||
// TestImportSyncPrefix pins the ONE value runImportRecordsClassic hands to both
|
||||
// of its issue_prefix writers — the pre-batch seed and the post-batch sync.
|
||||
//
|
||||
// It is deliberately a plain unit test: no store, no Dolt binary, no container.
|
||||
// The classic seed's end-to-end cases (import_missing_config_prefix_test.go,
|
||||
// import_embedded_test.go) are gated behind a Dolt server or
|
||||
// BEADS_TEST_EMBEDDED_DOLT and skip under the BEADS_TEST_SKIP=dolt the required
|
||||
// integration lanes set, so the resolver's rules are asserted here instead,
|
||||
// where every lane runs them.
|
||||
func TestImportSyncPrefix(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
yaml string
|
||||
global bool
|
||||
want string
|
||||
}{
|
||||
{name: "absent is reconcile nothing", yaml: "", want: ""},
|
||||
{name: "plain value passes through", yaml: "bd", want: "bd"},
|
||||
{
|
||||
// Both writers must see the SAME trimmed value. When the seed
|
||||
// stored the trimmed one and the sync compared the raw one, the
|
||||
// sync found them unequal and overwrote the seed's value with the
|
||||
// padded one.
|
||||
name: "padded value is trimmed for both writers",
|
||||
yaml: " bd\t",
|
||||
want: "bd",
|
||||
},
|
||||
{
|
||||
// The guard covers the sync too, not just the seed: an invalid
|
||||
// prefix reconciles nothing at all.
|
||||
name: "invalid value reconciles nothing",
|
||||
yaml: "BAD",
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
// Trailing hyphens are NOT an invalid prefix: validatePrefix
|
||||
// TrimRight's them before matching, which is the same tolerance
|
||||
// `bd rename-prefix` has. The resolver defers to that guard rather
|
||||
// than inventing a stricter rule of its own, so the value reaches
|
||||
// both writers unchanged.
|
||||
name: "trailing hyphens are tolerated by the shared guard",
|
||||
yaml: "bd--",
|
||||
want: "bd--",
|
||||
},
|
||||
{
|
||||
// --global: config.yaml is per-project and must not speak for the
|
||||
// shared global store, whatever it says.
|
||||
name: "global mode ignores config.yaml entirely",
|
||||
yaml: "localproj",
|
||||
global: true,
|
||||
want: "",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
// Isolate CWD and BEADS_DIR before config.Initialize(): a leaked
|
||||
// BEADS_DIR or a cwd inside the repo tree lets Initialize() load a
|
||||
// real ambient config.yaml whose issue-prefix would win over the
|
||||
// value under test.
|
||||
t.Chdir(t.TempDir())
|
||||
t.Setenv("BEADS_DIR", "")
|
||||
t.Setenv("BEADS_TEST_IGNORE_REPO_CONFIG", "1")
|
||||
|
||||
config.ResetForTesting()
|
||||
t.Cleanup(func() { config.ResetForTesting() })
|
||||
if err := config.Initialize(); err != nil {
|
||||
t.Fatalf("config.Initialize: %v", err)
|
||||
}
|
||||
config.Set("issue-prefix", tt.yaml)
|
||||
|
||||
origGlobal := globalFlag
|
||||
globalFlag = tt.global
|
||||
t.Cleanup(func() { globalFlag = origGlobal })
|
||||
|
||||
if got := importSyncPrefix(); got != tt.want {
|
||||
t.Errorf("importSyncPrefix() with issue-prefix %q (global=%v) = %q, want %q", tt.yaml, tt.global, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -53,6 +53,35 @@ func (o *importer) ImportBatch(ctx context.Context, request publicops.ImportBatc
|
|||
return RunTxResult(ctx, o.provider, func(ctx context.Context, uw UnitOfWork) (publicops.ImportBatchResult, string, error) {
|
||||
var result publicops.ImportBatchResult
|
||||
|
||||
// Seed issue_prefix before CreateIssuesInTxWithResult needs it (via
|
||||
// NewBatchContext -> ReadConfigPrefix), for the same reason the sync
|
||||
// below exists: an externally-provisioned database's config table can
|
||||
// be missing the row config.yaml already carries. Seeding only when
|
||||
// the table is empty — never overwriting an existing prefix — keeps
|
||||
// this a pure precondition for the sync 40 lines down, not a second,
|
||||
// conflicting writer.
|
||||
//
|
||||
// PrefixSynced is set HERE and not left to the sync: once the seed has
|
||||
// written the value, the sync's `stored != request.SyncIssuePrefix`
|
||||
// test is false and it reports nothing, so a batch that lands no rows
|
||||
// and no memories would build an empty commit message and RunTxResult
|
||||
// would roll the seeded row back (tx.go, importBatchCommitMessage).
|
||||
// The seed IS the prefix write in that case and has to say so.
|
||||
//
|
||||
// Unlike the sync, a failure here does NOT degrade to "not synced":
|
||||
// this write is the precondition for the very next statement, so
|
||||
// swallowing it would surface as the `issue_prefix config is missing`
|
||||
// error this seed exists to prevent, and would hide a serialization
|
||||
// failure from RunTxResult's retry classifier.
|
||||
if request.SyncIssuePrefix != "" {
|
||||
if stored, _ := uw.ConfigUseCase().GetConfig(ctx, "issue_prefix"); stored == "" {
|
||||
if err := uw.ConfigUseCase().SetConfig(ctx, "issue_prefix", request.SyncIssuePrefix); err != nil {
|
||||
return publicops.ImportBatchResult{}, "", fmt.Errorf("seed issue_prefix: %w", err)
|
||||
}
|
||||
result.PrefixSynced = true
|
||||
}
|
||||
}
|
||||
|
||||
if len(request.Issues) > 0 {
|
||||
runner, err := importStatementRunner(uw)
|
||||
if err != nil {
|
||||
|
|
|
|||
|
|
@ -40,6 +40,14 @@ func TestImporterUOW(t *testing.T) {
|
|||
}
|
||||
return n
|
||||
}
|
||||
countHistoryMatching := func(t *testing.T, pattern string) int {
|
||||
t.Helper()
|
||||
n, err := kit.CountHistoryMatching(ctx, pattern)
|
||||
if err != nil {
|
||||
t.Fatalf("CountHistoryMatching %q: %v", pattern, err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
queryInt := func(t *testing.T, query string, args ...any) int {
|
||||
t.Helper()
|
||||
var n int
|
||||
|
|
@ -261,6 +269,120 @@ func TestImporterUOW(t *testing.T) {
|
|||
}
|
||||
})
|
||||
|
||||
t.Run("SeedsPrefixBeforeCreateNeedsIt", func(t *testing.T) {
|
||||
// Simulate an externally-provisioned database: config.yaml (modeled
|
||||
// here by SyncIssuePrefix, which is what the CLI populates it from)
|
||||
// carries a prefix, but the database's own config table does not —
|
||||
// exactly what NewBatchContext's ReadConfigPrefix treats as "missing"
|
||||
// (sql.ErrNoRows OR an empty value), the state a provisioner that
|
||||
// creates the Dolt database directly (bypassing `bd init`, which is
|
||||
// what normally writes this row) leaves it in.
|
||||
if err := kit.SetConfig(ctx, "issue_prefix", ""); err != nil {
|
||||
t.Fatalf("clear issue_prefix: %v", err)
|
||||
}
|
||||
|
||||
when := time.Date(2026, 8, 2, 12, 0, 0, 0, time.UTC)
|
||||
result, err := imp.ImportBatch(ctx, publicops.ImportBatchRequest{
|
||||
Actor: "importer-test",
|
||||
Issues: []*types.Issue{{
|
||||
ID: "imp-seed", Title: "Seeded via prefix precondition", Status: types.StatusOpen,
|
||||
IssueType: types.TypeTask, Priority: 2, CreatedAt: when, UpdatedAt: when,
|
||||
}},
|
||||
SkipPrefixValidation: true,
|
||||
SyncIssuePrefix: "imp",
|
||||
Source: "seed.jsonl",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("ImportBatch against a database with no config-table prefix: %v", err)
|
||||
}
|
||||
if result.Created != 1 {
|
||||
t.Errorf("Created = %d, want 1", result.Created)
|
||||
}
|
||||
if got := queryString(t, "SELECT value FROM config WHERE `key` = 'issue_prefix'"); got != "imp" {
|
||||
t.Errorf("issue_prefix after seed = %q, want %q", got, "imp")
|
||||
}
|
||||
// Restore for any later subtest, as the neighbours do. The seed under
|
||||
// test happens to leave "imp" behind, but a fixture restored by the
|
||||
// code under test would silently change what a subtest inserted after
|
||||
// this one runs against.
|
||||
if err := kit.SetConfig(ctx, "issue_prefix", "imp"); err != nil {
|
||||
t.Fatalf("restore issue_prefix: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("SeedsPrefixEvenWhenEveryRowIsRejected", func(t *testing.T) {
|
||||
// The seed must report itself as the prefix write, or a batch that
|
||||
// lands no rows commits NOTHING and discards it:
|
||||
// importBatchCommitMessage returns "" when Created == 0,
|
||||
// MemoriesImported == 0 and !PrefixSynced, RunTxResultWithin skips
|
||||
// uw.Commit on an empty message (tx.go), and the deferred closeAttempt
|
||||
// ROLLS BACK the seeded row. Reachable against an
|
||||
// externally-provisioned database whenever every input row is dropped
|
||||
// — here by the in-transaction stale guard, in the CLI also by the
|
||||
// title dedup. SeedsPrefixBeforeCreateNeedsIt cannot catch this
|
||||
// (Created == 1 carries the commit), and
|
||||
// PrefixSyncAloneCommitsUnderTheSyncMessage runs with a NON-empty
|
||||
// stored prefix, so the sync writes there and the seed never fires.
|
||||
landed := time.Date(2026, 8, 3, 12, 0, 0, 0, time.UTC)
|
||||
if _, err := imp.ImportBatch(ctx, publicops.ImportBatchRequest{
|
||||
Actor: "importer-test",
|
||||
Issues: []*types.Issue{{
|
||||
ID: "imp-rejected", Title: "Row the stale guard will keep", Status: types.StatusOpen,
|
||||
IssueType: types.TypeTask, Priority: 2, CreatedAt: landed, UpdatedAt: landed,
|
||||
}},
|
||||
Source: "reject-setup.jsonl",
|
||||
}); err != nil {
|
||||
t.Fatalf("seed the row the stale guard will keep: %v", err)
|
||||
}
|
||||
if err := kit.SetConfig(ctx, "issue_prefix", ""); err != nil {
|
||||
t.Fatalf("clear issue_prefix: %v", err)
|
||||
}
|
||||
|
||||
// Both counts are snapshotted: TestImporterUOW shares one database in
|
||||
// subtest order, and PrefixSyncAloneCommitsUnderTheSyncMessage has
|
||||
// already committed under the sync message by the time this runs — so
|
||||
// only a DELTA can tell that the seed's own commit carries it.
|
||||
const syncMessageLike = "%sync issue_prefix from config.yaml%"
|
||||
before := countHistory(t)
|
||||
syncNamedBefore := countHistoryMatching(t, syncMessageLike)
|
||||
stale := landed.Add(-24 * time.Hour)
|
||||
result, err := imp.ImportBatch(ctx, publicops.ImportBatchRequest{
|
||||
Actor: "importer-test",
|
||||
Issues: []*types.Issue{{
|
||||
ID: "imp-rejected", Title: "Stale snapshot", Status: types.StatusOpen,
|
||||
IssueType: types.TypeTask, Priority: 3, CreatedAt: stale, UpdatedAt: stale,
|
||||
}},
|
||||
SkipPrefixValidation: true,
|
||||
SyncIssuePrefix: "impseed",
|
||||
Source: "reject.jsonl",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("all-rejected ImportBatch against a database with no config-table prefix: %v", err)
|
||||
}
|
||||
// Assert the batch really landed nothing for the reason this case is
|
||||
// about, so the prefix assertion below cannot be carried by a row.
|
||||
if result.Created != 0 || len(result.StaleRejectedIDs) != 1 {
|
||||
t.Fatalf("result = %+v, want Created 0 with the one row stale-rejected", result)
|
||||
}
|
||||
if !result.PrefixSynced {
|
||||
t.Errorf("PrefixSynced = false, want true — the seed wrote the prefix, so the batch must report it")
|
||||
}
|
||||
if got := queryString(t, "SELECT value FROM config WHERE `key` = 'issue_prefix'"); got != "impseed" {
|
||||
t.Errorf("issue_prefix after an all-rejected batch = %q, want %q (the seed was rolled back)", got, "impseed")
|
||||
}
|
||||
if after := countHistory(t); after != before+1 {
|
||||
t.Errorf("history entries moved %d -> %d, want exactly one commit carrying the seeded prefix", before, after)
|
||||
}
|
||||
if syncNamedAfter := countHistoryMatching(t, syncMessageLike); syncNamedAfter != syncNamedBefore+1 {
|
||||
t.Errorf("history entries naming the issue_prefix sync moved %d -> %d, want exactly one more — the seed's own commit must carry that message", syncNamedBefore, syncNamedAfter)
|
||||
}
|
||||
|
||||
// Restore for any later subtest.
|
||||
if err := kit.SetConfig(ctx, "issue_prefix", "imp"); err != nil {
|
||||
t.Fatalf("restore issue_prefix: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("EmptyActorIsRefused", func(t *testing.T) {
|
||||
if _, err := imp.ImportBatch(ctx, publicops.ImportBatchRequest{Source: "noactor.jsonl"}); err == nil {
|
||||
t.Fatal("ImportBatch with empty actor should be refused")
|
||||
|
|
|
|||
|
|
@ -67,8 +67,9 @@ type ImportBatchResult struct {
|
|||
StaleRejectedIDs []string
|
||||
// SkippedDependencies lists the edges dropped by the batch, deduplicated.
|
||||
SkippedDependencies []SkippedDependency
|
||||
// PrefixSynced reports that the stored issue_prefix was rewritten to
|
||||
// SyncIssuePrefix.
|
||||
// PrefixSynced reports that the batch wrote the stored issue_prefix to
|
||||
// SyncIssuePrefix — either seeding it, when the config table carried no
|
||||
// prefix at all, or rewriting a stored value that disagreed.
|
||||
PrefixSynced bool
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue