beads/cmd/bd/import_missing_config_prefix_test.go
cyb3rd0g01 48b9e904dc
fix(import): seed issue_prefix from config.yaml before the batch guard needs it (#6624)
* fix(import): seed issue_prefix from config.yaml before the batch guard needs it

NewBatchContext (via ReadConfigPrefix) unconditionally requires an
issue_prefix row in the database's own config table before creating
any issue. bd init writes that row via SetConfig, but a database an
external tool provisioned directly (e.g. an orchestrator that created
the Dolt database via SQL and pointed config.yaml at it without ever
calling bd init) never gets one. Every bd import then failed with
"issue_prefix config is missing", even though config.yaml carried the
prefix and bd where/bd context resolved it correctly.

bd import already had logic to sync config.yaml's prefix into that
table, but only after a successful import — useless when the import
itself can't start without the row. This moves an equivalent, guarded
sync (only fires when the table is empty, never overwrites an existing
prefix) before the import attempt, reading config.yaml directly via
GetStringFromDir rather than the global config singleton, since that
singleton isn't guaranteed to be initialized this early in the import
path.

Added a regression test that reproduces the exact failure with stock
bd (bd init --server, then delete the config-table row and write
config.yaml by hand — no orchestrator involved) and confirms it now
imports successfully.

* fix(import): address review — proxied-mode parity, --global exclusion, GetString

Three issues from @bee-ghosttrack's review of ffb32fb:

1. Proxied-server mode had the identical bug: importer.go's ImportBatch ran
   CreateIssuesInTxWithResult (-> NewBatchContext -> ReadConfigPrefix) before
   its own SyncIssuePrefix reconciliation, in the same transaction. Seeds the
   prefix before create there too, when the table is empty.

2. --global was not excluded from either sync (import.go's existing
   post-import sync and this fix's new pre-import seed, plus the proxied
   path's SyncIssuePrefix source). config.yaml is per-project; the shared
   global store's own prefix must win in --global mode (selectCreateIDPrefix
   already established this for `bd create`), or a project's local prefix
   silently overwrites the shared database with two dolt commits.

3. GetStringFromDir's rationale didn't hold: config.Initialize() has already
   run by main.go's init() and prepareSelectedCommandContext, and
   GetStringFromDir skips the config.local.yaml overlay GetString honors —
   disagreeing with the sync 40 lines below it. Switched to GetString,
   matching import.go:415 and import_proxied_server.go's existing usage.

Smaller fixes: TrimSpace + validatePrefix on the seeded value (a failed
validation now warns instead of writing an invalid prefix); SetConfig/Commit
failures now warn to stderr instead of being silently swallowed.

Test changes:
- Added TestImporterUOW/SeedsPrefixBeforeCreateNeedsIt, exercising the
  proxied-mode fix directly at the unit-of-work layer (no CLI/container
  overhead) by clearing config-table issue_prefix and confirming
  ImportBatch still creates issues. Revert-mutant confirmed: fails with
  the identical "issue_prefix config is missing" without the fix.
- Rewrote the E2E test with this repo's server-mode conventions:
  skipIfNoDolt, uniqueTestDBName + dropTestDatabase, explicit --server-port
  — no longer leaks a database on the shared test server.
- Added TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix (separate
  file, cgo-only build tag, using the shared-server harness from
  global_identity_integration_test.go): confirms a --global import does not
  adopt a project's config.yaml prefix. Revert-mutant confirmed: without the
  --global guards, the shared global store's prefix is overwritten by the
  local project's ("global" -> "localproj").

go build ./... clean; go vet ./cmd/bd/... (both with and without -tags
"cgo integration") clean; gofmt -l clean on all touched files.

* fix(import): commit the seeded prefix on an empty batch; one resolved prefix

Maintainer review fixup for PR #6624 (iterations 1-2). Addresses the two major
findings and the nearby minors/nits.

MAJOR — the proxied seed was discarded on a batch that landed no rows.
The seed wrote issue_prefix but left ImportBatchResult.PrefixSynced false, so
importBatchCommitMessage produced "" for a batch with Created == 0 and no
memories, RunTxResultWithin skipped uw.Commit on the empty message, and the
deferred closeAttempt ROLLED BACK the seeded row. `bd import` reported success,
the prefix never landed, and a re-run dedup'd again — the state never
converged. Reachable against a missing-prefix database whenever every input row
is dropped by the in-transaction stale guard or the CLI's title dedup. The seed
now reports itself through PrefixSynced, restoring the "the prefix sync runs
even when the batch is otherwise empty" contract (be-llaf) that
import_proxied_server.go's comment promises; the field's doc says so too.

MAJOR — that arm had no coverage in any lane that runs.
TestImporterUOW/SeedsPrefixEvenWhenEveryRowIsRejected pins it: empty stored
prefix + SyncIssuePrefix set + every row stale-rejected => the prefix is
committed, under the sync message, with history moving exactly one entry. It
fails on the pre-fixup tree on all four assertions. The existing
SeedsPrefixBeforeCreateNeedsIt cannot catch this (Created == 1 carries the
commit) and PrefixSyncAloneCommitsUnderTheSyncMessage runs with a non-empty
stored prefix, so its seed never fires. Both are tag-free and container-free,
so unlike the classic-path e2e they execute in ordinary lanes.

Minors and nits:

- Seed errors are returned, not warned. A SetConfig failure here is a
  precondition failure for the very next statement: swallowing it surfaced as
  the "issue_prefix config is missing" error this seed exists to prevent, and
  hid serialization failures from RunTxResult's retry classifier. Drops the
  os import.
- Classic path resolves ONE prefix for both its writers (importSyncPrefix;
  iteration 2 below extends the same resolver to the proxied path). The
  seed used a TrimSpace'd + validatePrefix'd value while the post-import sync
  used the raw one, so (a) a prefix the seed refused still reached the database
  through the sync, bypassing the guard this PR adds, and (b) a padded
  `issue-prefix: " bd"` made the two disagree and the sync overwrote the
  trimmed value. TestImportSyncPrefix pins the resolver with no store and no
  container; each arm mutation-proved independently. Side effect: the flagged
  worst_changed function drops from cyclo 25/cognitive 43 to 22/34.
- Warnings name the whole effect on both paths: an invalid issue-prefix
  disables the sync as well as the seed, which base did not do.
- The --global isolation test anchors what its "before" value IS, so the
  after == before comparison cannot go vacuous. The anchor is
  doltserver.GlobalIssuePrefix, NOT the --prefix passed to init: --prefix names
  the project's prefix and `bd init --global` always writes the constant
  (init.go), which an unanchored equality had been hiding.
- Comments record two boundaries the diff left unstated: the classic seed
  commits before an import that may fail (deferring it is not an option — the
  post-import CommitWithConfig is gated on dbPrefix != syncPrefix, which a seed
  makes false, so the row would stay uncommitted because store.Commit excludes
  config (GH#2455), and a dirty internal config key blocks the next pull); and
  a --global import against a global database missing its own
  config.issue_prefix stays unseedable by design.

Iteration 2 — three non-gating findings (both majors above stayed fixed; the
fresh synthesis and quality scorecard both say approve):

- The classic seed now runs BEFORE the memory writes. CommitWithConfig is
  DOLT_COMMIT -Am, so it stages everything dirty at that instant: with the seed
  below the memory loop, a `bd import` carrying memories against a
  missing-prefix database whose issue phase then failed left those kv.memory.*
  rows PERMANENTLY COMMITTED, under a message asserting the commit is the
  prefix seed — where at base the failure left the whole working set for the
  operator to discard. Seeding first restores "the only thing this commit can
  carry is the prefix". Pure statement reorder: the seed reads only issues,
  syncPrefix and the config table, and the ordering is now recorded as
  load-bearing at the block so a later edit does not undo it.
- The proxied path CALLS importSyncPrefix() instead of restating its four rules
  inline. The two copies were identical rule for rule — same globalFlag guard,
  same TrimSpace, same validatePrefix, byte-identical warning — but only the
  classic copy was pinned, so the underscore-key fallback ReadConfigPrefix
  honors, or hyphen normalization, could have reintroduced ACROSS import modes
  exactly the divergence the bullet above closed within one mode. One
  implementation now, so TestImportSyncPrefix's six rows govern both routes.
  Drops the os, strings and config imports from import_proxied_server.go.
- The uow subtests no longer lean on suite order.
  SeedsPrefixEvenWhenEveryRowIsRejected snapshots the sync-message history
  count before the batch and asserts +1: the old `n != 0` form could not fail
  in a full-suite run, because PrefixSyncAloneCommitsUnderTheSyncMessage has
  already committed under that message by then. Mutation-proved — routing an
  all-rejected batch to a "0 issues from ..." message reds only the new
  assertion ("moved 1 -> 1"), and the 1 in that output is precisely the count
  that would have satisfied the old form. SeedsPrefixBeforeCreateNeedsIt also
  restores issue_prefix the way its two neighbours do, instead of leaving the
  fixture to be restored by the code under test.

The PR description's stale "GetStringFromDir / config.GetString returned empty
in this exact spot" claim and its "single-file production change" scope claim
were corrected on GitHub, and the CI-lane reality for each added test is now
recorded there.

Validation (iteration 1): gofmt clean; go build ./...; go vet on cmd/bd,
internal/storage/uow and issueops; full internal/storage/uow package green
(570s, under a private temp root); focused cmd/bd import tests,
TestImportSyncPrefix, TestEmbeddedImport/prefix_sync{,_no_config_key} and the
--global isolation e2e all green.

Validation (iteration 2): gofmt and `git diff --check` clean; go build
./cmd/... ./internal/storage/uow/...; go vet on cmd/bd and
internal/storage/uow; full TestImporterUOW green in subtest order (9/9) plus
the mutation A/B above; TestImportSyncPrefix green (6/6). Two reds in the wider
`go test -run Import ./cmd/bd` are environmental on this box and reproduce
identically with the fix set stashed: TestCLI_Import_GlobalModeIgnoresProjectConfigYAMLPrefix
dies inside `bd init` because the cloned remote is one schema version behind
this build (v66 -> v67), and TestImportBlockedDuringMigrationFreeze cannot
start its server (port 3307 busy).

---------

Co-authored-by: ilasic <ilasic@reversinglabs.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
2026-09-26 15:01:04 -07:00

96 lines
3.9 KiB
Go

//go:build cgo && integration
package main
import (
"fmt"
"os"
"strings"
"testing"
)
// TestCLI_Import_ServerDatabaseMissingConfigPrefix_E2E pins that `bd import`
// succeeds against a server-mode database whose `config` table has no
// issue_prefix row, as long as config.yaml carries one.
//
// This is the state an externally-provisioned database is in: config.yaml
// carries issue-prefix (written directly by the provisioner, not via `bd
// init`), but the database's own config table has no issue_prefix row (only
// `bd init`'s SetConfig call writes one). No orchestrator is involved in
// reproducing this — the test simulates it directly by deleting the
// config-table row and writing config.yaml by hand, exactly like
// dolt_metadata_e2e_test.go simulates a pre-Phase-1 database.
//
// Before the fix, NewBatchContext (via ReadConfigPrefix) required this row
// unconditionally, so `bd import` failed with "issue_prefix config is
// missing" even though config.yaml had the prefix and `bd where`/`bd
// context` resolved it correctly.
func TestCLI_Import_ServerDatabaseMissingConfigPrefix_E2E(t *testing.T) {
skipIfNoDolt(t)
tmpDir := t.TempDir()
env := os.Environ()
database := uniqueTestDBName(t)
t.Cleanup(func() {
dropTestDatabase(database, testDoltServerPort)
})
initOut, initErr := runBDExecAllowErrorWithEnv(t, tmpDir, env,
"init", "--backend", "dolt", "--server", "--external",
"--server-host", "127.0.0.1",
"--server-port", fmt.Sprintf("%d", testDoltServerPort),
"--database", database,
"--prefix", "extdb", "--quiet")
if initErr != nil {
t.Fatalf("bd init --server failed: %v\n%s", initErr, initOut)
}
// Delete the config-table row bd init just wrote, and write issue-prefix
// into config.yaml directly (bd init does not do this on its own — an
// external provisioner writing config.yaml by hand, as an orchestrator
// would, is what puts a project in this state: config.yaml has the
// prefix, the database's own config table does not).
sqlOut, sqlErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "sql",
"DELETE FROM config WHERE `key` = 'issue_prefix'")
if sqlErr != nil {
t.Fatalf("bd sql DELETE failed: %v\n%s", sqlErr, sqlOut)
}
configPath := tmpDir + "/.beads/config.yaml"
configBytes, err := os.ReadFile(configPath)
if err != nil {
t.Fatalf("failed to read config.yaml: %v", err)
}
configBytes = append(configBytes, []byte("\nissue-prefix: extdb\n")...)
if err := os.WriteFile(configPath, configBytes, 0644); err != nil {
t.Fatalf("failed to write issue-prefix into config.yaml: %v", err)
}
// Confirm the precondition: reads that need the config-table prefix now
// fail, matching the reported bug's symptom.
precheckOut, precheckErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "create", "--title", "should fail before fix")
if precheckErr == nil {
t.Fatalf("expected bd create to fail with the config-table row deleted, but it succeeded: %s", precheckOut)
}
if !strings.Contains(precheckOut, "issue_prefix config is missing") {
t.Fatalf("expected 'issue_prefix config is missing', got: %s", precheckOut)
}
issue := `{"id":"extdb-1","title":"Externally provisioned issue","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-01T00:00:00Z"}`
if err := os.WriteFile(tmpDir+"/external.jsonl", []byte(issue+"\n"), 0644); err != nil {
t.Fatalf("failed to write JSONL fixture: %v", err)
}
importOut, importErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "import", "-i", "external.jsonl")
if importErr != nil {
t.Fatalf("bd import failed against a server-mode database missing its config-table prefix: %v\nOutput: %s", importErr, importOut)
}
listOut, listErr := runBDExecAllowErrorWithEnv(t, tmpDir, env, "list", "--id", "extdb-1", "--json")
if listErr != nil {
t.Fatalf("bd list failed: %v\n%s", listErr, listOut)
}
if !strings.Contains(listOut, "extdb-1") {
t.Fatalf("expected extdb-1 to be imported, but list output was: %s", listOut)
}
}