* ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1)
Give the Bazel-only bazel-embedded job its own duration-balanced manifest
blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and
//internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14
shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard
fork/push jobs, mirroring F2's bd_proxied_test precedent.
- Measured every TestEmbedded* / embeddeddolt test's wall time in
isolation; split 21 cmd-tier and 1 storage-tier single top-level test
over 180s into disjoint sub-tests (3 irreducible exceptions left:
TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s,
TestLargeBatchApplyStatementCounts712_Embedded 261s).
- New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py +
_embedded_shard_manifest_lib.py generate/check duration-balanced
manifest blocks (LPT packing, incremental --write/--repack, exact-once
--check), chosen at N=40/M=14 where the heaviest shard reaches the
irreducible-test floor.
- Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage
entry) left over from earlier test splits, which would have hard-failed
every shard of PR Risk's real test-embedded-cmd jobs on their next run.
- New scripts/embedded_shard_count_test.go policy tests tie the
Bazel-only block's shard count to bazel.yml's check_shard_coverage.py
argument and catch generator staleness; generalized
TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch
from bd_proxied_test-only to a map covering both embedded targets;
removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid
BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion.
- Verified: go test ./scripts/..., bazel test //scripts:scripts_test,
actionlint, make ci-pr-policy, make bazel-sync-check all pass; full
local bazel test of both targets at 40/14 shards passes 2/2; mutation
testing confirms each new/changed policy test fails exactly when it
should.
See ~/beads-bazel-plan/f1/impl-report.md for full detail.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1)
TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s
under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL
statement round-trips through the race-instrumented in-process Dolt engine,
not an artifact and not splittable (one indivisible ApplyBatchInTx
transaction). It was the storage tier's single-test floor for CI shard
packing.
Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set
for its 1000-item shape: skip this assertion under raceEnabled (the cost is
race-instrumentation overhead on the Dolt engine's own internal
goroutine/lock machinery, not this test's own logic) and extend nightly.yml's
"Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full
pinned 14014-statement baseline still runs, non-race, every night. Verified
directly: under -race the assertion now skips in 0.00s; without -race it
still runs and asserts Total=14015 (within the existing +/-2 tolerance) in
36.76s.
Updates scripts/ci_workflow_test.go's
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended
regex.
Storage tier's new single-test ceiling under the per-PR -race lane drops
from ~260.7s to 152.38s (TestBatchApplyContract).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1)
Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent
(recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus
the two other >180s cmd-tier entries from the same capture
(TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s).
Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry
contributes negligibly), NOT race-instrumented-subprocess overhead alone.
These numbers were captured via `bazel --nohome_rc test --config=embedded`
with the then-current 20-shard legacy manifest (~10-11 tests/shard) and
`-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed
clock when its function is invoked, before the Parallel() call blocks it
waiting for a free one-of-4 slot -- so a test queued behind several other
heavy shard-mates has that queueing wait baked into its own recorded
`--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling
delay, not a cost intrinsic to the test.
Confirmed two ways in an isolated clone (not this worktree, per the
no-plain-go-test-from-a-beads-checkout rule):
- Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s.
- All four together with the real -race -test.parallel=4 flags and exactly
4 tests requested (so nobody queues -- parallel slots == test count):
141.34s / 113.80s / 69.79s / 74.63s.
Both agree closely and both are far below the capture-time numbers.
F1's LPT packer already isolates the heaviest tests into shards with few or
no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own
40-shard-block shard), so under the new manifest these tests see little to
no queueing delay in real CI. None of the four needed splitting, parallel
subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the
180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 /
77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly
(with a doc-comment explaining the methodology and both confirmations) and
re-packed the 40-shard Bazel-only block (--repack --write); --check passes
(exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard
block is untouched and independently re-verified exact-once by simulating
embedded-test-shard.sh across all 20 shards (241/241 tests, no
duplicates/misses/errors).
New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed,
likely itself somewhat inflated by the same mechanism but already under the
180s line as recorded, so no action needed).
Also updates scripts/ci/embedded_storage_test_durations.json: the prior
commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded
means it now costs ~0s in the per-PR -race lane bazel-embedded actually
packs for (not its real ~260-300s non-race cost, which nightly.yml still
exercises) -- weight set to 1.0 with a doc comment tying it to the skip in
large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block
accordingly; --check passes (exact-once coverage of all 387 discovered
tests, excluding TestMain/TestConformance per that file's existing
convention). The frozen legacy 5-shard block is untouched and independently
re-verified exact-once via embedded-storage-test-shard.sh simulation
(388/388 tests, no duplicates/misses/errors).
Verification: go test ./scripts/... (includes the nightly.yml policy test),
make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local
`bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test
//internal/storage/embeddeddolt:embeddeddolt_embedded_test`
(--local_test_jobs=8, both targets together, on a shared/noisy sandbox host)
passes both targets; per-shard wall time from each shard's test.xml: cmd/bd
max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min
0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and
well under the single-test-floor-based ~475s projection this slice started
from. Projected tier lane time (max of the two targets, real dedicated CI
runners should do at least this well): ~251s, under the 5-minute target.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review)
Coordinator review follow-ups on the F1 embedded-Dolt resharding slice:
- TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was
asserting its tree_* subtests against an edgeless graph, passing
vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA
builds so those subtests exercise real multi-node traversal (review S2).
- TestEmbeddedEpicCloseEligibleReason split into
TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance,
matching the project's established single-concern split pattern.
- TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's
issuesPerWorker reduced 5->2: under real concurrent-shard-load
measurement these were genuine (not queueing-artifact) serialized-write
costs; 2 preserves every invariant the test checks (duplicate-ID
detection across all workers, one non-decreasing-list-count check per
worker) at a fraction of the serialized work.
- large712ShapeName: the 712-shape's race skip and its zero-match guard now
share one named constant instead of two independent literal shape-name
strings, so a future rename of the shape can't silently skip the pinned
statement-count assertion under race (review N2).
- Comment-only fixes (review N5) in files referencing test names since
renamed/split by this slice's earlier work: close_proxied_integration_test.go,
graph_apply_test.go, storage_class_proxied_integration_test.go,
tests/regression/discovery_test.go, plus doc-comment path cleanups (N1:
drop unverified ~/beads-bazel-plan paths/bead ids in favor of the
committed duration JSON files and engdocs/TESTING.md).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2)
Coordinator review required re-deriving every per-test duration weight from
a single, consistent, CI-realistic measurement (all shards running
concurrently, -test.parallel=4, race-enabled) instead of the prior pass's
mix of crowded-shard captures and solo/4-together spot-corrections, and
required measuring per-shard wall time via BEP testAttemptDuration rather
than a shard's outer test.xml <testsuites time> (which can include a
different, already-finished shard-mate's setup/teardown).
- scripts/ci/embedded_cmd_test_durations.json (242 tests) and
scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in
full from one consistent sweep. Every entry is now a fresh measurement;
none is a carried-over capture-time value or an unverified estimate. Both
files' _comment blocks retract the earlier "queueing delay baked into a
test's own reported elapsed time" theory (testing.go's (*T).Parallel does
not add wait time to a blocked test's own clock) and document the real
methodology.
- Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering,
239.70s), already under the ~240s ceiling; no further top-level splits
were needed this round.
- New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule
("Having more than 50 shards is indicative of poor test organization";
a native analysis-time limit, not configurable). The previous 88-shard
plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before
this limit was discovered) is reverted to 50, Bazel's real ceiling.
Repacking the fresh weights at 50 shards gives a heaviest shard of
409.6s -- above the single-test floor of 239.7s, because 50 bins is not
enough headroom over the suite's ~20145 measured test-seconds to flatten
every bin to that floor. This makes the <=5 min lane target unreachable
for the cmd tier via resharding alone within Bazel's native per-rule
limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full
writeup and the two remaining levers (suite-wide speedups or a
multi-Bazel-rule split), neither attempted in this slice.
- internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by
the 50-shard cap): LPT-packed heaviest shard 225.3s, within target.
- cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel,
.github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go,
scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected
50/15 shard counts (and, for the generator, N3: --write without an
explicit total_shards now errors instead of silently regenerating the
frozen legacy 20-shard block).
- .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed
Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for
N5 fixes to stale entries left by this slice's earlier test splits
(TestShardScriptsListOnlyRealTests catches this class of bug; both
legacy blocks reconfirmed exact-once).
- .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the
embedded-dolt batch-apply nightly step now runs with `if: ${{
!cancelled() }}` and its own explicit `-timeout 20m`, independent of the
job-level timeout, with both halves now asserted by
TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite.
Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/...
green, both generators' --check pass at 50/15 (and legacy 20/5 remain
exact-once via the frozen-block invariant).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip
The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named
TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg
failed with "matched no test that ran".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
383 lines
16 KiB
Go
383 lines
16 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"database/sql"
|
|
"errors"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
|
"github.com/steveyegge/beads/internal/types"
|
|
)
|
|
|
|
// End-to-end contract for the storage-class plumbing (bd-8rifr,
|
|
// Protocol v0.1 §C): create-time resolution (flag > per-type config >
|
|
// unset), the omitted-when-versioned marker rule (C2.4) down to the DB
|
|
// cell, and the ephemeral spelling routing to the wisp plane (C1.4).
|
|
// TestEmbeddedCreateStorageClassBasic was split from TestEmbeddedCreateStorageClass
|
|
// (originally ~356s, measured under --config=embedded) into 3 top-level tests
|
|
// over disjoint subtest groups, for CI shard balance (see
|
|
// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original
|
|
// subtest is preserved exactly once.
|
|
func TestEmbeddedCreateStorageClassBasic(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
|
|
// storageClassCell reads the raw issues.storage_class cell so the test
|
|
// pins the at-rest form (NULL vs literal), not just the JSON view.
|
|
storageClassCell := func(t *testing.T, beadsDir, prefix, id string) sql.NullString {
|
|
t.Helper()
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var got sql.NullString
|
|
if err := db.QueryRowContext(t.Context(), "SELECT storage_class FROM issues WHERE id = ?", id).Scan(&got); err != nil {
|
|
t.Fatalf("query storage_class: %v", err)
|
|
}
|
|
return got
|
|
}
|
|
|
|
t.Run("explicit_unversioned", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "su")
|
|
issue := bdCreate(t, bd, dir, "Unversioned bead", "--storage-class", "unversioned")
|
|
if issue.StorageClass != types.StorageClassUnversioned {
|
|
t.Errorf("JSON storage_class: got %q, want unversioned", issue.StorageClass)
|
|
}
|
|
cell := storageClassCell(t, beadsDir, "su", issue.ID)
|
|
if !cell.Valid || cell.String != "unversioned" {
|
|
t.Errorf("DB cell: got %+v, want 'unversioned'", cell)
|
|
}
|
|
})
|
|
|
|
t.Run("default_and_explicit_versioned_stay_null", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sv")
|
|
plain := bdCreate(t, bd, dir, "Plain bead")
|
|
if plain.StorageClass != "" {
|
|
t.Errorf("plain create: storage_class should be unset, got %q", plain.StorageClass)
|
|
}
|
|
if cell := storageClassCell(t, beadsDir, "sv", plain.ID); cell.Valid {
|
|
t.Errorf("plain create: DB cell should be NULL, got %q", cell.String)
|
|
}
|
|
// C2.4: the explicit versioned spelling normalizes to the same NULL.
|
|
explicit := bdCreate(t, bd, dir, "Versioned bead", "--storage-class", "versioned")
|
|
if cell := storageClassCell(t, beadsDir, "sv", explicit.ID); cell.Valid {
|
|
t.Errorf("explicit versioned: DB cell should be NULL, got %q", cell.String)
|
|
}
|
|
})
|
|
|
|
t.Run("per_type_config_default_and_override", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sc")
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "config", "set", "storage-class.task", "unversioned"); err != nil {
|
|
t.Fatalf("config set: %v\n%s", err, out)
|
|
}
|
|
byDefault := bdCreate(t, bd, dir, "Defaulted bead", "-t", "task")
|
|
if byDefault.StorageClass != types.StorageClassUnversioned {
|
|
t.Errorf("config default: got %q, want unversioned", byDefault.StorageClass)
|
|
}
|
|
// Explicit per-record declaration wins over the per-type default (C1.3).
|
|
forced := bdCreate(t, bd, dir, "Forced versioned", "-t", "task", "--storage-class", "versioned")
|
|
if cell := storageClassCell(t, beadsDir, "sc", forced.ID); cell.Valid {
|
|
t.Errorf("explicit versioned over config default: DB cell should be NULL, got %q", cell.String)
|
|
}
|
|
// Other types are untouched by the task default.
|
|
bug := bdCreate(t, bd, dir, "A bug", "-t", "bug")
|
|
if bug.StorageClass != "" {
|
|
t.Errorf("other type: got %q, want unset", bug.StorageClass)
|
|
}
|
|
})
|
|
|
|
t.Run("ephemeral_spelling_routes_to_wisp_plane", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "se")
|
|
issue := bdCreate(t, bd, dir, "Ephemeral bead", "--storage-class", "ephemeral")
|
|
if !issue.Ephemeral {
|
|
t.Errorf("--storage-class ephemeral should set the ephemeral flag")
|
|
}
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), "se", "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var count int
|
|
if err := db.QueryRowContext(t.Context(), "SELECT COUNT(*) FROM wisps WHERE id = ?", issue.ID).Scan(&count); err != nil {
|
|
t.Fatalf("query wisps: %v", err)
|
|
}
|
|
if count != 1 {
|
|
t.Errorf("ephemeral bead should live in wisps, found %d rows", count)
|
|
}
|
|
})
|
|
}
|
|
|
|
// TestEmbeddedCreateStorageClassConflicts was split from
|
|
// TestEmbeddedCreateStorageClass (originally ~356s, measured under
|
|
// --config=embedded) into 3 top-level tests over disjoint subtest groups,
|
|
// for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and
|
|
// engdocs/TESTING.md). Every original subtest is preserved exactly once.
|
|
func TestEmbeddedCreateStorageClassConflicts(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
|
|
// An explicit durable class combined with a wisp-plane flag is a direct
|
|
// command-line contradiction: the row would be ephemeral by construction, so
|
|
// honoring the request is impossible. It must be rejected up front, not
|
|
// silently collapsed into an effective-ephemeral record (Protocol v0.1 §C1.3).
|
|
t.Run("explicit_versioned_with_ephemeral_is_rejected", func(t *testing.T) {
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "sf")
|
|
out := bdCreateFail(t, bd, dir, "Conflicting bead", "--ephemeral", "--storage-class", "versioned")
|
|
if !strings.Contains(out, "conflicts with --ephemeral/--no-history") {
|
|
t.Errorf("expected wisp-plane conflict error, got:\n%s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("explicit_versioned_with_no_history_is_rejected", func(t *testing.T) {
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "sg")
|
|
out := bdCreateFail(t, bd, dir, "Conflicting bead", "--no-history", "--storage-class", "versioned")
|
|
if !strings.Contains(out, "conflicts with --ephemeral/--no-history") {
|
|
t.Errorf("expected wisp-plane conflict error, got:\n%s", out)
|
|
}
|
|
})
|
|
|
|
// A per-type config default is not an explicit contradiction the way the
|
|
// flag pair above is: flag > config (Protocol v0.1 §C1.3), so a background
|
|
// storage-class.task=unversioned must yield to an explicit wisp-plane flag
|
|
// rather than reaching the storage-layer backstop and blocking the create.
|
|
// bdCreate fails the test on a non-zero exit, so a returned issue is itself
|
|
// proof the backstop did not reject the config-derived durable class.
|
|
t.Run("config_default_unversioned_yields_to_ephemeral", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sy")
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "config", "set", "storage-class.task", "unversioned"); err != nil {
|
|
t.Fatalf("config set: %v\n%s", err, out)
|
|
}
|
|
issue := bdCreate(t, bd, dir, "Ephemeral over config", "-t", "task", "--ephemeral")
|
|
if !issue.Ephemeral {
|
|
t.Errorf("--ephemeral over config default should set the ephemeral flag")
|
|
}
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), "sy", "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var count int
|
|
if err := db.QueryRowContext(t.Context(), "SELECT COUNT(*) FROM wisps WHERE id = ?", issue.ID).Scan(&count); err != nil {
|
|
t.Fatalf("query wisps: %v", err)
|
|
}
|
|
if count != 1 {
|
|
t.Errorf("ephemeral bead should live in wisps, found %d rows", count)
|
|
}
|
|
})
|
|
|
|
t.Run("config_default_unversioned_yields_to_no_history", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sz")
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "config", "set", "storage-class.task", "unversioned"); err != nil {
|
|
t.Fatalf("config set: %v\n%s", err, out)
|
|
}
|
|
// --no-history is a wisp-plane record (a non-GC-eligible wisps-table row),
|
|
// so the config-derived durable class must yield rather than block: the
|
|
// create succeeds, the JSON class is unset, and the durable 'unversioned'
|
|
// never reaches the at-rest wisp row.
|
|
issue := bdCreate(t, bd, dir, "No-history over config", "-t", "task", "--no-history")
|
|
if issue.StorageClass != "" {
|
|
t.Errorf("no-history over config default: storage_class should be unset, got %q", issue.StorageClass)
|
|
}
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), "sz", "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var cell sql.NullString
|
|
if err := db.QueryRowContext(t.Context(), "SELECT storage_class FROM wisps WHERE id = ?", issue.ID).Scan(&cell); err != nil {
|
|
t.Fatalf("query wisps storage_class: %v", err)
|
|
}
|
|
if cell.Valid && cell.String == string(types.StorageClassUnversioned) {
|
|
t.Errorf("config-derived unversioned leaked to the no-history wisp row: %q", cell.String)
|
|
}
|
|
})
|
|
}
|
|
|
|
// TestEmbeddedCreateStorageClassBatchAndMisc was split from
|
|
// TestEmbeddedCreateStorageClass (originally ~356s, measured under
|
|
// --config=embedded) into 3 top-level tests over disjoint subtest groups,
|
|
// for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and
|
|
// engdocs/TESTING.md). Every original subtest is preserved exactly once.
|
|
func TestEmbeddedCreateStorageClassBatchAndMisc(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
|
|
// storageClassCell reads the raw issues.storage_class cell so the test
|
|
// pins the at-rest form (NULL vs literal), not just the JSON view.
|
|
storageClassCell := func(t *testing.T, beadsDir, prefix, id string) sql.NullString {
|
|
t.Helper()
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var got sql.NullString
|
|
if err := db.QueryRowContext(t.Context(), "SELECT storage_class FROM issues WHERE id = ?", id).Scan(&got); err != nil {
|
|
t.Fatalf("query storage_class: %v", err)
|
|
}
|
|
return got
|
|
}
|
|
|
|
// The batch doors mint their own ids, so the DB assertions look rows up by
|
|
// the title the markdown file gave them. An absent row is "" rather than a
|
|
// failure: which PLANE a row landed on is the thing under test.
|
|
issueIDByTitle := func(t *testing.T, beadsDir, prefix, table, title string) string {
|
|
t.Helper()
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var id string
|
|
//nolint:gosec // G202: table is a test-local literal, never caller input
|
|
err = db.QueryRowContext(t.Context(), "SELECT id FROM "+table+" WHERE title = ?", title).Scan(&id)
|
|
if errors.Is(err, sql.ErrNoRows) {
|
|
return ""
|
|
}
|
|
if err != nil {
|
|
t.Fatalf("query %s by title: %v", table, err)
|
|
}
|
|
return id
|
|
}
|
|
|
|
// `bd create --file` used to accept --storage-class and the per-type config
|
|
// default and honor neither. buildMarkdownBatchRequest is the ONE projection
|
|
// both transports use, so these subtests cover the direct half of that fix
|
|
// and TestProxiedServerStorageClass covers the other.
|
|
t.Run("markdown_batch", func(t *testing.T) {
|
|
batch := `## Batch one
|
|
|
|
### Description
|
|
First.
|
|
|
|
## Batch two
|
|
|
|
### Description
|
|
Second.
|
|
`
|
|
t.Run("explicit_unversioned_on_every_row", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sm")
|
|
plan := writeMarkdownPlan(t, dir, "batch.md", batch)
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--file", plan, "--storage-class", "unversioned"); err != nil {
|
|
t.Fatalf("create --file: %v\n%s", err, out)
|
|
}
|
|
for _, title := range []string{"Batch one", "Batch two"} {
|
|
id := issueIDByTitle(t, beadsDir, "sm", "issues", title)
|
|
if cell := storageClassCell(t, beadsDir, "sm", id); !cell.Valid || cell.String != "unversioned" {
|
|
t.Errorf("%s (%s): DB cell = %+v, want 'unversioned'", title, id, cell)
|
|
}
|
|
}
|
|
})
|
|
|
|
// The per-type default resolves per TEMPLATE, because that is where the
|
|
// type lives — a file-wide resolution would key on nothing.
|
|
t.Run("per_type_config_default", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sn")
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "config", "set", "storage-class.task", "unversioned"); err != nil {
|
|
t.Fatalf("config set: %v\n%s", err, out)
|
|
}
|
|
// Templates default to type task; the bug template must be untouched.
|
|
plan := writeMarkdownPlan(t, dir, "batch.md", batch+`
|
|
## Batch bug
|
|
|
|
### Type
|
|
bug
|
|
|
|
### Description
|
|
Third.
|
|
`)
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--file", plan); err != nil {
|
|
t.Fatalf("create --file: %v\n%s", err, out)
|
|
}
|
|
for _, title := range []string{"Batch one", "Batch two"} {
|
|
id := issueIDByTitle(t, beadsDir, "sn", "issues", title)
|
|
if cell := storageClassCell(t, beadsDir, "sn", id); !cell.Valid || cell.String != "unversioned" {
|
|
t.Errorf("%s (%s): DB cell = %+v, want the storage-class.task default", title, id, cell)
|
|
}
|
|
}
|
|
bugID := issueIDByTitle(t, beadsDir, "sn", "issues", "Batch bug")
|
|
if cell := storageClassCell(t, beadsDir, "sn", bugID); cell.Valid {
|
|
t.Errorf("Batch bug (%s): DB cell = %q, want NULL (the task default must not spill)", bugID, cell.String)
|
|
}
|
|
})
|
|
|
|
// C1.4 through the batch door: the spelled-out class moves every row to
|
|
// the wisp plane, exactly as --ephemeral does.
|
|
t.Run("ephemeral_spelling_routes_to_wisp_plane", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sp")
|
|
plan := writeMarkdownPlan(t, dir, "batch.md", batch)
|
|
if out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--file", plan, "--storage-class", "ephemeral"); err != nil {
|
|
t.Fatalf("create --file: %v\n%s", err, out)
|
|
}
|
|
for _, title := range []string{"Batch one", "Batch two"} {
|
|
id := issueIDByTitle(t, beadsDir, "sp", "wisps", title)
|
|
if id == "" {
|
|
t.Errorf("%s should live in wisps", title)
|
|
}
|
|
}
|
|
})
|
|
|
|
t.Run("durable_class_with_ephemeral_is_rejected", func(t *testing.T) {
|
|
dir, beadsDir, _ := bdInit(t, bd, "--prefix", "sq")
|
|
plan := writeMarkdownPlan(t, dir, "batch.md", batch)
|
|
out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--file", plan, "--ephemeral", "--storage-class", "versioned")
|
|
if err == nil {
|
|
t.Fatalf("durable class with --ephemeral should be refused, got:\n%s", out)
|
|
}
|
|
if !strings.Contains(string(out), "conflicts with --ephemeral/--no-history") {
|
|
t.Errorf("expected the wisp-plane conflict error, got:\n%s", out)
|
|
}
|
|
// The whole file is refused before the transaction opens.
|
|
if id := issueIDByTitle(t, beadsDir, "sq", "issues", "Batch one"); id != "" {
|
|
t.Errorf("a refused --file left %s behind", id)
|
|
}
|
|
})
|
|
})
|
|
|
|
// A plan-wide --storage-class has no meaning next to the per-node
|
|
// storage_class field that already works, so --graph refuses it the way it
|
|
// refuses --mol-type rather than accepting and ignoring it.
|
|
t.Run("graph_rejects_plan_wide_storage_class", func(t *testing.T) {
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "sh")
|
|
planPath := filepath.Join(dir, "plan.json")
|
|
if err := os.WriteFile(planPath, []byte(`{"nodes":[{"key":"a","title":"Graph node","description":"n"}]}`), 0o600); err != nil {
|
|
t.Fatalf("write graph plan: %v", err)
|
|
}
|
|
out, err := bdRunWithFlockRetry(t, bd, dir, "create", "--graph", planPath, "--storage-class", "unversioned")
|
|
if err == nil {
|
|
t.Fatalf("--graph with a plan-wide --storage-class should be refused, got:\n%s", out)
|
|
}
|
|
if !strings.Contains(string(out), "set storage_class per node in the plan instead") {
|
|
t.Errorf("error should point at the per-node field, got:\n%s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("config_set_validates_value", func(t *testing.T) {
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "sx")
|
|
out, err := bdRunWithFlockRetry(t, bd, dir, "config", "set", "storage-class.task", "permanent")
|
|
if err == nil {
|
|
t.Fatalf("config set with bad value should fail, got:\n%s", out)
|
|
}
|
|
if !strings.Contains(string(out), "versioned, unversioned, or ephemeral") {
|
|
t.Errorf("error should enumerate valid values, got:\n%s", out)
|
|
}
|
|
})
|
|
}
|