From 1137aeb70d427d013ada46b9dbdef669d2e4655d Mon Sep 17 00:00:00 2001 From: Julian Knutsen Date: Sat, 3 Oct 2026 21:30:03 -0700 Subject: [PATCH] =?UTF-8?q?ci(bazel):=20embedded-Dolt=20tier=2020/5=20?= =?UTF-8?q?=E2=86=92=2050/15=20duration-balanced=20Bazel=20shards=20(F1)?= =?UTF-8?q?=20(#7156)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ci(bazel): embedded-Dolt tier 20/5 -> 40/14 duration-balanced shards (F1) Give the Bazel-only bazel-embedded job its own duration-balanced manifest blocks for //cmd/bd:bd_embedded_test (20 -> 40 shards) and //internal/storage/embeddeddolt:embeddeddolt_embedded_test (5 -> 14 shards), decoupled from PR Risk's/main.yml's legacy frozen 20/5-shard fork/push jobs, mirroring F2's bd_proxied_test precedent. - Measured every TestEmbedded* / embeddeddolt test's wall time in isolation; split 21 cmd-tier and 1 storage-tier single top-level test over 180s into disjoint sub-tests (3 irreducible exceptions left: TestEmbeddedUpdateConcurrent 475s, TestEmbeddedCloseConcurrent 308s, TestLargeBatchApplyStatementCounts712_Embedded 261s). - New scripts/ci/gen_embedded_{cmd,storage}_shard_manifest.py + _embedded_shard_manifest_lib.py generate/check duration-balanced manifest blocks (LPT packing, incremental --write/--repack, exact-once --check), chosen at N=40/M=14 where the heaviest shard reaches the irreducible-test floor. - Fixed 18 stale legacy 20-shard cmd manifest entries (and 1 storage entry) left over from earlier test splits, which would have hard-failed every shard of PR Risk's real test-embedded-cmd jobs on their next run. - New scripts/embedded_shard_count_test.go policy tests tie the Bazel-only block's shard count to bazel.yml's check_shard_coverage.py argument and catch generator staleness; generalized TestBazelRetiredLanesCheckListedTestsRan's live-shard-count dispatch from bd_proxied_test-only to a map covering both embedded targets; removed TestBazelEmbeddedJobMirrorsEmbeddedTier's now-invalid BUILD.bazel-shard-count-equals-PR-Risk-matrix assertion. - Verified: go test ./scripts/..., bazel test //scripts:scripts_test, actionlint, make ci-pr-policy, make bazel-sync-check all pass; full local bazel test of both targets at 40/14 shards passes 2/2; mutation testing confirms each new/changed policy test fails exactly when it should. See ~/beads-bazel-plan/f1/impl-report.md for full detail. Co-Authored-By: Claude Sonnet 5 * fix(embedded-dolt): skip 712-item statement-count assertion under -race (F1) TestLargeBatchApplyStatementCounts712_Embedded genuinely costs ~260-300s under -race (confirmed by solo re-measurement: 298.60s) -- 14014 real SQL statement round-trips through the race-instrumented in-process Dolt engine, not an artifact and not splittable (one indivisible ApplyBatchInTx transaction). It was the storage tier's single-test floor for CI shard packing. Apply the exact precedent TestLargeBatchApplyWallClock_Embedded already set for its 1000-item shape: skip this assertion under raceEnabled (the cost is race-instrumentation overhead on the Dolt engine's own internal goroutine/lock machinery, not this test's own logic) and extend nightly.yml's "Embedded Dolt batch-apply suite (non-race)" step's -run regex so the full pinned 14014-statement baseline still runs, non-race, every night. Verified directly: under -race the assertion now skips in 0.00s; without -race it still runs and asserts Total=14015 (within the existing +/-2 tolerance) in 36.76s. Updates scripts/ci_workflow_test.go's TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite to match the extended regex. Storage tier's new single-test ceiling under the per-PR -race lane drops from ~260.7s to 152.38s (TestBatchApplyContract). Co-Authored-By: Claude Sonnet 5 * ci(bazel): correct cmd-tier long-pole durations, re-pack embedded manifolds (F1) Investigated coordinator item 2 (single-test long poles): TestEmbeddedUpdateConcurrent (recorded 475.39s) and TestEmbeddedCloseConcurrent (recorded 308.06s), plus the two other >180s cmd-tier entries from the same capture (TestEmbeddedInitA 196.94s, TestEmbeddedEpicCloseEligibleReason 192.47s). Root cause: NOT host contention, NOT backoff/retry sleeps (bdRunWithFlockRetry contributes negligibly), NOT race-instrumented-subprocess overhead alone. These numbers were captured via `bazel --nohome_rc test --config=embedded` with the then-current 20-shard legacy manifest (~10-11 tests/shard) and `-test.parallel=4`. Go's t.Parallel() starts each parallel test's elapsed clock when its function is invoked, before the Parallel() call blocks it waiting for a free one-of-4 slot -- so a test queued behind several other heavy shard-mates has that queueing wait baked into its own recorded `--- PASS: TestX (Ys)` time. This is shard-composition-dependent scheduling delay, not a cost intrinsic to the test. Confirmed two ways in an isolated clone (not this worktree, per the no-plain-go-test-from-a-beads-checkout rule): - Solo (no rivals at all): 151.83s / 106.48s / 63.58s / 66.08s. - All four together with the real -race -test.parallel=4 flags and exactly 4 tests requested (so nobody queues -- parallel slots == test count): 141.34s / 113.80s / 69.79s / 74.63s. Both agree closely and both are far below the capture-time numbers. F1's LPT packer already isolates the heaviest tests into shards with few or no other heavy rivals (TestEmbeddedUpdateConcurrent lands alone in its own 40-shard-block shard), so under the new manifest these tests see little to no queueing delay in real CI. None of the four needed splitting, parallel subtests, or raceEnabled-scaled N: once correctly measured, none exceeds the 180s single-test guideline (new corrected weights: 155.0 / 115.0 / 72.0 / 77.0s). Corrected scripts/ci/embedded_cmd_test_durations.json accordingly (with a doc-comment explaining the methodology and both confirmations) and re-packed the 40-shard Bazel-only block (--repack --write); --check passes (exact-once coverage of all 241 discovered tests). The frozen legacy 20-shard block is untouched and independently re-verified exact-once by simulating embedded-test-shard.sh across all 20 shards (241/241 tests, no duplicates/misses/errors). New cmd-tier single-test ceiling: 179.47s (TestEmbeddedCloseAlreadyClosed, likely itself somewhat inflated by the same mechanism but already under the 180s line as recorded, so no action needed). Also updates scripts/ci/embedded_storage_test_durations.json: the prior commit's raceEnabled-skip for TestLargeBatchApplyStatementCounts712_Embedded means it now costs ~0s in the per-PR -race lane bazel-embedded actually packs for (not its real ~260-300s non-race cost, which nightly.yml still exercises) -- weight set to 1.0 with a doc comment tying it to the skip in large_batch_apply_measure_test.go. Re-packed the 14-shard Bazel-only block accordingly; --check passes (exact-once coverage of all 387 discovered tests, excluding TestMain/TestConformance per that file's existing convention). The frozen legacy 5-shard block is untouched and independently re-verified exact-once via embedded-storage-test-shard.sh simulation (388/388 tests, no duplicates/misses/errors). Verification: go test ./scripts/... (includes the nightly.yml policy test), make ci-pr-policy, make bazel-sync-check, actionlint all pass. Real local `bazel --nohome_rc test --config=embedded //cmd/bd:bd_embedded_test //internal/storage/embeddeddolt:embeddeddolt_embedded_test` (--local_test_jobs=8, both targets together, on a shared/noisy sandbox host) passes both targets; per-shard wall time from each shard's test.xml: cmd/bd max 218.9s (shard 23/40, min 20.6s), storage max 251.1s (shard 5/14, min 0.98s) -- down from the prior oversubscribed sweep's 783.41s / 351.94s and well under the single-test-floor-based ~475s projection this slice started from. Projected tier lane time (max of the two targets, real dedicated CI runners should do at least this well): ~251s, under the 5-minute target. Co-Authored-By: Claude Sonnet 5 * test(embedded): un-vacuous TestEmbeddedDepB edges, split epic-close test, trim concurrency stress N (F1 review) Coordinator review follow-ups on the F1 embedded-Dolt resharding slice: - TestEmbeddedDepB (one of TestEmbeddedDep's two CI-speed split halves) was asserting its tree_* subtests against an edgeless graph, passing vacuously; restores the same parent-child/blocks edges TestEmbeddedDepA builds so those subtests exercise real multi-node traversal (review S2). - TestEmbeddedEpicCloseEligibleReason split into TestEmbeddedEpicCloseEligibleReasonFlag/...Output for CI shard balance, matching the project's established single-concern split pattern. - TestEmbeddedUpdateConcurrent/TestEmbeddedCloseConcurrent's issuesPerWorker reduced 5->2: under real concurrent-shard-load measurement these were genuine (not queueing-artifact) serialized-write costs; 2 preserves every invariant the test checks (duplicate-ID detection across all workers, one non-decreasing-list-count check per worker) at a fraction of the serialized work. - large712ShapeName: the 712-shape's race skip and its zero-match guard now share one named constant instead of two independent literal shape-name strings, so a future rename of the shape can't silently skip the pinned statement-count assertion under race (review N2). - Comment-only fixes (review N5) in files referencing test names since renamed/split by this slice's earlier work: close_proxied_integration_test.go, graph_apply_test.go, storage_class_proxied_integration_test.go, tests/regression/discovery_test.go, plus doc-comment path cleanups (N1: drop unverified ~/beads-bazel-plan paths/bead ids in favor of the committed duration JSON files and engdocs/TESTING.md). Co-Authored-By: Claude Sonnet 5 * ci(bazel): re-derive embedded-Dolt durations under concurrent load; cap cmd shards at Bazel's 50-shard limit (F1 round 2) Coordinator review required re-deriving every per-test duration weight from a single, consistent, CI-realistic measurement (all shards running concurrently, -test.parallel=4, race-enabled) instead of the prior pass's mix of crowded-shard captures and solo/4-together spot-corrections, and required measuring per-shard wall time via BEP testAttemptDuration rather than a shard's outer test.xml (which can include a different, already-finished shard-mate's setup/teardown). - scripts/ci/embedded_cmd_test_durations.json (242 tests) and scripts/ci/embedded_storage_test_durations.json (387 tests) rewritten in full from one consistent sweep. Every entry is now a fresh measurement; none is a carried-over capture-time value or an unverified estimate. Both files' _comment blocks retract the earlier "queueing delay baked into a test's own reported elapsed time" theory (testing.go's (*T).Parallel does not add wait time to a blocked test's own clock) and document the real methodology. - Only one cmd-tier test remains over 200s (TestEmbeddedListFiltering, 239.70s), already under the ~240s ceiling; no further top-level splits were needed this round. - New finding: Bazel hard-caps shard_count at 50 per sh_test-like rule ("Having more than 50 shards is indicative of poor test organization"; a native analysis-time limit, not configurable). The previous 88-shard plan for //cmd/bd:bd_embedded_test (chosen as the LPT-packed floor before this limit was discovered) is reverted to 50, Bazel's real ceiling. Repacking the fresh weights at 50 shards gives a heaviest shard of 409.6s -- above the single-test floor of 239.7s, because 50 bins is not enough headroom over the suite's ~20145 measured test-seconds to flatten every bin to that floor. This makes the <=5 min lane target unreachable for the cmd tier via resharding alone within Bazel's native per-rule limit; see ~/beads-bazel-plan/f1/impl-report.md section 3c for the full writeup and the two remaining levers (suite-wide speedups or a multi-Bazel-rule split), neither attempted in this slice. - internal/storage/embeddeddolt gets 15 Bazel-only shards (unaffected by the 50-shard cap): LPT-packed heaviest shard 225.3s, within target. - cmd/bd/BUILD.bazel, internal/storage/embeddeddolt/BUILD.bazel, .github/workflows/bazel.yml, engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md, scripts/embedded_shard_count_test.go, scripts/pr_risk_bazel_coverage_test.go, scripts/ci/gen_embedded_cmd_shard_manifest.py updated to the corrected 50/15 shard counts (and, for the generator, N3: --write without an explicit total_shards now errors instead of silently regenerating the frozen legacy 20-shard block). - .github/scripts/embedded-{cmd,storage}-test-shards.txt: re-packed Bazel-only blocks at 50/15; frozen legacy 20/5 blocks untouched except for N5 fixes to stale entries left by this slice's earlier test splits (TestShardScriptsListOnlyRealTests catches this class of bug; both legacy blocks reconfirmed exact-once). - .github/workflows/nightly.yml / scripts/ci_workflow_test.go (N4): the embedded-dolt batch-apply nightly step now runs with `if: ${{ !cancelled() }}` and its own explicit `-timeout 20m`, independent of the job-level timeout, with both halves now asserted by TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite. Verified: gofmt -l clean, go vet ./scripts/... clean, go test ./scripts/... green, both generators' --check pass at 50/15 (and legacy 20/5 remain exact-once via the frozen-block invariant). Co-Authored-By: Claude Sonnet 5 * test(ci): rename embedded manifest staleness tests out of the ^TestEmbedded skip The allowlisted go-test leg runs with -skip ^TestEmbedded, so tests named TestEmbedded*ShardManifestGeneratorNotStale never ran there and the leg failed with "matched no test that ran". Co-Authored-By: Claude Opus 5.5 (1M context) --------- Co-authored-by: Gas City Adopt-PR Co-authored-by: Claude Sonnet 5 --- .github/scripts/embedded-cmd-test-shards.txt | 381 +++++++++++++- .../scripts/embedded-storage-test-shards.txt | 444 ++++++++++++++++- .github/workflows/bazel.yml | 41 +- .github/workflows/nightly.yml | 22 +- cmd/bd/BUILD.bazel | 20 +- cmd/bd/autocommit_embedded_test.go | 23 +- cmd/bd/close_embedded_test.go | 49 +- cmd/bd/close_proxied_integration_test.go | 13 +- cmd/bd/create_embedded_test.go | 21 +- cmd/bd/dep_embedded_test.go | 115 +++-- cmd/bd/dolt_embedded_test.go | 26 +- cmd/bd/epic_embedded_test.go | 72 ++- cmd/bd/export_embedded_test.go | 16 +- cmd/bd/federation_embedded_test.go | 16 +- cmd/bd/graph_apply_test.go | 4 +- cmd/bd/import_embedded_test.go | 16 +- cmd/bd/init_embedded_test.go | 25 +- cmd/bd/label_embedded_test.go | 37 +- cmd/bd/list_embedded_test.go | 20 +- cmd/bd/max_rows_embedded_test.go | 95 +++- cmd/bd/prune_embedded_test.go | 21 +- cmd/bd/purge_embedded_test.go | 16 +- cmd/bd/rename_prefix_embedded_test.go | 18 +- cmd/bd/show_embedded_test.go | 21 +- cmd/bd/storage_class_embedded_test.go | 95 +++- .../storage_class_proxied_integration_test.go | 4 +- cmd/bd/update_embedded_test.go | 33 +- cmd/bd/vc_embedded_test.go | 16 +- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md | 11 +- .../dolt/large_batch_apply_measure_test.go | 10 +- internal/storage/embeddeddolt/BUILD.bazel | 18 +- .../large_batch_apply_measure_test.go | 88 +++- scripts/BUILD.bazel | 1 + scripts/ci/_embedded_shard_manifest_lib.py | 168 +++++++ scripts/ci/embedded_cmd_test_durations.json | 315 ++++++++++++ .../ci/embedded_storage_test_durations.json | 469 ++++++++++++++++++ scripts/ci/gen_embedded_cmd_shard_manifest.py | 210 ++++++++ .../ci/gen_embedded_storage_shard_manifest.py | 209 ++++++++ scripts/ci_workflow_test.go | 33 +- scripts/embedded_shard_count_test.go | 123 +++++ scripts/pr_risk_bazel_coverage_test.go | 62 ++- tests/regression/discovery_test.go | 4 +- tools/bazel/equivalence_allowlist.txt | 2 + 43 files changed, 3209 insertions(+), 194 deletions(-) create mode 100644 scripts/ci/_embedded_shard_manifest_lib.py create mode 100644 scripts/ci/embedded_cmd_test_durations.json create mode 100644 scripts/ci/embedded_storage_test_durations.json create mode 100755 scripts/ci/gen_embedded_cmd_shard_manifest.py create mode 100755 scripts/ci/gen_embedded_storage_shard_manifest.py create mode 100644 scripts/embedded_shard_count_test.go diff --git a/.github/scripts/embedded-cmd-test-shards.txt b/.github/scripts/embedded-cmd-test-shards.txt index d756a3a4c..c3bfa552f 100644 --- a/.github/scripts/embedded-cmd-test-shards.txt +++ b/.github/scripts/embedded-cmd-test-shards.txt @@ -16,7 +16,9 @@ 20 8 TestEmbeddedBranchConcurrent 20 9 TestEmbeddedChildren 20 10 TestEmbeddedChildrenConcurrent -20 11 TestEmbeddedClose +20 11 TestEmbeddedCloseBasic +20 11 TestEmbeddedCloseGuardsAndEpics +20 11 TestEmbeddedCloseAlreadyClosed 20 12 TestEmbeddedCloseConcurrent 20 13 TestEmbeddedComments 20 14 TestEmbeddedCommentsCLI @@ -32,7 +34,8 @@ 20 4 TestEmbeddedContributorCreate 20 5 TestEmbeddedCount 20 6 TestEmbeddedCountConcurrent -20 7 TestEmbeddedCreate +20 7 TestEmbeddedCreateScalarAndRelationships +20 7 TestEmbeddedCreateGraphAndBulk 20 8 TestEmbeddedCreateCommitPending 20 9 TestEmbeddedCreateConcurrent 20 10 TestEmbeddedCreateCrossRepo @@ -42,21 +45,28 @@ 20 14 TestEmbeddedDefer 20 15 TestEmbeddedDeferConcurrent 20 16 TestEmbeddedDelete -20 17 TestEmbeddedDep +20 17 TestEmbeddedDepA +20 17 TestEmbeddedDepB 20 18 TestEmbeddedDepConcurrent 20 19 TestEmbeddedDiff 20 20 TestEmbeddedDiffConcurrent -20 1 TestEmbeddedDolt +20 1 TestEmbeddedDoltBlockedAndCommit +20 1 TestEmbeddedDoltRemoteAndPush 20 2 TestEmbeddedDoltConcurrent 20 3 TestEmbeddedDuplicate 20 4 TestEmbeddedDuplicateConcurrent 20 5 TestEmbeddedDuplicates 20 6 TestEmbeddedDuplicatesConcurrent -20 7 TestEmbeddedEpic +20 7 TestEmbeddedEpicStatus +20 7 TestEmbeddedEpicCloseEligible +20 7 TestEmbeddedEpicCloseEligibleReasonFlag +20 7 TestEmbeddedEpicCloseEligibleReasonOutput 20 8 TestEmbeddedEpicConcurrent -20 9 TestEmbeddedExport +20 9 TestEmbeddedExportFormats +20 9 TestEmbeddedExportFlags 20 10 TestEmbeddedExportConcurrent -20 11 TestEmbeddedFederation +20 11 TestEmbeddedFederationPeers +20 11 TestEmbeddedFederationStatus 20 12 TestEmbeddedFederationConcurrent 20 13 TestEmbeddedFindDuplicates 20 14 TestEmbeddedFindDuplicatesConcurrent @@ -76,22 +86,33 @@ 20 8 TestEmbeddedHooksConcurrent 20 9 TestEmbeddedHuman 20 10 TestEmbeddedHumanConcurrent -20 11 TestEmbeddedImport +20 11 TestEmbeddedImportFromSources +20 11 TestEmbeddedImportUpsertAndPrefixSync 20 12 TestEmbeddedImportConcurrent 20 13 TestEmbeddedInfo 20 14 TestEmbeddedInfoConcurrent -20 15 TestEmbeddedInit +20 15 TestEmbeddedInitA +20 15 TestEmbeddedInitB +20 15 TestEmbeddedInitC 20 16 TestEmbeddedInitConcurrent 20 17 TestEmbeddedKV 20 18 TestEmbeddedKVConcurrent -20 19 TestEmbeddedLabel +20 19 TestEmbeddedLabelAddRemove +20 19 TestEmbeddedLabelEditReports +20 19 TestEmbeddedLabelPropagateRename 20 20 TestEmbeddedLabelConcurrent 20 1 TestEmbeddedLint 20 2 TestEmbeddedLintConcurrent -20 3 TestEmbeddedList +20 3 TestEmbeddedListFiltering +20 3 TestEmbeddedListSearchAndOutput 20 4 TestEmbeddedListConcurrent -20 13 TestEmbeddedMaxRowsList -20 11 TestEmbeddedMaxRowsNonListPaths +20 13 TestEmbeddedMaxRowsListBasic +20 13 TestEmbeddedMaxRowsListLimitInteraction +20 11 TestEmbeddedMaxRowsDepTreeAndDuplicates +20 12 TestEmbeddedMaxRowsDoctorAndConfig +20 14 TestEmbeddedMaxRowsGraph +20 15 TestEmbeddedMaxRowsReady +20 16 TestEmbeddedMaxRowsReadyMerge 20 5 TestEmbeddedMemory 20 6 TestEmbeddedMemoryConcurrent 20 7 TestEmbeddedMigrate @@ -105,8 +126,10 @@ 20 15 TestEmbeddedPromote 20 16 TestEmbeddedPromoteCLI 20 17 TestEmbeddedPromoteCLIConcurrent -20 18 TestEmbeddedPrune -20 19 TestEmbeddedPurge +20 18 TestEmbeddedPruneSafetyAndScope +20 18 TestEmbeddedPruneDryRunAndFilters +20 19 TestEmbeddedPurgeBasics +20 19 TestEmbeddedPurgeFiltered 20 20 TestEmbeddedPurgeConcurrent 20 1 TestEmbeddedQuery 20 2 TestEmbeddedQueryConcurrent @@ -116,7 +139,8 @@ 20 6 TestEmbeddedQuickstartConcurrent 20 7 TestEmbeddedReady 20 8 TestEmbeddedReadyConcurrent -20 9 TestEmbeddedRenamePrefix +20 9 TestEmbeddedRenamePrefixBasic +20 9 TestEmbeddedRenamePrefixEdgeCases 20 10 TestEmbeddedRenamePrefixConcurrent 20 11 TestEmbeddedReopen 20 12 TestEmbeddedReopenConcurrent @@ -128,7 +152,8 @@ 20 18 TestEmbeddedSetupConcurrent 20 19 TestEmbeddedShip 20 20 TestEmbeddedShipConcurrent -20 1 TestEmbeddedShow +20 1 TestEmbeddedShowBasicsAndJSON +20 1 TestEmbeddedShowDetailAndCurrent 20 2 TestEmbeddedShowConcurrent 20 3 TestEmbeddedStale 20 4 TestEmbeddedStaleConcurrent @@ -146,11 +171,329 @@ 20 16 TestEmbeddedTypesConcurrent 20 17 TestEmbeddedUndefer 20 18 TestEmbeddedUndeferConcurrent -20 19 TestEmbeddedUpdate +20 19 TestEmbeddedUpdateFields +20 19 TestEmbeddedUpdateLifecycle 20 20 TestEmbeddedUpdateConcurrent -20 1 TestEmbeddedVC +20 1 TestEmbeddedVCStatusAndCommit +20 1 TestEmbeddedVCCommitVariantsAndMerge 20 2 TestEmbeddedVCConcurrent 20 3 TestEmbeddedVersion 20 4 TestEmbeddedVersionConcurrent 20 5 TestEmbeddedWhere 20 6 TestEmbeddedWhereConcurrent + +# Embedded-Dolt cmd/bd test shard manifest. +# +# Format: +# +# 50-shard split for the Bazel-only embedded-Dolt cmd tier +# (bazel-embedded in .github/workflows/bazel.yml), bin-packed +# longest-processing-time-first by measured wall-time from +# scripts/ci/embedded_cmd_test_durations.json (see that file for provenance +# and its "relative weight, not absolute SLA" caveat). +# Regenerate with scripts/ci/gen_embedded_cmd_shard_manifest.py 50 +# --weights=duration --write after adding, splitting or removing +# TestEmbedded* functions in cmd/bd/*_embedded_test.go. --write is +# incremental: it keeps every already-assigned test on its current shard +# and only places newly-discovered names, onto whichever shard is +# currently lightest, so two unrelated PRs that each add one test do not +# textually conflict or silently invalidate each other's packing. Pass +# --repack to force a full from-scratch rebalance instead (a deliberate, +# separate change — expect a large diff). --check only verifies every +# discovered name is listed here exactly once (no stale or duplicate +# entries); it does NOT verify the packing is still well-balanced. +# +# This file is generated: notes added here are erased by the next +# regeneration. Add them to the generator or the durations file instead. + +50 1 TestEmbeddedCommentsCLI +50 1 TestEmbeddedDepAddWispSource +50 1 TestEmbeddedListFiltering +50 1 TestEmbeddedPrimeChangeDirDefaultTemplateMatchesTarget + +50 2 TestEmbeddedCreateRepoFromNonBeadsCwd +50 2 TestEmbeddedDeleteJSONDependencyErrorContract +50 2 TestEmbeddedDepBulkNoCycleCheckKeepsWholeGraphGate +50 2 TestEmbeddedHistoryConcurrent +50 2 TestEmbeddedInitA + +50 3 TestEmbeddedAdminCompactTier2Honesty +50 3 TestEmbeddedMaxRowsListLimitInteraction +50 3 TestEmbeddedOrphans +50 3 TestEmbeddedShowUnresolvableDeps +50 3 TestEmbeddedStale + +50 4 TestEmbeddedBootstrapConcurrent +50 4 TestEmbeddedDuplicatesConcurrent +50 4 TestEmbeddedInfo +50 4 TestEmbeddedMaxRowsDoctorAndConfig +50 4 TestEmbeddedRoutedSiblingWritesCommitTargetHead + +50 5 TestEmbeddedCreateScalarAndRelationships +50 5 TestEmbeddedDefer +50 5 TestEmbeddedDoltBlockedAndCommit +50 5 TestEmbeddedGCConcurrent +50 5 TestEmbeddedPromote + +50 6 TestEmbeddedCreateStorageClassConflicts +50 6 TestEmbeddedGC +50 6 TestEmbeddedGetDependencies +50 6 TestEmbeddedQuickConcurrent + +50 7 TestEmbeddedCreateCrossRepoDryRunWithParent +50 7 TestEmbeddedEpicStatus +50 7 TestEmbeddedGate +50 7 TestEmbeddedWhere + +50 8 TestEmbeddedChangeDirOverridesInheritedBeadsDir +50 8 TestEmbeddedMemoryConcurrent +50 8 TestEmbeddedUpdateConcurrent +50 8 TestEmbeddedVCConcurrent + +50 9 TestEmbeddedCloseAlreadyClosed +50 9 TestEmbeddedConfigConcurrent +50 9 TestEmbeddedExternalCapabilityReadyAfterShip +50 9 TestEmbeddedPrimeConcurrent + +50 10 TestEmbeddedDepB +50 10 TestEmbeddedHuman +50 10 TestEmbeddedInitExplicitBeadsDirIsAuthoritative +50 10 TestEmbeddedPromoteCLI + +50 11 TestEmbeddedBlockedConcurrent +50 11 TestEmbeddedDepA +50 11 TestEmbeddedMaxRowsDepTreeAndDuplicates +50 11 TestEmbeddedState + +50 12 TestEmbeddedDirectCommitPathsBatchAutoCommitDoesNotAdvanceHead +50 12 TestEmbeddedPrimeChangeDirPrimesTarget +50 12 TestEmbeddedPromoteCLIConcurrent +50 12 TestEmbeddedSearchConcurrent +50 12 TestEmbeddedTypes + +50 13 TestEmbeddedAutoImportJSONLNoExplicitCommit +50 13 TestEmbeddedDiff +50 13 TestEmbeddedDuplicates +50 13 TestEmbeddedShipConcurrent + +50 14 TestEmbeddedInitConcurrent +50 14 TestEmbeddedOrphansConcurrent +50 14 TestEmbeddedPrimeChangeDirRedirectNoticeFollowsTarget +50 14 TestEmbeddedRoutedMutatingSiblingDeleteClose +50 14 TestEmbeddedSwarm + +50 15 TestEmbeddedChildrenConcurrent +50 15 TestEmbeddedConfigValidateJSONNoWorkspaceWritesStdout +50 15 TestEmbeddedCreateDryRunCrossRepoDoesNotMigrateTarget +50 15 TestEmbeddedEpicCloseEligibleReasonFlag +50 15 TestEmbeddedExplicitHierarchicalParentEdges +50 15 TestEmbeddedVersionConcurrent + +50 16 TestEmbeddedBlockedLabelFilters +50 16 TestEmbeddedKV +50 16 TestEmbeddedMaxRowsReady +50 16 TestEmbeddedOpen_EmptyDatabaseRejected +50 16 TestEmbeddedRestoreConcurrent +50 16 TestEmbeddedVCCommitVariantsAndMerge +50 16 TestEmbeddedVersion + +50 17 TestEmbeddedCloseGuardsAndEpics +50 17 TestEmbeddedCompact +50 17 TestEmbeddedGateConcurrent +50 17 TestEmbeddedListComments + +50 18 TestEmbeddedContext +50 18 TestEmbeddedContributorCreate +50 18 TestEmbeddedImportClosedChildWhoseParentWasDeleted +50 18 TestEmbeddedReopen +50 18 TestEmbeddedStateConcurrent + +50 19 TestEmbeddedAdminBlocked +50 19 TestEmbeddedCreateRepoRelativeExistingWorkspaceUnaffected +50 19 TestEmbeddedDoltConcurrent +50 19 TestEmbeddedMaxRowsListBasic +50 19 TestEmbeddedPruneSafetyAndScope + +50 20 TestEmbeddedCreateStorageClassBatchAndMisc +50 20 TestEmbeddedDepListSingleIDWarnsOnDroppedExternalEdge +50 20 TestEmbeddedKVConcurrent +50 20 TestEmbeddedSetup + +50 21 TestEmbeddedBackup +50 21 TestEmbeddedCreateCrossRepoUninit +50 21 TestEmbeddedDiffConcurrent +50 21 TestEmbeddedQuick +50 21 TestEmbeddedQuickstart + +50 22 TestEmbeddedAdminCompactReadOnlyAllowed +50 22 TestEmbeddedCreateRepoProxiedTargetError +50 22 TestEmbeddedDuplicateConcurrent +50 22 TestEmbeddedFlatten +50 22 TestEmbeddedTypesAgreeWithCreate + +50 23 TestEmbeddedExportConcurrent +50 23 TestEmbeddedHistory +50 23 TestEmbeddedMaxRowsGraph +50 23 TestEmbeddedSetupConcurrent +50 23 TestEmbeddedShip + +50 24 TestEmbeddedCloseBasic +50 24 TestEmbeddedImportUpsertAndPrefixSync +50 24 TestEmbeddedQuery +50 24 TestEmbeddedUpdateRoutedStoreCommitsTargetHead + +50 25 TestEmbeddedBackupConcurrent +50 25 TestEmbeddedCreateStorageClassBasic +50 25 TestEmbeddedFindDuplicatesConcurrent +50 25 TestEmbeddedInitGitBootstrapRouting +50 25 TestEmbeddedInitHooksRouting + +50 26 TestEmbeddedCreateCommitPending +50 26 TestEmbeddedInitC +50 26 TestEmbeddedRenamePrefixBasic +50 26 TestEmbeddedRoutedMutatingSiblingAssignTagDep +50 26 TestEmbeddedSupersedeConcurrent + +50 27 TestEmbeddedComments +50 27 TestEmbeddedMaxRowsReadyMerge +50 27 TestEmbeddedPrime +50 27 TestEmbeddedPruneDryRunAndFilters +50 27 TestEmbeddedReopenConcurrent + +50 28 TestEmbeddedConfigNoWorkspaceStoreFreeCommands +50 28 TestEmbeddedEpicCloseEligibleReasonOutput +50 28 TestEmbeddedMigrate +50 28 TestEmbeddedStatus +50 28 TestEmbeddedSwarmConcurrent + +50 29 TestEmbeddedClaimPools +50 29 TestEmbeddedCreateGraphAndBulk +50 29 TestEmbeddedCreateJSONEchoesSubSecondTimestamps +50 29 TestEmbeddedHumanConcurrent +50 29 TestEmbeddedInitSelectedExcludeRouting + +50 30 TestEmbeddedAutoImportFallback_ConflictSkip_EmptyDBImportsAll +50 30 TestEmbeddedDepConcurrent +50 30 TestEmbeddedExportFormats +50 30 TestEmbeddedReadyClaimConcurrent +50 30 TestEmbeddedStaleConcurrent + +50 31 TestEmbeddedAutoImportFallback_ConflictSkip_DoesNotClobber +50 31 TestEmbeddedAutoImportJSONLConcurrentWriter +50 31 TestEmbeddedDeferAutoWake +50 31 TestEmbeddedExternalMutationPolicy +50 31 TestEmbeddedShowConcurrent + +50 32 TestEmbeddedCreateCrossRepo +50 32 TestEmbeddedDuplicate +50 32 TestEmbeddedInitArtifactRouting +50 32 TestEmbeddedShowBasicsAndJSON +50 32 TestEmbeddedUpdateLifecycle + +50 33 TestEmbeddedCreateConcurrent +50 33 TestEmbeddedCreateRepoRelativeUninitRefused +50 33 TestEmbeddedEpicCloseEligible +50 33 TestEmbeddedSearch +50 33 TestEmbeddedTodo + +50 34 TestEmbeddedCreateDryRunRepoDoesNotInitializeTarget +50 34 TestEmbeddedGateListScoped +50 34 TestEmbeddedLintConcurrent +50 34 TestEmbeddedPurgeWispsPlaneRetention +50 34 TestEmbeddedReady + +50 35 TestEmbeddedCommentsCLIConcurrent +50 35 TestEmbeddedHooksConcurrent +50 35 TestEmbeddedLabelPropagateRename +50 35 TestEmbeddedListSearchAndOutput +50 35 TestEmbeddedSupersede + +50 36 TestEmbeddedDeferConcurrent +50 36 TestEmbeddedFederationPeers +50 36 TestEmbeddedFlattenConcurrent +50 36 TestEmbeddedStatusConcurrent +50 36 TestEmbeddedTodoConcurrent + +50 37 TestEmbeddedInfoSuppressGitHooks +50 37 TestEmbeddedInitB +50 37 TestEmbeddedQueryConcurrent +50 37 TestEmbeddedUndeferConcurrent +50 37 TestEmbeddedUpdateBatchAutoCommitDoesNotAdvanceHead + +50 38 TestEmbeddedBlocked +50 38 TestEmbeddedCreateCrossRepoWithParent +50 38 TestEmbeddedEpicConcurrent +50 38 TestEmbeddedHooks +50 38 TestEmbeddedUndefer + +50 39 TestEmbeddedBranchConcurrent +50 39 TestEmbeddedCountIncludeInfra +50 39 TestEmbeddedCreateWithGitRemote +50 39 TestEmbeddedDelete +50 39 TestEmbeddedUpdateFields + +50 40 TestEmbeddedCountConcurrent +50 40 TestEmbeddedCountIncludeEphemeral +50 40 TestEmbeddedCreateFormCommitsLabelOnlyCreate +50 40 TestEmbeddedLabelEditReports +50 40 TestEmbeddedPreviewDoesNotConsumeVersionMarker + +50 41 TestEmbeddedGateCreate +50 41 TestEmbeddedLabelAddRemove +50 41 TestEmbeddedLabelConcurrent +50 41 TestEmbeddedListConcurrent +50 41 TestEmbeddedTypesConcurrent + +50 42 TestEmbeddedImportFromSources +50 42 TestEmbeddedReadyConcurrent +50 42 TestEmbeddedRenamePrefixEdgeCases +50 42 TestEmbeddedRestore +50 42 TestEmbeddedSquashWispMoleculeClearsRootEphemeral + +50 43 TestEmbeddedFindDuplicates +50 43 TestEmbeddedGraph +50 43 TestEmbeddedImportPromotedWispRoundtrip +50 43 TestEmbeddedInfoConcurrent +50 43 TestEmbeddedQuickstartConcurrent + +50 44 TestEmbeddedDoltRemoteAndPush +50 44 TestEmbeddedGraphConcurrent +50 44 TestEmbeddedImportConcurrent +50 44 TestEmbeddedInitRoleRouting +50 44 TestEmbeddedWhereConcurrent + +50 45 TestEmbeddedChildren +50 45 TestEmbeddedCompactConcurrent +50 45 TestEmbeddedMigrateConcurrent +50 45 TestEmbeddedShowDetailAndCurrent +50 45 TestEmbeddedVCStatusAndCommit + +50 46 TestEmbeddedCompactMisdatedChild +50 46 TestEmbeddedCount +50 46 TestEmbeddedCreateDryRunDoesNotMigrate +50 46 TestEmbeddedFederationStatus +50 46 TestEmbeddedStaleLabelFilters + +50 47 TestEmbeddedBootstrap +50 47 TestEmbeddedBranch +50 47 TestEmbeddedContextConcurrent +50 47 TestEmbeddedLint +50 47 TestEmbeddedPurgeConcurrent + +50 48 TestEmbeddedCloseConcurrent +50 48 TestEmbeddedConfig +50 48 TestEmbeddedDepAndLinkBatchAutoCommitDoesNotAdvanceHead +50 48 TestEmbeddedOnboardConcurrent +50 48 TestEmbeddedQuietFlagSuppressesSuccessOutput + +50 49 TestEmbeddedDepNoCycleCheck +50 49 TestEmbeddedFederationConcurrent +50 49 TestEmbeddedMemory +50 49 TestEmbeddedOnboard +50 49 TestEmbeddedPurgeFiltered + +50 50 TestEmbeddedAutoImportJSONLSkipsNonEmpty +50 50 TestEmbeddedExportFlags +50 50 TestEmbeddedPurgeBasics +50 50 TestEmbeddedRenamePrefixConcurrent +50 50 TestEmbeddedUnclaim diff --git a/.github/scripts/embedded-storage-test-shards.txt b/.github/scripts/embedded-storage-test-shards.txt index cfd0dbe78..4d972644d 100644 --- a/.github/scripts/embedded-storage-test-shards.txt +++ b/.github/scripts/embedded-storage-test-shards.txt @@ -16,14 +16,21 @@ # shards by sub-test path (core/audit), not by top-level function name, and # is a different mechanism from this manifest. # -# Three tests are placed by measured -race duration instead (PR Risk run +# Several tests are placed by measured -race duration instead (PR Risk run # 36807191489, at the 1000-item issueops.MaxApplyBatchItems cap): # TestBatchApplyContract (604s) on shard 4, which has ~3m of other tests, -# and TestLargeBatchApplyStatementCounts_Embedded (247s) and -# TestLargeBatchApplyWallClock_Embedded (261s) on shards 2 and 5. Hashed, +# and TestLargeBatchApplyWallClock_Embedded (261s) on shard 5. Hashed, # TestBatchApplyContract shared shard 2 with ~5m of other tests and ran it # past the job's old 15m timeout; placed so, the heaviest shard projects to # ~13m. +# +# TestLargeBatchApplyStatementCounts_Embedded (247s in that same run) was +# later split into TestLargeBatchApplyStatementCounts356_Embedded, +# TestLargeBatchApplyStatementCounts712_Embedded and +# TestLargeBatchApplyStatementCountsClassic40_Embedded (slice F1, CI shard +# balance; see internal/storage/embeddeddolt/large_batch_apply_measure_test.go). +# All 3 are kept on shard 2, where the single test used to live, rather than +# rebalanced across this frozen 5-shard block. 5 1 TestActiveDatabaseSizeFailsForMissingEmbeddedDatabase 5 2 TestActiveDatabaseSizeScopesToActiveEmbeddedDatabase 5 3 TestAddDependency @@ -83,7 +90,9 @@ 5 1 TestHistory_NullTextColumns 5 2 TestHookFiringStoreCreateIssuesFiresDependencyUpdatesFromEmbeddedStore 5 3 TestIsBlockedBatchEmbedded -5 2 TestLargeBatchApplyStatementCounts_Embedded +5 2 TestLargeBatchApplyStatementCounts356_Embedded +5 2 TestLargeBatchApplyStatementCounts712_Embedded +5 2 TestLargeBatchApplyStatementCountsClassic40_Embedded 5 5 TestLargeBatchApplyWallClock_Embedded 5 4 TestLeaseLifecycleEmbedded 5 5 TestListWisps @@ -100,3 +109,430 @@ 5 1 TestUpdateIssueType 5 2 TestUpdateMergeOps_DoltEngine 5 3 TestUpdateMergeOps_DoltEngine_TwoIssues + +# Embedded-Dolt storage test shard manifest. +# +# Format: +# +# 15-shard split for the Bazel-only embedded-Dolt storage tier +# (bazel-embedded in .github/workflows/bazel.yml), bin-packed +# longest-processing-time-first by measured wall-time from +# scripts/ci/embedded_storage_test_durations.json (see that file for provenance +# and its "relative weight, not absolute SLA" caveat). +# Regenerate with scripts/ci/gen_embedded_storage_shard_manifest.py 15 +# --weights=duration --write after adding or removing Test* functions in +# internal/storage/embeddeddolt/*_test.go. --write is incremental: it +# keeps every already-assigned test on its current shard and only places +# newly-discovered names, onto whichever shard is currently lightest, so +# two unrelated PRs that each add one test do not textually conflict or +# silently invalidate each other's packing. Pass --repack to force a full +# from-scratch rebalance instead (a deliberate, separate change — expect a +# large diff). --check only verifies every discovered name is listed here +# exactly once (no stale or duplicate entries); it does NOT verify the +# packing is still well-balanced. +# +# This file is generated: notes added here are erased by the next +# regeneration. Add them to the generator or the durations file instead. + +15 1 TestBatchApplyContract +15 1 TestEmbeddedClaimerClaimsAnUnassignedOpenIssueAndAnswersTheBareRow +15 1 TestEmbeddedClaimerReclaimsAcrossSpellingWithoutWriting +15 1 TestEmbeddedDependencyEditorWritesTheTargetIntoItsTypedColumn +15 1 TestEmbeddedIssueOperationsCreateParentChildRecomputesWaitsForClosure +15 1 TestEmbeddedIssueOperationsCreateUnderAParentMintsTheNextChildID +15 1 TestEmbeddedIssueOperationsUpdateClaimCASAndTransferMatrix +15 1 TestEmbeddedIssueOperationsUpdateClaimConflictCarriesTheLosingState +15 1 TestEmbeddedMergeAndSettleMetadataConflict +15 1 TestEmbeddedMergeAndSettleRefusesUnknownTable +15 1 TestEmbeddedMigration0047DelegatesToSplitTargetRepairAndConvergesThroughIgnoredChain +15 1 TestEmbeddedReaderListDefaultExclusionsAndTheirOverrides +15 1 TestEmbeddedReaderListSkipCountsDropsTheCardinalitiesAndNothingElse +15 1 TestEmbeddedReaderListWispTypeNarrowsTheAdmittedPlaneRatherThanAdmittingIt +15 1 TestEmbeddedReaderReadyBriefDropsTheFreeFormTextAndNothingElse +15 1 TestEmbeddedRelationsAnswersAWispTargetInTheOutDirection +15 1 TestEmbeddedRelationsLeavesAnExternalTargetOutOfTheAnswer +15 1 TestEmbeddedRoleFixtureKitHooksAreUsable +15 1 TestLifecycleCreateContract +15 1 TestMergeWithStrategy_OursKeepsLocal + +15 2 TestActiveDatabaseSizeScopesToActiveEmbeddedDatabase +15 2 TestCleanGitRemoteCacheGarbage +15 2 TestDepRekeyMarkerStillInIgnoredSeries +15 2 TestDescendantWalkReachesEveryTargetKindAndStopsOnCycles +15 2 TestDualWriteFixtureKitIsWired +15 2 TestEmbeddedDepRekeyAbortLeavesMarkerUnrecorded +15 2 TestEmbeddedDependencyEditorAppliesParentChildBeforeBlockingEdges +15 2 TestEmbeddedDependencyEditorRefusesACycleThroughAParentChildHop +15 2 TestEmbeddedDependencyEditorRefusesAGhostSource +15 2 TestEmbeddedDependencyEditorRoutesWispSourcedRemovalToTheWispPlane +15 2 TestEmbeddedDependencyEditorSkipPerEdgeCycleCheckDropsOnlyTheProbe +15 2 TestEmbeddedIssueOperationsCreateAggregatesRelations +15 2 TestEmbeddedIssueOperationsUpdateStatusCrossingSettlesDependers +15 2 TestEmbeddedOpenReadOnly_FreshDatabaseNoWrites +15 2 TestEmbeddedReaderListCountsAreBlocksOnlyWhereGetCountsEveryEdge +15 2 TestEmbeddedRelationsDirectionSelectsTheInverseGraph +15 2 TestEmbeddedRelationsLeavesTheCallersRequestAlone +15 2 TestEmbeddedRelationsOrdersNeighborsFromBothPlanesTogether +15 2 TestEmbeddedRemoteMigrateGate_BlocksReopen +15 2 TestGitRefPinsMatchDolt +15 2 TestHeartbeatRejectsWispEmbedded +15 2 TestHookFiringStoreCreateIssuesFiresDependencyUpdatesFromEmbeddedStore +15 2 TestIgnoredCursorHealRespectsOverrideUnderBroaderPattern +15 2 TestLeaseLifecycleEmbedded +15 2 TestLenientGateWarningBody_ShapedDecisions +15 2 TestReadEventsAutoPruneStateResolvesBothHalvesInOneQuery +15 2 TestSchemaAfterInit +15 2 TestStageAndCommitAfterSQLCommitResponseLossIsIndeterminateAndNotReplayed +15 2 TestWithPeerAuth_WarnsWhenStoredPeerSuppressesAmbientPassword + +15 3 TestEmbeddedBatchCloserDoesNotMutateTheCallerRequest +15 3 TestEmbeddedBatchCloserOutcomeSnapshotIsTheDocumentedShape +15 3 TestEmbeddedClaimerRefusesAWispIDAsNotFound +15 3 TestEmbeddedConfigCommitStagesConfig +15 3 TestEmbeddedDependencyEditorAcceptsBlockingAcrossIssueTypes +15 3 TestEmbeddedDependencyEditorRefusalWritesNothing +15 3 TestEmbeddedDependencyEditorRemoveIsIdempotent +15 3 TestEmbeddedDependencyEditorVersionsOnlyARealAddOrRemove +15 3 TestEmbeddedIgnoredMigration0023NoopsOnHealthyWorkspace +15 3 TestEmbeddedMergeAndSettleFKCascadeRepair +15 3 TestEmbeddedReaderListParentReachesEveryDescendantAndOnlyItsOwn +15 3 TestEmbeddedReaderListPriorityKeysetWalkOverAnOversizedEqualKeyRunLosesNothingAndRepeatsNothing +15 3 TestEmbeddedReaderReadyLimitBoundary +15 3 TestEmbeddedReadyClaimerFencesTheClaimByEveryLabelSetAndTheParentItWasGiven +15 3 TestEmbeddedRelationsRefusesATypeFilterOverTheColumnLength +15 3 TestEmbeddedRenameKeepsDependencyIDsDeterministic +15 3 TestEventsAutoPruneKeepsAnUnboundedJournalUnbounded +15 3 TestEventsJournalReadRefusesAnInteriorHole +15 3 TestForeignSkipDetailIsBounded +15 3 TestListWisps +15 3 TestMergeAndSettleWithStrategy_ResolvesDeclinedConflict +15 3 TestPushCommitsPendingChangesFirst +15 3 TestUpdateMergeOps_DoltEngine_TwoIssues +15 3 TestWithPeerAuth_RestoresAbsentEnv + +15 4 TestEmbeddedBatchCloserSettlesTheDependersOfWhatItClosed +15 4 TestEmbeddedBlockedRecheckSettlesWhatTheWriteRecorded +15 4 TestEmbeddedClaimIssueWrapperClaimsTheWispPlane +15 4 TestEmbeddedClaimerRefusesABuiltInIneligibleStatusWithTheStateThatRefusedIt +15 4 TestEmbeddedClaimerTakesAPoolAssignedIssueButRefusesAForeignHolder +15 4 TestEmbeddedCommitToleratesNothingToCommit +15 4 TestEmbeddedDependencyEditorRecordsOneHistoryEntryForAMixedPlaneRequest +15 4 TestEmbeddedDependencyEditorRefusesBlockingEdgeAcrossItsOwnHierarchy +15 4 TestEmbeddedIgnoredMigration0023RepairsForkLineageJournalShape +15 4 TestEmbeddedReaderListEmptyPageIsWellFormed +15 4 TestEmbeddedRelationsFiltersByAnOpenTypeVocabulary +15 4 TestEmbeddedReopenIssueCategorySemantics +15 4 TestEmbeddedRoleFixtureKitCountHistoryMovesWithACommit +15 4 TestEventsAutoPruneThrottleSkipsTheSecondPass +15 4 TestEventsJournalRestartAndRetentionBoundary +15 4 TestIgnoredCursorHealNeverResurrectsRolledBackVersions +15 4 TestMain +15 4 TestMemoriesContract +15 4 TestPromoteClearsNoHistory +15 4 TestRemoteVerbPlainRemoteKeepsAmbientEnv +15 4 TestReopenIssue +15 4 TestWithPeerAuth +15 4 TestWithPeerAuth_EnvFallbackDoesNotObservePeerCredentials + +15 5 TestActiveDatabaseSizeFailsForMissingEmbeddedDatabase +15 5 TestBlockingAnnotatorContract +15 5 TestBootstrapperContract +15 5 TestBuildDSN_WithDatabase +15 5 TestCrossRecordInvariantContract +15 5 TestEmbeddedBatchCloserClaimNextSeesAnUnblockingFromItsOwnBatch +15 5 TestEmbeddedClaimerStampsStartedAtOnceAcrossTheTwoWritesItChoosesBetween +15 5 TestEmbeddedDepRekeyConvergesRenameChain +15 5 TestEmbeddedDependencyEditorAcceptsADiamond +15 5 TestEmbeddedDependencyEditorRemoveUnmarksItsSourceAndDescendants +15 5 TestEmbeddedDependencyEditorRetypeRefusalLeavesTheOriginalEdge +15 5 TestEmbeddedDependencyEditorValidationRefusalsWriteNothing +15 5 TestEmbeddedDoltStoreIssueLifecycleRejectsNilStore +15 5 TestEmbeddedIssueOperationsCreateReverseNonBlockingStagesConcreteTables +15 5 TestEmbeddedIssueOperationsPersistenceRollbackWispAndDetachedResults +15 5 TestEmbeddedMergeAndSettleReportsOperatorConflicts +15 5 TestEmbeddedMigration0058AddsOnlyMissingConstraintInMixedPopulation +15 5 TestEmbeddedReaderGetBriefDepsProjectsTheDependencyRows +15 5 TestEmbeddedReaderListKeysetPositionResumesTheCreatedDescIDAscOrder +15 5 TestEmbeddedReaderListMaxRowsIsHonored +15 5 TestEmbeddedReadyClaimerDoesNotMutateTheCallerRequest +15 5 TestEmbeddedRelationsAnswersInThePinnedOrder +15 5 TestEventsAutoPruneBoundsTheJournalToTheFloors +15 5 TestExpectedRevisionContract +15 5 TestGetDependentRecordsForIssuesEmbedded +15 5 TestLargeBatchApplyWallClock_Embedded +15 5 TestLenientGateWarningBody_BluntRefusalKeepsSharedGuidance +15 5 TestOpenReturnsCachedStore +15 5 TestPartitionWispIDs +15 5 TestRetentionContract +15 5 TestWithPeerAuth_KeyMismatchFailsClosed + +15 6 TestCreateIssuesRejectStaleUpserts +15 6 TestDecryptPassword_KeyMismatchNamesTheLocalKey +15 6 TestEmbeddedBatchCloserPerItemRefusalIsAResultAndSurvivorsCommit +15 6 TestEmbeddedBatchCloserWispItemClosesAndEarnsTheClaim +15 6 TestEmbeddedBatchCreateFirstVersionCarriesCreationTimeEdges +15 6 TestEmbeddedDependencyEditorClosedChildAddSatisfiesAnAnyChildrenGate +15 6 TestEmbeddedDependencyEditorMaintainsBlockedStateAcrossPlanes +15 6 TestEmbeddedDependencyEditorSameTypeReAddWithChangedMetadataMintsOneVersion +15 6 TestEmbeddedIssueOperationsUpdateClaimPatchAppliesAfterClaim +15 6 TestEmbeddedReaderGetMissIsNotFoundAndBackendFailureDoesNotDecay +15 6 TestEmbeddedReadyClaimerClaimsTheFrontRowAndReturnsThePostClaimState +15 6 TestEmbeddedReadyClaimerRecordsOneHistoryEntryForAWin +15 6 TestEmbeddedSyncCommitsConfigBeforeMerge +15 6 TestGetIssue +15 6 TestHeartbeatMatchesActorAcrossSpellingEmbedded +15 6 TestHistory_NullTextColumns +15 6 TestIsBlockedBatchEmbedded +15 6 TestLargeBatchApplyStatementCounts356_Embedded +15 6 TestRemoteGitDataRefOnEngine +15 6 TestRenameLabel +15 6 TestRenameLabelMintsOneVersionPerTouchedBead +15 6 TestRunInTransactionPostCommitStageFailureIsIndeterminate +15 6 TestStatsReporterContract +15 6 TestWithPeerAuth_ConcurrentPeersDoNotMixCredentials +15 6 TestWithPeerAuth_RestoresEnvWhenCallbackFails + +15 7 TestCommitAllOutOfBandChangeAndHonestNoOp +15 7 TestConcurrencyMultiProcess +15 7 TestConcurrentOpenReturnsCached +15 7 TestDualWriteContractFlagOff +15 7 TestEmbeddedBatchCloserClaimNextIsNilWhenTheFrontIsEmpty +15 7 TestEmbeddedBatchCloserForceBypassesOnlyClosePolicy +15 7 TestEmbeddedDepRekeyRepairsHalfRekeyedDuplicateEdge +15 7 TestEmbeddedDependencyEditorGateScopeFollowsTheEdgeType +15 7 TestEmbeddedDependencyEditorRoutesWispSourcedEdgeToTheWispPlane +15 7 TestEmbeddedDependencyEditorSameTypeReAddWithIdenticalMetadataIsANoOp +15 7 TestEmbeddedIssueOperationsUpdateStatusCrossingSettlesAConditionalBlocksDepender +15 7 TestEmbeddedMigration0057ConvertsTextEventsColumnsToLongtext +15 7 TestEmbeddedReaderListReadyFlagAnswersTheBlockerAwareSet +15 7 TestEmbeddedReaderReadyDefaultTypeExclusionsYieldToAnExplicitType +15 7 TestEmbeddedReaderReadySortPoliciesOrderTheSameRows +15 7 TestEmbeddedReadyClaimerAnswersTheQuestionReaderReadyLists +15 7 TestEmbeddedReadyClaimerLeasesADurableWinButNotAnEphemeralOne +15 7 TestEventsJournalAccessor +15 7 TestHistoryPresenceContract +15 7 TestJournalContract +15 7 TestLifecycleCloseReopenContract +15 7 TestReclaimAnyReplicaOverride +15 7 TestTreeWalkerContract +15 7 TestWithPeerAuth_DivergedRemoteURLFailsClosed + +15 8 TestAddLabel +15 8 TestBuildDSN_NoDatabase +15 8 TestCleanGitRemoteCacheGarbage_NoCacheDir +15 8 TestDoltCommitResponseLossIsIndeterminate +15 8 TestEmbeddedBatchCloserClaimNextHydratesWhenSomethingClosed +15 8 TestEmbeddedBatchCloserClaimNextIsNilWhenNothingClosed +15 8 TestEmbeddedBatchCloserOutcomesMirrorItemsIndexForIndex +15 8 TestEmbeddedClaimerRefusalMessagesCarryTheirFragments +15 8 TestEmbeddedDependencyEditorAddMarksItsSourceInTheSameVerb +15 8 TestEmbeddedDependencyEditorRecordsOneHistoryEntryPerLandedRequest +15 8 TestEmbeddedGuardedUpdateMintsOnceAfterItsPatches +15 8 TestEmbeddedIssueOperationsUpdateFoldsMetadataIntoOneEvent +15 8 TestEmbeddedIssueOperationsUpdateStampsStartedAtOnceOnTheFirstInProgress +15 8 TestEmbeddedLenientOpenWarning_DataBehind +15 8 TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict +15 8 TestEmbeddedMigrateRepairedDependenciesIDColumnCommitsAtomicallyWithVersion53_4690 +15 8 TestEmbeddedMigration0058HealsAlreadyAffectedDatabase +15 8 TestEmbeddedOpenReadOnly_SkipsGateAndMigrations +15 8 TestEmbeddedReaderListMaxRowsBoundaryIsLimitPlusOffset +15 8 TestEmbeddedReaderListReadyFlagCarriesTheAssigneeAndPriorityFilters +15 8 TestEmbeddedReaderReadyDeferredAndEphemeralGates +15 8 TestEmbeddedReaderReadyPageIsThePrefixOfTheUnboundedAnswerCountsIncluded +15 8 TestEmbeddedRelationsResolvesTheAnchorIDExactly +15 8 TestEmbeddedSQLCommitResponseLossIsIndeterminate +15 8 TestGetReadyWorkIncludeEphemeralAppliesWispFilters +15 8 TestHasRepository +15 8 TestLenientGateWarning_ComposedOutput +15 8 TestMergeAndSettleWithStrategy_EmptyStrategyMatchesMergeAndSettle +15 8 TestReclaimExpiredLeaseSurvivesRestartEmbedded +15 8 TestRemoteVerbPlainRemoteWithoutAuthPresentsNone +15 8 TestTransactionCleanupAfterSQLCommitIsIndeterminate + +15 9 TestCommenterContract +15 9 TestEmbeddedBatchCloserBackendFailureReturnsNoOutcomes +15 9 TestEmbeddedClaimerReclaimsItsOwnInProgressIssueWithoutWriting +15 9 TestEmbeddedClaimerRefusesIncompleteRequestsAndAnAbsentIDWithoutTouchingState +15 9 TestEmbeddedDependencyEditorMixedBatchWritesBothPlanes +15 9 TestEmbeddedDependencyEditorRefusesBlockingEdgeAcrossAWispHierarchy +15 9 TestEmbeddedDependencyEditorRelatesToAddLeavesItsSourceUnblocked +15 9 TestEmbeddedDependencyEditorRemovesOnlyTheNamedEdge +15 9 TestEmbeddedIgnoredMigration0019CreatesEventsForFreshClones +15 9 TestEmbeddedIssueOperationsCloseAndReopenAWisp +15 9 TestEmbeddedIssueOperationsUpdateClosedFieldsMatchClose +15 9 TestEmbeddedIssueOperationsUpdateInvalidIssueTypePreservesValidationSentinel +15 9 TestEmbeddedOpenForRemoteSync_OtherRefusalsStayFatal +15 9 TestEmbeddedReaderListLimitBoundaryUnderASortTheDatabaseCanExpress +15 9 TestEmbeddedReaderListReadyFlagRefusesAFilterItCannotCarry +15 9 TestEmbeddedReaderOffsetSkipsTheRowsBeforeThePage +15 9 TestEmbeddedReaderReadyPageWiderThanTheHydrationBatchIsStillThatPrefix +15 9 TestEventsSinceEmbedded +15 9 TestIgnoredCursorHealConvergesFromEveryCrashState +15 9 TestLegacyTrackedIgnoredCursorUnwedgesPull +15 9 TestMetadataCASContract +15 9 TestPromoteStillWorksWithCollisionGuard +15 9 TestReadyWorkWithMissingDeferredParentTables +15 9 TestUpdateMergeOps_DoltEngine + +15 10 TestBuildDSN_SpacesInPath +15 10 TestCleanGitRemoteCacheGarbage_Throttled +15 10 TestCreateIssues +15 10 TestCreateRejectsCrossTableIDCollision +15 10 TestDeleterContract +15 10 TestEmbeddedBatchCloserAllIdempotentBatchLandsNothing +15 10 TestEmbeddedDepRekeyMergesFreshDuplicateEdges +15 10 TestEmbeddedDependencyEditorAddedEchoesTheRequestOrder +15 10 TestEmbeddedDependencyEditorRefusesCrossPlaneCycle +15 10 TestEmbeddedDoltStoreIssueLifecycleBuildsOperations +15 10 TestEmbeddedIgnoredMigration0023NoopsWithoutItsTables +15 10 TestEmbeddedIssueOperationsCreateClosedDerivesTheClosedStamp +15 10 TestEmbeddedIssueOperationsCreateEmptyIDPreservesParentAndReverseDependencies +15 10 TestEmbeddedIssueOperationsUpdateParentStagesBlockedStateAndRollsBackFailure +15 10 TestEmbeddedIssueOperationsUpdateRawMetadataTakesTheFunnelsValueShapes +15 10 TestEmbeddedMigrateStepCommitLeavesUntouchedUserDirt_4566 +15 10 TestEmbeddedReaderDoesNotMutateTheCallerRequest +15 10 TestEmbeddedReadyClaimerHydratesOnlyItsBlocksEdgesIntoTheCardinalities +15 10 TestEmbeddedReadyClaimerSkipsIneligibleFrontRows +15 10 TestEmbeddedRecomputeAllBlockedDirtyGraphIsTyped +15 10 TestEmbeddedTempRebuildIntermediatesNeverReachHistory +15 10 TestEmbeddedTransactionWispLifecyclePersistsDurableDependent +15 10 TestEpochContract +15 10 TestGetReadyWorkMoleculeFilter +15 10 TestHelperProcess +15 10 TestIgnoredCursorHealResumeRespectsOperatorOverride +15 10 TestReclaimRefusesForeignReplicaLease +15 10 TestRemoteEntryPointsUseVersionControlOps +15 10 TestTransactionCleanupBeforeSQLCommitRemainsDefinite +15 10 TestVersionReconcilerContract +15 10 TestWithPeerAuth_MissingRemoteFailsClosed + +15 11 TestCycleDetectorContract +15 11 TestEmbeddedBatchCloserClaimFilterValueFailureIsARequestValidationFailure +15 11 TestEmbeddedBatchCloserIdempotentRecloseIsAPerItemSuccess +15 11 TestEmbeddedCheckedWrappersRouteTheirPreconditions +15 11 TestEmbeddedDependencyEditorAcceptsAForeignRepoTarget +15 11 TestEmbeddedDependencyEditorMixedBatchRefusalRollsBackBothPlanes +15 11 TestEmbeddedDependencyEditorRefusesASamePlaneEdgeClosingACrossPlaneCycle +15 11 TestEmbeddedDependencyEditorSameTypeReAddIsIdempotent +15 11 TestEmbeddedGetMergeBlockersOnAQuietStore +15 11 TestEmbeddedIssueOperationsCreateRoutesInfraTypesToWisps +15 11 TestEmbeddedIssueOperationsUpdateNotesConflictPreservesValidationSentinel +15 11 TestEmbeddedLenientOpenWarning_OtherRefusalsKeepBluntGuidance +15 11 TestEmbeddedPre0058RepairMarchesDriftedWispDependenciesToFinalShape +15 11 TestEmbeddedReaderGetDetailShapeMatchesTheSeededIssue +15 11 TestEmbeddedReaderListNaturalNumericIDSortTrimsAfterTheFetch +15 11 TestEmbeddedReopenCustomDoneFallsBackWhenNormalizedTableMissing +15 11 TestEmbeddedTransactionStandaloneWispMutationDoesNotPublishUnrelatedDurableChanges +15 11 TestGetDependentRecordsEmbedded +15 11 TestGetLabels +15 11 TestGetNextChildID +15 11 TestGraphCounterContract +15 11 TestLargeBatchApplyStatementCountsClassic40_Embedded +15 11 TestPristineEmbeddedDoltFixtureRelocatesPrefixes +15 11 TestReadyWorkPageTotal +15 11 TestUpdateIssueType + +15 12 TestBatchCreatorContract +15 12 TestDualWriteContract +15 12 TestEmbeddedClaimerTakesAnOpenIssueTheActorAlreadyHolds +15 12 TestEmbeddedDependencyEditorAttributesItsEventsToTheActor +15 12 TestEmbeddedDependencyEditorSnapshotsTheRequest +15 12 TestEmbeddedIgnoredSeriesConvergesWithFreshInitShape +15 12 TestEmbeddedIssueOperationsCreateWithDependenciesSettlesInTheCreatingTransaction +15 12 TestEmbeddedIssueOperationsRequestValuesAreNotMutated +15 12 TestEmbeddedMergeAndSettleMemoryConfigConflict +15 12 TestEmbeddedReaderGetOptionalRowListsAreOffByDefault +15 12 TestEmbeddedReaderGetResolvesTheExactIDAcrossBothPlanes +15 12 TestEmbeddedReaderReadyEphemeralPageKeepsBothPlanesCountsAtItsBoundary +15 12 TestEmbeddedReaderReadyParentScopesToItsTransitiveDescendants +15 12 TestEmbeddedReadyClaimerLeavesEphemeralRowsOutOfTheDefaultReadySet +15 12 TestEmbeddedRecomputeAllBlockedWiring +15 12 TestEmbeddedRelationsResolvesAWispAnchor +15 12 TestEmbeddedTransactionSameStatusDoesNotPublishUnrelatedDurableChanges +15 12 TestLargeBatchApplyStatementCounts712_Embedded +15 12 TestLifecycleUpdateContract +15 12 TestPreviewStoreRefusesLocalMetadataWrite +15 12 TestQuerierContract +15 12 TestReadyWorkPageTotalFilterMatrix +15 12 TestRemoteVerbsPresentStoredPeerCredentials +15 12 TestSearchWithMissingWispPlaneTables +15 12 TestWorkspaceConfigContract + +15 13 TestCountsIncludeWispDependencies +15 13 TestDecryptPassword_ShortCiphertextClassifiesAsKeyMismatch +15 13 TestEmbeddedBatchCloserDurableHistoryNeverNamesAWisp +15 13 TestEmbeddedBatchCloserRequestValidationReturnsZeroResultAndChangesNothing +15 13 TestEmbeddedClaimerGrantsALiveLeaseOnTheRowItWonAndLeavesARefusedOneAlone +15 13 TestEmbeddedDependencyEditorRefusesAMissingLocalTarget +15 13 TestEmbeddedIssueOperationsUpdateAllScalarAndPointerFieldsReportChanged +15 13 TestEmbeddedIssueOperationsUpdateClaimHonorsConfiguredActiveStatuses +15 13 TestEmbeddedIssueOperationsUpdateMetadataReplaceClearsAndValidates +15 13 TestEmbeddedIssueOperationsUpdateRefusesATypeOutsideTheWorkspaceVocabulary +15 13 TestEmbeddedIssueOperationsUpdateWritesEveryScalarPatchField +15 13 TestEmbeddedMigrateConvergesUnderConcurrentInterruptedRetries_4566 +15 13 TestEmbeddedMigration0058CleansOrphanedAndInvalidWispDependenciesRowsBeforeAddingConstraints +15 13 TestEmbeddedMigration0062FlipsEventsToIgnoredPlane +15 13 TestEmbeddedReaderListIncludeEphemeralMergesThePlanesIntoOneOrder +15 13 TestEmbeddedReaderListKeysetPositionNarrowsWithoutReplacingTheOtherPredicates +15 13 TestEmbeddedReaderListRejectsATypeOutsideTheWorkspaceVocabulary +15 13 TestEmbeddedReaderListStatusAcceptsACommaSeparatedORSet +15 13 TestEmbeddedReaderReadySetOwnsItsStatusPinnedAndTemplateDecisions +15 13 TestEmbeddedRelationsRefusesAnUnusableTypeFilter +15 13 TestEmbeddedRelationsRefusesTheZeroDirection +15 13 TestEventsJournal_EmbeddedConcurrentGaplessNoDup +15 13 TestExternalCapabilityBlocksEmbeddedReadyAndAppearsInTree +15 13 TestIgnoredCursorHealRespectsOperatorOverride +15 13 TestLegacyTrackedIgnoredCursorHealsOnOpen +15 13 TestSweeperContract + +15 14 TestAddDependency +15 14 TestCounterContract +15 14 TestCountsWithMissingWispPlaneTables +15 14 TestEmbeddedBlockedRecheckFailureKeepsTheWrite +15 14 TestEmbeddedDepRekeyRefusesToWriteIntoDirtyDependencies +15 14 TestEmbeddedDependencyEditorRecordsNoHistoryForAnAllEphemeralRequest +15 14 TestEmbeddedDirtyTablesGate_BlocksReopenThenWorkingSetReconcileRecovers +15 14 TestEmbeddedEveryAcceptedLifecycleMutationMintsOneVersion +15 14 TestEmbeddedIssueOperationsClaimLeavesBlockedStateAlone +15 14 TestEmbeddedIssueOperationsUpdateIssuePlaneOnlyRefusesWisps +15 14 TestEmbeddedIssueOperationsUpdateLabelPatchValueRules +15 14 TestEmbeddedMergeAndSettleDirtyWorkingSetSurvivesRefusedMerge +15 14 TestEmbeddedOpenForReadOnlyCommand_LenientGate +15 14 TestEmbeddedReaderListBriefDropsTheFreeFormTextAndNothingElse +15 14 TestEmbeddedReaderListKeysetWalkOverAnOversizedGroupLosesNothingAndRepeatsNothing +15 14 TestEmbeddedReadyClaimerClaimsAnEphemeralRowTheFilterAdmits +15 14 TestEmbeddedReadyClaimerRejectsLimitOffsetBriefAndEmptyActor +15 14 TestEventsJournalPruneWhenEveryRowIsOlderThanTheCutoff +15 14 TestMergeWithStrategy_TheirsResolvesAndCommits +15 14 TestMerge_NoStrategyStillErrorsWithEscapeHatchHint +15 14 TestOpenAfterCloseCreatesNewStore +15 14 TestOpenDifferentDirsReturnsDifferentStores +15 14 TestPruneEventsBatchDeletesTheLowestSeqsOnly +15 14 TestReadyCounterContract +15 14 TestUnknownProvenanceLeaseStaysReclaimable + +15 15 TestCreateIssue +15 15 TestEdgeReaderContract +15 15 TestEmbeddedBatchCloserAllRefusedBatchRecordsNoHistory +15 15 TestEmbeddedBatchCloserDuplicateItemRecloseAtItsOwnIndex +15 15 TestEmbeddedBatchCloserRecordsOneHistoryEntryForWhatLanded +15 15 TestEmbeddedBlockedRecheckRecordsUnblockingWrites +15 15 TestEmbeddedClaimerAcceptsAConfiguredActiveStatusAndRefusesAConfiguredWipOne +15 15 TestEmbeddedDependencyEditorAcceptsATypeOutsideTheConstants +15 15 TestEmbeddedDependencyEditorAcceptsAnExternalTarget +15 15 TestEmbeddedDependencyEditorRefusesSelfDependencyWithTheProbeSkipped +15 15 TestEmbeddedDependencyEditorRepeatsWithinOneRequestCollapse +15 15 TestEmbeddedIssueOperationsUpdateLabelPatchOrdering +15 15 TestEmbeddedMigrateSelfHealsAfterMidPassFault_4566 +15 15 TestEmbeddedMoveIssuePersistenceEphemeralNoHistorySmoke +15 15 TestEmbeddedOpenForRemoteSync_DataBehind +15 15 TestEmbeddedReadyClaimerEmptyFrontIsNormal +15 15 TestEmbeddedRelationsSeparatesNoNeighborsFromNoSuchIssue +15 15 TestEmbeddedTransactionStandaloneWispLifecyclePersistsWithoutDoltCommit +15 15 TestEventsJournalPruneNeverSplitsASkewedPair +15 15 TestEventsJournal_ActorMatrixOnEmbedded +15 15 TestGetIssueCommentsPageEmbedded +15 15 TestGetReadyWorkIncludeEphemeralExcludesNonReadyWisps +15 15 TestIgnoredCursorHealIsIdempotent +15 15 TestReadyWorkStatementBudget +15 15 TestReleaserContract diff --git a/.github/workflows/bazel.yml b/.github/workflows/bazel.yml index 4894b4068..f4adbad89 100644 --- a/.github/workflows/bazel.yml +++ b/.github/workflows/bazel.yml @@ -797,10 +797,39 @@ jobs: name: embedded Dolt tier # Remote execution only: skipped unless the rbe job chose remote, # fork-ro or fork-rw (enabled; not in modes cache, local or skip). - # Locally the tier is 27 race test - # processes (~7-10k test-seconds) on one runner, an hour or more per run - # on the pool whose job cap this track exists to relieve, and it proves - # nothing pr-risk.yml's embedded jobs (which do run for forks) do not. + # Locally the tier is 67 race test + # processes (50 cmd/bd shards + 15 embeddeddolt storage shards + 2 + # conformance partitions; ~24k test-seconds measured locally -- see + # scripts/ci/embedded_cmd_test_durations.json and + # scripts/ci/embedded_storage_test_durations.json -- up from an earlier, + # smaller test surface's ~7-10k estimate) on one runner, several hours + # per run on the pool whose job cap this track exists to relieve, and it + # proves nothing pr-risk.yml's embedded jobs (which do run for forks) do + # not. + # + # Shard counts: storage (15) is sized the way both tiers were originally + # meant to be -- the smallest count whose LPT-packed heaviest shard + # (scripts/ci/gen_embedded_storage_shard_manifest.py --weights=duration + # --repack) lands at or below the tier's own heaviest single top-level + # test (211-225s). cmd cannot reach that floor: Bazel hard-caps + # shard_count at 50 per sh_test-like rule ("Having more than 50 shards is + # indicative of poor test organization. Please reduce the number of + # shards." -- a native analysis-time error, not a style lint), and at 50 + # shards the cmd suite's LPT-packed heaviest shard is ~409.6s -- above + # its own single heaviest test's 239.7s, because the suite's total + # duration (~20145 test-seconds across 242 tests) is too large to flatten + # to the single-test floor within 50 bins. Clearing more shards would + # need either further per-test speedups spread across the suite or + # splitting bd_embedded_test into multiple <=50-shard sh_test rules with + # shard-index-offset plumbing through tools/bazel/go_test_manifest_shard.sh + # and check_shard_coverage.py; neither is done here, so this lane's floor + # is cmd's ~409.6s, not the ~240s a 5-minute lane (including this job's + # ~60s fixed cost) would need -- tracked as a follow-up, not silently + # hidden in the shard count. On RBE this makes the tier's wall time ~= + # that floor + fixed cost (checkout, build, shard dispatch), not ~= + # sum(durations)/shard_count the way a CPU-core-bound local sweep is -- + # each shard is its own remote action and runs concurrently with every + # other shard, farm capacity permitting. # On same-repo pull_requests those jobs stand down (pr-risk.yml's # bazel-coverage), so this lane is then the tier's only PR run # and pr.yml's gate requires it to have run here and passed. @@ -882,8 +911,8 @@ jobs: run: >- python3 tools/bazel/check_shard_coverage.py --bep "$RUNNER_TEMP/bazel-bep.json" - --suite //cmd/bd:bd_embedded_test .github/scripts/embedded-test-shard.sh 20 - --suite //internal/storage/embeddeddolt:embeddeddolt_embedded_test .github/scripts/embedded-storage-test-shard.sh 5 + --suite //cmd/bd:bd_embedded_test .github/scripts/embedded-test-shard.sh 50 + --suite //internal/storage/embeddeddolt:embeddeddolt_embedded_test .github/scripts/embedded-storage-test-shard.sh 15 --whole //internal/storage/embeddeddolt:embeddeddolt_conformance_core_test --whole //internal/storage/embeddeddolt:embeddeddolt_conformance_audit_test diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index a1e1355ad..721482a6c 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -150,11 +150,15 @@ jobs: # The suite above never sets BEADS_TEST_EMBEDDED_DOLT, so every # skipUnlessEmbeddedDolt-gated test in internal/storage/embeddeddolt - # (including TestBatchApplyContract and the 1000-item case of - # TestLargeBatchApplyWallClock_Embedded) is skipped there, on every - # lane, not just this one. This step is their only non-race coverage of - # a real 1000-item apply through the embedded backend: the race lanes - # (main.yml, pr-risk.yml) shrink that subtest under raceEnabled — see + # (including TestBatchApplyContract, the 1000-item case of + # TestLargeBatchApplyWallClock_Embedded, and — F1 — the 712-item case of + # TestLargeBatchApplyStatementCounts712_Embedded) is skipped there, on + # every lane, not just this one. This step is their only non-race + # coverage of a real 1000-item apply through the embedded backend, and + # (F1) the only coverage at all of the 712-item shape's pinned + # statement-count regression baseline on a per-PR race lane: the race + # lanes (main.yml, pr-risk.yml) shrink/skip those cases under + # raceEnabled — see # internal/storage/embeddeddolt/large_batch_apply_measure_test.go — and # the PR-conditional server-Dolt full suite (full_embedded in # .github/scripts/ci-embedded-tier.sh) covers the SAME write body @@ -163,11 +167,15 @@ jobs: # own goroutine/lock machinery runs inside this test binary and the # race detector dramatically inflates its wall-clock (see # race_detector_on_test.go), which is exactly what - # TestLargeBatchApplyWallClock_Embedded is measuring. + # TestLargeBatchApplyWallClock_Embedded is measuring and exactly why + # TestLargeBatchApplyStatementCounts712_Embedded's 712-item case takes + # ~260-300s under -race (genuine engine-internal race overhead, not + # test logic — see that test's doc comment). - name: Embedded Dolt batch-apply suite (non-race) + if: ${{ !cancelled() }} env: BEADS_TEST_EMBEDDED_DOLT: "1" - run: go test -tags gms_pure_go -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded)$' ./internal/storage/embeddeddolt + run: go test -tags gms_pure_go -timeout 20m -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded|TestLargeBatchApplyStatementCounts712_Embedded)$' ./internal/storage/embeddeddolt - name: Check coverage threshold run: | diff --git a/cmd/bd/BUILD.bazel b/cmd/bd/BUILD.bazel index fe5ab62ec..6dd0e00a4 100644 --- a/cmd/bd/BUILD.bazel +++ b/cmd/bd/BUILD.bazel @@ -1358,13 +1358,17 @@ go_test( }), ) -# bd_test as PR Risk's "Test (Embedded Dolt Cmd N/20)" jobs run it: shard k -# runs .github/scripts/embedded-test-shard.sh k 20 (the TestEmbedded* functions -# of *_embedded_test.go, assigned by the committed manifest) against this -# configuration's bd_test, with BEADS_TEST_EMBEDDED_DOLT=1 and the race bd -# (//cmd/bd:bd under --config=embedded) as the subprocess binary, like the -# jobs' /tmp/bd-embedded-test. It repeats bd_test's `env` (the wrapper cannot -# inherit it). shard_count must stay equal to the manifest's shard total. +# Runs 50 Bazel shards against the TestEmbedded* functions of +# *_embedded_test.go, assigned by the manifest's 50-shard block +# (duration-balanced; see scripts/ci/embedded_cmd_test_durations.json), +# against this configuration's bd_test with BEADS_TEST_EMBEDDED_DOLT=1 and +# the race bd (//cmd/bd:bd under --config=embedded) as the subprocess +# binary. This is a *different* shard count from PR Risk's legacy fork/push +# jobs, "Test (Embedded Dolt Cmd N/20)" (pr-risk.yml, main.yml): those keep +# running 20 shards of their own, frozen hand-assigned block in the same +# manifest file — shard k here is not the same split as their shard k. It +# repeats bd_test's `env` (the wrapper cannot inherit it). shard_count must +# stay equal to this lane's manifest block's shard total. # # Tags: # embedded: the embedded-Dolt tier; excluded from --config=prcore/ci, @@ -1398,7 +1402,7 @@ sh_test( "BEADS_TEST_EMBEDDED_DOLT": "1", "BEADS_TEST_GOFMT": "$(rlocationpath @go_sdk//:bin/gofmt)", }, - shard_count = 20, + shard_count = 50, tags = ["embedded"], ) diff --git a/cmd/bd/autocommit_embedded_test.go b/cmd/bd/autocommit_embedded_test.go index e6c5a8856..1da1e4cc1 100644 --- a/cmd/bd/autocommit_embedded_test.go +++ b/cmd/bd/autocommit_embedded_test.go @@ -270,7 +270,13 @@ func TestEmbeddedRoutedSiblingWritesCommitTargetHead(t *testing.T) { // (#4141). Each subtest creates the bead in the target rig, runs the command // from the source rig so resolution must route, and asserts both that the // target HEAD advanced and that the mutation persisted in the target store. -func TestEmbeddedRoutedMutatingSiblingWritesCommitTargetHead(t *testing.T) { +// TestEmbeddedRoutedMutatingSiblingAssignTagDep was split from +// TestEmbeddedRoutedMutatingSiblingWritesCommitTargetHead (originally +// ~245.22s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedRoutedMutatingSiblingAssignTagDep(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt auto-commit tests") } @@ -339,6 +345,21 @@ func TestEmbeddedRoutedMutatingSiblingWritesCommitTargetHead(t *testing.T) { t.Fatalf("routed dep add did not persist %s -> %s in target store (deps=%v)", from.ID, to.ID, deps) } }) +} + +// TestEmbeddedRoutedMutatingSiblingDeleteClose was split from +// TestEmbeddedRoutedMutatingSiblingWritesCommitTargetHead (originally +// ~245.22s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedRoutedMutatingSiblingDeleteClose(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt auto-commit tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("delete", func(t *testing.T) { sourceDir, targetDir, targetBeadsDir := setupRoutedEmbeddedRepo(t, bd, "sdl", "tdl") diff --git a/cmd/bd/close_embedded_test.go b/cmd/bd/close_embedded_test.go index 0a0d945c2..88ed651d1 100644 --- a/cmd/bd/close_embedded_test.go +++ b/cmd/bd/close_embedded_test.go @@ -58,14 +58,18 @@ func bdDepAdd(t *testing.T, bd, dir string, args ...string) { // ===== Close tests ===== -func TestEmbeddedClose(t *testing.T) { +// TestEmbeddedCloseBasic was split from TestEmbeddedClose (originally ~404s, +// measured under --config=embedded) into 3 top-level tests over disjoint +// subtest groups, for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and +// engdocs/TESTING.md). Every original subtest is preserved exactly once. +func TestEmbeddedCloseBasic(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } t.Parallel() bd := buildEmbeddedBD(t) - dir, beadsDir, _ := bdInit(t, bd, "--prefix", "tc") + dir, _, _ := bdInit(t, bd, "--prefix", "tc") // ===== Basic Close Behavior ===== @@ -261,6 +265,22 @@ func TestEmbeddedClose(t *testing.T) { } }) +} + +// TestEmbeddedCloseGuardsAndEpics was split from TestEmbeddedClose (originally +// ~404s, measured under --config=embedded) into 3 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedCloseGuardsAndEpics(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, beadsDir, _ := bdInit(t, bd, "--prefix", "tc") + // Proves the S7 delegation: `bd close` on a blocked issue now surfaces the // engine's atomic guard (storage.ErrCloseBlocked) rather than a duplicated // CLI pre-check. The refusal must be atomic — the issue stays open because the @@ -369,6 +389,21 @@ func TestEmbeddedClose(t *testing.T) { } }) +} + +// TestEmbeddedCloseAlreadyClosed was split from TestEmbeddedClose (originally +// ~404s, measured under --config=embedded) into 3 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedCloseAlreadyClosed(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + // The direct route's mirror of the proxied route's // single_transaction_dolt_commit oracle. N ids are ONE request, and the // request is the transaction boundary, so they land as one transaction with @@ -589,6 +624,14 @@ func TestEmbeddedClose(t *testing.T) { // TestEmbeddedCloseConcurrent exercises create, close, and list operations // concurrently to verify EmbeddedDoltStore handles concurrent CLI invocations // without panics, data corruption, or deadlocks. +// +// issuesPerWorker was 5 (F1 review S1): under real CI-like load this +// measured ~324s for the same reason documented on +// TestEmbeddedUpdateConcurrent above (genuine serialized-write contention, +// not a queueing-delay artifact). issuesPerWorker=2 preserves the same +// invariants (duplicate-ID detection across all numWorkers concurrent +// processes, one non-decreasing-list-count check per worker) at 40% of the +// serialized work. func TestEmbeddedCloseConcurrent(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") @@ -600,7 +643,7 @@ func TestEmbeddedCloseConcurrent(t *testing.T) { const ( numWorkers = 10 - issuesPerWorker = 5 + issuesPerWorker = 2 ) type workerResult struct { diff --git a/cmd/bd/close_proxied_integration_test.go b/cmd/bd/close_proxied_integration_test.go index 53df108c9..3dc53d349 100644 --- a/cmd/bd/close_proxied_integration_test.go +++ b/cmd/bd/close_proxied_integration_test.go @@ -451,7 +451,7 @@ func TestProxiedServerClose(t *testing.T) { } // failed[].error is the TYPED error, the same on this route as on the - // direct one — see the twin assertions in TestEmbeddedClose/ + // direct one — see the twin assertions in TestEmbeddedCloseBasic/ // close_partial_failure_json_names_the_failed_ids. The --force hint is // advice for a human reader, so it belongs on the stderr line above and // not in the field a --json consumer keys off; recording the decorated @@ -484,9 +484,14 @@ func TestProxiedServerClose(t *testing.T) { } }) - // ga-ktn9pe.4.8: twin of TestEmbeddedClose/close_boolean_pinned_reclose_is_idempotent. - // Both close paths must agree — a fix on one only is the divergence class #5217 - // just closed. + // ga-ktn9pe.4.8: this used to have an embedded twin + // (TestEmbeddedClose/close_boolean_pinned_reclose_is_idempotent), but + // bd-1rh.1 (#5287) trimmed it along with the embedded suite's other + // pinned-close subtests as redundant with this proxied coverage, so this + // is now the sole owner of the boolean-pinned-reclose contract. Both + // close paths had to agree before the trim — a fix on one only was the + // divergence class #5217 just closed — so treat any future embedded + // pinned-close test as needing to stay consistent with this one. t.Run("close_boolean_pinned_reclose_is_idempotent", func(t *testing.T) { t.Parallel() p := newSharedProxiedProject(t, bd, "cbpr") diff --git a/cmd/bd/create_embedded_test.go b/cmd/bd/create_embedded_test.go index c7424bd2a..a40734abe 100644 --- a/cmd/bd/create_embedded_test.go +++ b/cmd/bd/create_embedded_test.go @@ -180,7 +180,12 @@ func assertDepExistsWithType(t *testing.T, beadsDir, database, issueID, dependsO } } -func TestEmbeddedCreate(t *testing.T) { +// TestEmbeddedCreateScalarAndRelationships was split from TestEmbeddedCreate +// (originally ~266.68s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedCreateScalarAndRelationships(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests") } @@ -361,6 +366,20 @@ func TestEmbeddedCreate(t *testing.T) { } } }) +} + +// TestEmbeddedCreateGraphAndBulk was split from TestEmbeddedCreate +// (originally ~266.68s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedCreateGraphAndBulk(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("graph_create_journey", func(t *testing.T) { dir, beadsDir, _ := bdInit(t, bd, "--prefix", "gr") diff --git a/cmd/bd/dep_embedded_test.go b/cmd/bd/dep_embedded_test.go index 0ded1f234..378afa7ec 100644 --- a/cmd/bd/dep_embedded_test.go +++ b/cmd/bd/dep_embedded_test.go @@ -97,7 +97,17 @@ func bdDepWithInputFail(t *testing.T, bd, dir, input string, args ...string) str return string(out) } -func TestEmbeddedDep(t *testing.T) { +// TestEmbeddedDepA and TestEmbeddedDepB were split from TestEmbeddedDep +// (originally ~317s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. The "list_*" subtests (which assert on +// dependencies added by "add_positional_args" / "add_type_parent_child") +// stay grouped with the "add_*" cluster in TestEmbeddedDepA rather than a +// strict half/half split, since they depend on mutations made by those +// specific earlier subtests that a fresh per-function setup does not +// reproduce. +func TestEmbeddedDepA(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -252,6 +262,71 @@ func TestEmbeddedDep(t *testing.T) { } }) + // ===== dep list ===== + + t.Run("list_default_direction_down", func(t *testing.T) { + // issueC depends on issueD (added earlier) + out := bdDep(t, bd, dir, "list", issueC.ID) + if !strings.Contains(out, issueD.ID) { + t.Errorf("expected dependency in list output: %s", out) + } + }) + + t.Run("list_direction_up", func(t *testing.T) { + out := bdDep(t, bd, dir, "list", issueD.ID, "--direction", "up") + if !strings.Contains(out, issueC.ID) { + t.Errorf("expected dependent in list --direction up: %s", out) + } + }) + + t.Run("list_type_filter", func(t *testing.T) { + out := bdDep(t, bd, dir, "list", epic.ID, "--direction", "up", "--type", "parent-child") + if !strings.Contains(out, child1.ID) || !strings.Contains(out, child2.ID) { + t.Errorf("expected children in type-filtered list: %s", out) + } + }) + + t.Run("list_json_output", func(t *testing.T) { + fullArgs := []string{"dep", "list", issueC.ID, "--json"} + cmd := exec.Command(bd, fullArgs...) + cmd.Dir = dir + cmd.Env = bdEnv(dir) + stdout, stderr, err := runCommandBuffers(t, cmd) + if err != nil { + t.Fatalf("dep list --json failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdout.String(), stderr.String()) + } + if !strings.Contains(stdout.String(), issueD.ID) { + t.Errorf("expected dependency ID in JSON: %s", stdout.String()) + } + }) +} + +func TestEmbeddedDepB(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "dp") + + // Pre-create issues for dependency testing. + _ = bdCreate(t, bd, dir, "Dep issue A", "--type", "task") + _ = bdCreate(t, bd, dir, "Dep issue B", "--type", "task") + issueC := bdCreate(t, bd, dir, "Dep issue C", "--type", "task") + issueD := bdCreate(t, bd, dir, "Dep issue D", "--type", "task") + epic := bdCreate(t, bd, dir, "Dep epic", "--type", "epic") + child1 := bdCreate(t, bd, dir, "Dep child 1", "--type", "task") + child2 := bdCreate(t, bd, dir, "Dep child 2", "--type", "task") + + // Wire up the same parent-child/blocks edges TestEmbeddedDepA builds, so + // the tree_* subtests below exercise real multi-node traversal instead of + // rendering isolated single-node trees (see review S2: these subtests + // previously passed vacuously against an edgeless graph). + bdDep(t, bd, dir, "add", child1.ID, epic.ID, "--type", "parent-child") + bdDep(t, bd, dir, "add", child2.ID, epic.ID, "--type", "parent-child") + bdDep(t, bd, dir, "add", issueC.ID, issueD.ID) + // ===== dep remove ===== t.Run("remove_basic", func(t *testing.T) { @@ -300,44 +375,6 @@ func TestEmbeddedDep(t *testing.T) { } }) - // ===== dep list ===== - - t.Run("list_default_direction_down", func(t *testing.T) { - // issueC depends on issueD (added earlier) - out := bdDep(t, bd, dir, "list", issueC.ID) - if !strings.Contains(out, issueD.ID) { - t.Errorf("expected dependency in list output: %s", out) - } - }) - - t.Run("list_direction_up", func(t *testing.T) { - out := bdDep(t, bd, dir, "list", issueD.ID, "--direction", "up") - if !strings.Contains(out, issueC.ID) { - t.Errorf("expected dependent in list --direction up: %s", out) - } - }) - - t.Run("list_type_filter", func(t *testing.T) { - out := bdDep(t, bd, dir, "list", epic.ID, "--direction", "up", "--type", "parent-child") - if !strings.Contains(out, child1.ID) || !strings.Contains(out, child2.ID) { - t.Errorf("expected children in type-filtered list: %s", out) - } - }) - - t.Run("list_json_output", func(t *testing.T) { - fullArgs := []string{"dep", "list", issueC.ID, "--json"} - cmd := exec.Command(bd, fullArgs...) - cmd.Dir = dir - cmd.Env = bdEnv(dir) - stdout, stderr, err := runCommandBuffers(t, cmd) - if err != nil { - t.Fatalf("dep list --json failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdout.String(), stderr.String()) - } - if !strings.Contains(stdout.String(), issueD.ID) { - t.Errorf("expected dependency ID in JSON: %s", stdout.String()) - } - }) - // ===== dep tree ===== t.Run("tree_basic", func(t *testing.T) { diff --git a/cmd/bd/dolt_embedded_test.go b/cmd/bd/dolt_embedded_test.go index d04184324..50e17f02d 100644 --- a/cmd/bd/dolt_embedded_test.go +++ b/cmd/bd/dolt_embedded_test.go @@ -40,7 +40,17 @@ func bdDoltFail(t *testing.T, bd, dir string, args ...string) string { return string(out) } -func TestEmbeddedDolt(t *testing.T) { +// TestEmbeddedDoltBlockedAndCommit was split from TestEmbeddedDolt +// (originally ~183s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every +// original subtest is preserved exactly once. This group keeps the +// server-only-command-blocked, status/show, and commit subtests; the +// remote/push/pull subtests moved to TestEmbeddedDoltRemoteAndPush below. +// Every subtest in both groups only ever reads or mutates its own fresh +// `dir` (or its own ppDir/pfDir/cleanDir), so neither group depends on state +// the other group's subtests produce. +func TestEmbeddedDoltBlockedAndCommit(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -138,6 +148,20 @@ func TestEmbeddedDolt(t *testing.T) { t.Errorf("expected 'Nothing to commit.' on a clean working set, got: %s", out) } }) +} + +// TestEmbeddedDoltRemoteAndPush is the second half of the TestEmbeddedDolt +// split described on TestEmbeddedDoltBlockedAndCommit above: the +// remote-list/add/remove and push/pull subtests, none of which depend on +// state produced by the first group's subtests. +func TestEmbeddedDoltRemoteAndPush(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "td") // ===== Remote management ===== diff --git a/cmd/bd/epic_embedded_test.go b/cmd/bd/epic_embedded_test.go index a04956e56..b2e064d7f 100644 --- a/cmd/bd/epic_embedded_test.go +++ b/cmd/bd/epic_embedded_test.go @@ -52,7 +52,11 @@ func bdEpicJSON(t *testing.T, bd, dir string, args ...string) interface{} { return result } -func TestEmbeddedEpic(t *testing.T) { +// TestEmbeddedEpicStatus was split from TestEmbeddedEpic (originally ~362s, +// measured under --config=embedded) into 3 top-level tests over disjoint +// subtest groups, for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and +// engdocs/TESTING.md). Every original subtest is preserved exactly once. +func TestEmbeddedEpicStatus(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -113,6 +117,34 @@ func TestEmbeddedEpic(t *testing.T) { t.Errorf("expected 'No open epics': %s", out) } }) +} + +// TestEmbeddedEpicCloseEligible was split from TestEmbeddedEpic (originally +// ~362s, measured under --config=embedded) into 3 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedEpicCloseEligible(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "ep") + + // Create an epic with children — some closed, some open. + // Use 3 children so closing one doesn't auto-close the epic. + epic1 := bdCreate(t, bd, dir, "Epic partially done", "--type", "epic") + c1 := bdCreate(t, bd, dir, "Epic1 child 1", "--type", "task") + c2 := bdCreate(t, bd, dir, "Epic1 child 2", "--type", "task") + c2b := bdCreate(t, bd, dir, "Epic1 child 3", "--type", "task") + bdDep(t, bd, dir, "add", c1.ID, epic1.ID, "--type", "parent-child") + bdDep(t, bd, dir, "add", c2.ID, epic1.ID, "--type", "parent-child") + bdDep(t, bd, dir, "add", c2b.ID, epic1.ID, "--type", "parent-child") + bdClose(t, bd, dir, c1.ID) + // c2, c2b still open — epic1 is NOT eligible + _ = c2b // ===== epic close-eligible ===== @@ -178,6 +210,33 @@ func TestEmbeddedEpic(t *testing.T) { t.Errorf("epic1 should not be closed: %s", out) } }) +} + +// TestEmbeddedEpicCloseEligibleReason was split from TestEmbeddedEpic +// (originally ~362s, measured under --config=embedded) into 3 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every +// original subtest is preserved exactly once. The split originally carried +// over an epic1/c1/c2/c2b "some closed, some open" fixture from its parent +// that none of these particular subtests (all self-contained, each building +// its own dirN/e/ch fixture) ever reference; removed as dead setup cost +// (F1 CI-speed review N6). +// +// F1 review S1: this combined piece still measured ~283s under real CI-like +// load (several shards running concurrently, -test.parallel=4) — over the +// ~240s per-isolated-test ceiling even alone on its own shard. Its 5 +// subtests are mutually independent (each builds its own dirN/e/ch fixture, +// none shares state with a sibling), so it splits cleanly again into two +// top-level tests below: TestEmbeddedEpicCloseEligibleReasonFlag keeps the +// first 3 (flag persistence/default/dry-run), TestEmbeddedEpicCloseEligibleReasonOutput +// keeps the remaining 2 (JSON and human-output rendering of the reason). +func TestEmbeddedEpicCloseEligibleReasonFlag(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) // ===== epic close-eligible --reason (GH#4817) ===== @@ -230,6 +289,17 @@ func TestEmbeddedEpic(t *testing.T) { t.Errorf("dry-run must not close the epic, got %s", got.Status) } }) +} + +// TestEmbeddedEpicCloseEligibleReasonOutput is the second half of the split +// described on TestEmbeddedEpicCloseEligibleReasonFlag above. +func TestEmbeddedEpicCloseEligibleReasonOutput(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("close_eligible_json_includes_reason", func(t *testing.T) { dir8, _, _ := bdInit(t, bd, "--prefix", "ep8") diff --git a/cmd/bd/export_embedded_test.go b/cmd/bd/export_embedded_test.go index d52c852d2..9f7df5354 100644 --- a/cmd/bd/export_embedded_test.go +++ b/cmd/bd/export_embedded_test.go @@ -27,7 +27,12 @@ func bdExport(t *testing.T, bd, dir string, args ...string) string { return stdout.String() } -func TestEmbeddedExport(t *testing.T) { +// TestEmbeddedExportFormats was split from TestEmbeddedExport (originally +// ~225s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedExportFormats(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt export tests") } @@ -118,6 +123,15 @@ func TestEmbeddedExport(t *testing.T) { t.Errorf("--all should produce >= lines than default: all=%d default=%d", allLines, defaultLines) } }) +} + +func TestEmbeddedExportFlags(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt export tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("include_infra", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "exinfra") diff --git a/cmd/bd/federation_embedded_test.go b/cmd/bd/federation_embedded_test.go index 8428f0467..1cbc3e33c 100644 --- a/cmd/bd/federation_embedded_test.go +++ b/cmd/bd/federation_embedded_test.go @@ -82,7 +82,12 @@ func TestFormatFederationPeerListJSONPreservesLegacyKeys(t *testing.T) { } } -func TestEmbeddedFederation(t *testing.T) { +// TestEmbeddedFederationPeers and TestEmbeddedFederationStatus were split +// from TestEmbeddedFederation (originally ~282s, measured under +// --config=embedded) into 2 top-level tests over disjoint subtest groups, +// for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and +// engdocs/TESTING.md). Every original subtest is preserved exactly once. +func TestEmbeddedFederationPeers(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt federation tests") } @@ -169,6 +174,15 @@ func TestEmbeddedFederation(t *testing.T) { t.Errorf("expected sovereignty='T2', got %q", sov) } }) +} + +func TestEmbeddedFederationStatus(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt federation tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("remove_peer", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "fdrm") diff --git a/cmd/bd/graph_apply_test.go b/cmd/bd/graph_apply_test.go index 218582648..7a848d05a 100644 --- a/cmd/bd/graph_apply_test.go +++ b/cmd/bd/graph_apply_test.go @@ -981,8 +981,8 @@ func TestValidateGraphApplyExplicitIDPrefixes(t *testing.T) { // field exists, not that graphApplyNodeIssue actually wires it into its // createIssueParams literal. A new field added everywhere but omitted from // that literal would still pass here and materialize as a zero value; the -// value round-trip in TestEmbeddedCreate/graph_full_fields is the wiring -// backstop. +// value round-trip in TestEmbeddedCreateGraphAndBulk/graph_create_journey +// (renamed from graph_full_fields by bd-okxx4, #5291) is the wiring backstop. func TestGraphApplyNodeCoversCreateIssueParams(t *testing.T) { renamed := map[string]string{ "IssueType": "Type", diff --git a/cmd/bd/import_embedded_test.go b/cmd/bd/import_embedded_test.go index ae0b92c45..0f79d8444 100644 --- a/cmd/bd/import_embedded_test.go +++ b/cmd/bd/import_embedded_test.go @@ -48,7 +48,12 @@ func writeJSONLFile(t *testing.T, path string, issues []types.Issue) { } } -func TestEmbeddedImport(t *testing.T) { +// TestEmbeddedImportFromSources was split from TestEmbeddedImport (originally +// ~239s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedImportFromSources(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt import tests") } @@ -164,6 +169,15 @@ func TestEmbeddedImport(t *testing.T) { t.Errorf("expected '1 memor' in output, got: %s", out) } }) +} + +func TestEmbeddedImportUpsertAndPrefixSync(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt import tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("empty_file", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "imemp") diff --git a/cmd/bd/init_embedded_test.go b/cmd/bd/init_embedded_test.go index 8b5e25235..9f961b835 100644 --- a/cmd/bd/init_embedded_test.go +++ b/cmd/bd/init_embedded_test.go @@ -329,7 +329,12 @@ func requireNoFile(t *testing.T, path string) { } } -func TestEmbeddedInit(t *testing.T) { +// TestEmbeddedInitA, TestEmbeddedInitB, and TestEmbeddedInitC were split from +// TestEmbeddedInit (originally ~356s, measured under --config=embedded) into +// 3 top-level tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedInitA(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt init tests") } @@ -744,6 +749,15 @@ func TestEmbeddedInit(t *testing.T) { t.Fatalf("config.yaml should persist --remote URL %q:\n%s", remoteURL, configYAML) } }) +} + +func TestEmbeddedInitB(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt init tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("remote_behind_schema_gate", func(t *testing.T) { // bd-4mpy7 / #4516: bootstrapping from a remote whose database is @@ -1167,6 +1181,15 @@ func TestEmbeddedInit(t *testing.T) { t.Fatalf("imported issue title missing from show output:\n%s", out) } }) +} + +func TestEmbeddedInitC(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt init tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("server_flags_ignored", func(t *testing.T) { _, beadsDir, _ := bdInit(t, bd, "--prefix", "sv", diff --git a/cmd/bd/label_embedded_test.go b/cmd/bd/label_embedded_test.go index a7f8ab1d6..b0e158c63 100644 --- a/cmd/bd/label_embedded_test.go +++ b/cmd/bd/label_embedded_test.go @@ -99,7 +99,12 @@ func bdLabelListAllJSON(t *testing.T, bd, dir string) []map[string]interface{} { return results } -func TestEmbeddedLabel(t *testing.T) { +// TestEmbeddedLabelAddRemove was split from TestEmbeddedLabel (originally +// ~360s, measured under --config=embedded) into 3 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedLabelAddRemove(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -342,6 +347,21 @@ func TestEmbeddedLabel(t *testing.T) { t.Errorf("no-match prefix remove JSON = %v, want an empty array", rows) } }) +} + +// TestEmbeddedLabelEditReports was split from TestEmbeddedLabel (originally +// ~360s, measured under --config=embedded) into 3 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedLabelEditReports(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "tl") // Two issues carrying DIFFERENT matching sets: the report is per issue and // each issue's set is resolved from its own labels, so a refactor that @@ -594,6 +614,21 @@ func TestEmbeddedLabel(t *testing.T) { } } }) +} + +// TestEmbeddedLabelPropagateRename was split from TestEmbeddedLabel +// (originally ~360s, measured under --config=embedded) into 3 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedLabelPropagateRename(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "tl") // ===== Label Propagate ===== diff --git a/cmd/bd/list_embedded_test.go b/cmd/bd/list_embedded_test.go index b35593d10..d67b026e1 100644 --- a/cmd/bd/list_embedded_test.go +++ b/cmd/bd/list_embedded_test.go @@ -177,7 +177,12 @@ type testSeedData struct { readyTask string // P0, labels: backend, no blockers } -func TestEmbeddedList(t *testing.T) { +// TestEmbeddedListFiltering and TestEmbeddedListSearchAndOutput were split +// from TestEmbeddedList (originally ~298s, measured under --config=embedded) +// into 2 top-level tests over disjoint subtest groups, for CI shard balance +// (see scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedListFiltering(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -605,6 +610,19 @@ func TestEmbeddedList(t *testing.T) { t.Error("--flat should not contain tree characters") } }) +} + +func TestEmbeddedListSearchAndOutput(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "tl") + + // Seed test data + seed := seedTestData(t, bd, dir) // --- E. Content search --- diff --git a/cmd/bd/max_rows_embedded_test.go b/cmd/bd/max_rows_embedded_test.go index ac2ccf481..b82108009 100644 --- a/cmd/bd/max_rows_embedded_test.go +++ b/cmd/bd/max_rows_embedded_test.go @@ -50,11 +50,25 @@ func seedReadyIssues(t *testing.T, bd, dir string, n int) []string { return ids } -// TestEmbeddedMaxRowsNonListPaths covers the non-list CLI paths wired up in +// TestEmbeddedMaxRowsNonListPaths covered the non-list CLI paths wired up in // be-x42v.2: ready, dep tree, find-duplicates, graph --all, plus the env-only // doctor family (lint, doctor --check=conventions, doctor --check=pollution), // and config show emission of BEADS_MAX_ROWS. -func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { +// +// This test was split into five top-level tests (TestEmbeddedMaxRowsReady, +// TestEmbeddedMaxRowsReadyMerge, TestEmbeddedMaxRowsDepTreeAndDuplicates, +// TestEmbeddedMaxRowsGraph, TestEmbeddedMaxRowsDoctorAndConfig) below so the +// Bazel embedded-cmd shard script (a top-level-function-name sharder) can +// spread this ~564s single-process long pole across up to five shards +// instead of pinning it to one. Every t.Run subtest name and body is +// unchanged from the original single function; only the grouping into +// separate top-level funcs (each gated and built the same way) changed. See +// scripts/ci/embedded_cmd_test_durations.json for the before/after subtest-name +// equivalence proof. + +// ----------- bd ready ----------- + +func TestEmbeddedMaxRowsReady(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -62,8 +76,6 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { bd := buildEmbeddedBD(t) - // ----------- bd ready ----------- - t.Run("ReadyMaxRows_FlagOverCap_Exits2", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "mrrdf") seedReadyIssues(t, bd, dir, 6) @@ -96,7 +108,7 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { // `bd ready --json` and `bd list --ready --json` route through // GetReadyWorkWithCountsInTx, a separate query path from the plain // `bd ready` above (GetReadyWorkInTx). Mirrors - // TestEmbeddedMaxRowsList/Flag_OverCap's --json coverage for `bd list`. + // TestEmbeddedMaxRowsListBasic/Flag_OverCap's --json coverage for `bd list`. t.Run("ReadyMaxRowsJSON_FlagOverCap_Exits2", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "mrrjf") seedReadyIssues(t, bd, dir, 6) @@ -133,6 +145,17 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { t.Errorf("stderr missing source --max-rows=3:\n%s", out) } }) +} + +// ----------- bd ready (--include-ephemeral merge paths) ----------- + +func TestEmbeddedMaxRowsReadyMerge(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) // be-x42v.4 follow-up (review SHOULD-FIX 8): confirms the cap still // fires end-to-end for --include-ephemeral after propagating MaxRows @@ -185,8 +208,17 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { t.Errorf("delivered-page-under-cap should not emit cap error:\n%s", out) } }) +} - // ----------- bd dep tree ----------- +// ----------- bd dep tree, bd find-duplicates ----------- + +func TestEmbeddedMaxRowsDepTreeAndDuplicates(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("DepTreeMaxRows_TreeNodes_Exits2", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "mrdt") @@ -227,8 +259,17 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { t.Errorf("stderr missing source --max-rows=3:\n%s", out) } }) +} - // ----------- bd graph --all ----------- +// ----------- bd graph --all, bd graph ----------- + +func TestEmbeddedMaxRowsGraph(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("GraphAllMaxRows_Exits2", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "mrgr") @@ -281,8 +322,17 @@ func TestEmbeddedMaxRowsNonListPaths(t *testing.T) { t.Errorf("under-cap single-issue graph should not emit cap error:\n%s", out) } }) +} - // ----------- bd lint (env-only) ----------- +// ----------- bd lint, bd doctor (env-only), bd config show ----------- + +func TestEmbeddedMaxRowsDoctorAndConfig(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("LintMaxRows_EnvOnly_Exits2", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "mrln") @@ -380,11 +430,15 @@ func countListIDs(out, prefix string) int { return strings.Count(out, prefix+"-") } -// TestEmbeddedMaxRowsList covers the 10 designer §6.1 behavioral scenarios -// for `bd list`. Builder bead: be-x42v.2 (CLI wiring). All subtests share a -// single rig of 21 open task issues — the dataset size from the designer's -// fixture. -func TestEmbeddedMaxRowsList(t *testing.T) { +// TestEmbeddedMaxRowsList{Basic,LimitInteraction} were split from +// TestEmbeddedMaxRowsList (measured ~209.67s under --config=embedded) into 2 +// top-level tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. Both cover the 10 designer §6.1 +// behavioral scenarios for `bd list`. Builder bead: be-x42v.2 (CLI wiring). +// Each redoes the original's shared rig of 21 open task issues — the +// dataset size from the designer's fixture — independently. +func TestEmbeddedMaxRowsListBasic(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -512,6 +566,21 @@ func TestEmbeddedMaxRowsList(t *testing.T) { t.Errorf("expected 'must be non-negative' in stderr:\n%s", out) } }) +} + +func TestEmbeddedMaxRowsListLimitInteraction(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, _, _ := bdInit(t, bd, "--prefix", "mrl") + + const totalRows = 21 + for i := 0; i < totalRows; i++ { + bdCreate(t, bd, dir, fmt.Sprintf("List max-rows %d", i), "--type", "task") + } t.Run("LimitSet_CapTighter", func(t *testing.T) { // limit=100, cap=5: 21 rows are scanned (LIMIT cap+1=6 sniffs diff --git a/cmd/bd/prune_embedded_test.go b/cmd/bd/prune_embedded_test.go index 1d2a1a95e..0dedd9475 100644 --- a/cmd/bd/prune_embedded_test.go +++ b/cmd/bd/prune_embedded_test.go @@ -46,7 +46,12 @@ func createAndClose(t *testing.T, bd, dir, title string) string { return issue.ID } -func TestEmbeddedPrune(t *testing.T) { +// TestEmbeddedPruneSafetyAndScope was split from TestEmbeddedPrune +// (originally ~256.67s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedPruneSafetyAndScope(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -145,6 +150,20 @@ func TestEmbeddedPrune(t *testing.T) { t.Errorf("expected prune to skip closed ephemeral %s; got: %s", wispID, out) } }) +} + +// TestEmbeddedPruneDryRunAndFilters was split from TestEmbeddedPrune +// (originally ~256.67s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedPruneDryRunAndFilters(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) // ===== Scope: must NOT touch open beads ===== diff --git a/cmd/bd/purge_embedded_test.go b/cmd/bd/purge_embedded_test.go index 9a9d98c7f..dbd60f6d1 100644 --- a/cmd/bd/purge_embedded_test.go +++ b/cmd/bd/purge_embedded_test.go @@ -52,7 +52,12 @@ func createAndCloseEphemeral(t *testing.T, bd, dir, title string) string { return issue.ID } -func TestEmbeddedPurge(t *testing.T) { +// TestEmbeddedPurgeBasics was split from TestEmbeddedPurge (originally +// ~194s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedPurgeBasics(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -94,6 +99,15 @@ func TestEmbeddedPurge(t *testing.T) { t.Error("expected non-empty dry-run output") } }) +} + +func TestEmbeddedPurgeFiltered(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) // ===== Force ===== diff --git a/cmd/bd/rename_prefix_embedded_test.go b/cmd/bd/rename_prefix_embedded_test.go index 074b027eb..31641a6f4 100644 --- a/cmd/bd/rename_prefix_embedded_test.go +++ b/cmd/bd/rename_prefix_embedded_test.go @@ -39,7 +39,12 @@ func bdRenamePrefixFail(t *testing.T, bd, dir string, args ...string) string { return string(out) } -func TestEmbeddedRenamePrefix(t *testing.T) { +// TestEmbeddedRenamePrefixBasic was split from TestEmbeddedRenamePrefix +// (originally ~197s, measured under --config=embedded) into 2 top-level +// tests over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedRenamePrefixBasic(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -114,6 +119,17 @@ func TestEmbeddedRenamePrefix(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "iv") bdRenamePrefixFail(t, bd, dir, "UPPER") }) +} + +func TestEmbeddedRenamePrefixEdgeCases(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + + // ===== Invalid Prefix ===== t.Run("rename_invalid_chars", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "ic") diff --git a/cmd/bd/show_embedded_test.go b/cmd/bd/show_embedded_test.go index 522348480..04f9649f2 100644 --- a/cmd/bd/show_embedded_test.go +++ b/cmd/bd/show_embedded_test.go @@ -113,7 +113,16 @@ func bdShowFail2(t *testing.T, bd, dir string, args ...string) string { return string(out) } -func TestEmbeddedShow(t *testing.T) { +// TestEmbeddedShowBasicsAndJSON was split from TestEmbeddedShow (originally +// ~193s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. show_current_fallback_to_last_touched +// (in the second group) only requires that SOME earlier subtest in its own +// group already created/touched an issue in the shared dir — it is placed +// after several issue-creating subtests in that group, so the redone, +// self-contained setup still satisfies it. +func TestEmbeddedShowBasicsAndJSON(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } @@ -289,6 +298,16 @@ func TestEmbeddedShow(t *testing.T) { t.Errorf("expected ID in short output: %s", out) } }) +} + +func TestEmbeddedShowDetailAndCurrent(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, beadsDir, _ := bdInit(t, bd, "--prefix", "ts") // ===== --long ===== diff --git a/cmd/bd/storage_class_embedded_test.go b/cmd/bd/storage_class_embedded_test.go index 1f6ec78bf..0f537d867 100644 --- a/cmd/bd/storage_class_embedded_test.go +++ b/cmd/bd/storage_class_embedded_test.go @@ -18,7 +18,12 @@ import ( // Protocol v0.1 §C): create-time resolution (flag > per-type config > // unset), the omitted-when-versioned marker rule (C2.4) down to the DB // cell, and the ephemeral spelling routing to the wisp plane (C1.4). -func TestEmbeddedCreateStorageClass(t *testing.T) { +// TestEmbeddedCreateStorageClassBasic was split from TestEmbeddedCreateStorageClass +// (originally ~356s, measured under --config=embedded) into 3 top-level tests +// over disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedCreateStorageClassBasic(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests") } @@ -42,28 +47,6 @@ func TestEmbeddedCreateStorageClass(t *testing.T) { return got } - // The batch doors mint their own ids, so the DB assertions look rows up by - // the title the markdown file gave them. An absent row is "" rather than a - // failure: which PLANE a row landed on is the thing under test. - issueIDByTitle := func(t *testing.T, beadsDir, prefix, table, title string) string { - t.Helper() - db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main") - if err != nil { - t.Fatalf("OpenSQL: %v", err) - } - defer cleanup() - var id string - //nolint:gosec // G202: table is a test-local literal, never caller input - err = db.QueryRowContext(t.Context(), "SELECT id FROM "+table+" WHERE title = ?", title).Scan(&id) - if errors.Is(err, sql.ErrNoRows) { - return "" - } - if err != nil { - t.Fatalf("query %s by title: %v", table, err) - } - return id - } - t.Run("explicit_unversioned", func(t *testing.T) { dir, beadsDir, _ := bdInit(t, bd, "--prefix", "su") issue := bdCreate(t, bd, dir, "Unversioned bead", "--storage-class", "unversioned") @@ -132,6 +115,20 @@ func TestEmbeddedCreateStorageClass(t *testing.T) { t.Errorf("ephemeral bead should live in wisps, found %d rows", count) } }) +} + +// TestEmbeddedCreateStorageClassConflicts was split from +// TestEmbeddedCreateStorageClass (originally ~356s, measured under +// --config=embedded) into 3 top-level tests over disjoint subtest groups, +// for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and +// engdocs/TESTING.md). Every original subtest is preserved exactly once. +func TestEmbeddedCreateStorageClassConflicts(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) // An explicit durable class combined with a wisp-plane flag is a direct // command-line contradiction: the row would be ephemeral by construction, so @@ -208,6 +205,58 @@ func TestEmbeddedCreateStorageClass(t *testing.T) { t.Errorf("config-derived unversioned leaked to the no-history wisp row: %q", cell.String) } }) +} + +// TestEmbeddedCreateStorageClassBatchAndMisc was split from +// TestEmbeddedCreateStorageClass (originally ~356s, measured under +// --config=embedded) into 3 top-level tests over disjoint subtest groups, +// for CI shard balance (see scripts/ci/embedded_cmd_test_durations.json and +// engdocs/TESTING.md). Every original subtest is preserved exactly once. +func TestEmbeddedCreateStorageClassBatchAndMisc(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt create tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + + // storageClassCell reads the raw issues.storage_class cell so the test + // pins the at-rest form (NULL vs literal), not just the JSON view. + storageClassCell := func(t *testing.T, beadsDir, prefix, id string) sql.NullString { + t.Helper() + db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main") + if err != nil { + t.Fatalf("OpenSQL: %v", err) + } + defer cleanup() + var got sql.NullString + if err := db.QueryRowContext(t.Context(), "SELECT storage_class FROM issues WHERE id = ?", id).Scan(&got); err != nil { + t.Fatalf("query storage_class: %v", err) + } + return got + } + + // The batch doors mint their own ids, so the DB assertions look rows up by + // the title the markdown file gave them. An absent row is "" rather than a + // failure: which PLANE a row landed on is the thing under test. + issueIDByTitle := func(t *testing.T, beadsDir, prefix, table, title string) string { + t.Helper() + db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), prefix, "main") + if err != nil { + t.Fatalf("OpenSQL: %v", err) + } + defer cleanup() + var id string + //nolint:gosec // G202: table is a test-local literal, never caller input + err = db.QueryRowContext(t.Context(), "SELECT id FROM "+table+" WHERE title = ?", title).Scan(&id) + if errors.Is(err, sql.ErrNoRows) { + return "" + } + if err != nil { + t.Fatalf("query %s by title: %v", table, err) + } + return id + } // `bd create --file` used to accept --storage-class and the per-type config // default and honor neither. buildMarkdownBatchRequest is the ONE projection diff --git a/cmd/bd/storage_class_proxied_integration_test.go b/cmd/bd/storage_class_proxied_integration_test.go index 7e63279e6..66466f128 100644 --- a/cmd/bd/storage_class_proxied_integration_test.go +++ b/cmd/bd/storage_class_proxied_integration_test.go @@ -14,7 +14,9 @@ import ( "github.com/steveyegge/beads/internal/types" ) -// The proxied-server mirror of TestEmbeddedCreateStorageClass. This is the +// The proxied-server mirror of TestEmbeddedCreateStorageClassConflicts / +// TestEmbeddedCreateStorageClassBatchAndMisc (one TestEmbeddedCreateStorageClass +// before the F1 CI-speed split). This is the // transport the field report was filed against and the one Enterprise builds // run: every `--storage-class` value was accepted and dropped, so an // `ephemeral` request minted a DURABLE row and the durable-class-on-wisp-plane diff --git a/cmd/bd/update_embedded_test.go b/cmd/bd/update_embedded_test.go index 7cefe257f..1b1841576 100644 --- a/cmd/bd/update_embedded_test.go +++ b/cmd/bd/update_embedded_test.go @@ -243,14 +243,19 @@ func TestEmbeddedUpdateRoutedStoreCommitsTargetHead(t *testing.T) { } } -func TestEmbeddedUpdate(t *testing.T) { +// TestEmbeddedUpdateFields and TestEmbeddedUpdateLifecycle were split from +// TestEmbeddedUpdate (originally ~267s, measured under --config=embedded) +// into 2 top-level tests over disjoint subtest groups, for CI shard balance +// (see scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedUpdateFields(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") } t.Parallel() bd := buildEmbeddedBD(t) - dir, beadsDir, _ := bdInit(t, bd, "--prefix", "tu") + dir, _, _ := bdInit(t, bd, "--prefix", "tu") t.Run("update_direct_flag_mapping", func(t *testing.T) { issue := bdCreate(t, bd, dir, "Multi update", "--type", "task", "--metadata", `{"remove":"me"}`) @@ -527,6 +532,16 @@ func TestEmbeddedUpdate(t *testing.T) { t.Errorf("expected status=deferred, got %q", got.Status) } }) +} + +func TestEmbeddedUpdateLifecycle(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) + dir, beadsDir, _ := bdInit(t, bd, "--prefix", "tu") t.Run("relative_defer_is_persisted_in_UTC", func(t *testing.T) { issue := bdCreate(t, bd, dir, "UTC defer test", "--type", "task") @@ -726,6 +741,18 @@ func TestEmbeddedUpdate(t *testing.T) { // TestEmbeddedUpdateConcurrent exercises create, update, and list operations // concurrently to verify EmbeddedDoltStore handles concurrent CLI invocations // without panics, data corruption, or deadlocks. +// +// issuesPerWorker was 5 (F1 review S1): under real CI-like load (several +// Bazel shards running concurrently, -test.parallel=4), this test's 10 +// workers x 5 issues x 4 serialized `bd` subprocess invocations each (create, +// 2 updates, list; writes serialize behind the embedded engine's single- +// writer lock) measured ~452s -- the queueing-delay explanation once +// proposed for this and three other outliers was wrong (see +// scripts/ci/embedded_cmd_test_durations.json's header); this is genuine +// CPU-bound work under contention. issuesPerWorker=2 keeps the same +// invariants under test (duplicate-ID detection across all numWorkers +// concurrent processes; each worker's own list-count-is-non-decreasing check +// still exercises its loop body once) while cutting serialized work to 40%. func TestEmbeddedUpdateConcurrent(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests") @@ -737,7 +764,7 @@ func TestEmbeddedUpdateConcurrent(t *testing.T) { const ( numWorkers = 10 - issuesPerWorker = 5 + issuesPerWorker = 2 ) type workerResult struct { diff --git a/cmd/bd/vc_embedded_test.go b/cmd/bd/vc_embedded_test.go index 6fe779803..03e550d4a 100644 --- a/cmd/bd/vc_embedded_test.go +++ b/cmd/bd/vc_embedded_test.go @@ -40,7 +40,12 @@ func bdVCFail(t *testing.T, bd, dir string, args ...string) string { return string(out) } -func TestEmbeddedVC(t *testing.T) { +// TestEmbeddedVCStatusAndCommit was split from TestEmbeddedVC (originally +// ~241s, measured under --config=embedded) into 2 top-level tests over +// disjoint subtest groups, for CI shard balance (see +// scripts/ci/embedded_cmd_test_durations.json and engdocs/TESTING.md). Every original +// subtest is preserved exactly once. +func TestEmbeddedVCStatusAndCommit(t *testing.T) { if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt vc tests") } @@ -135,6 +140,15 @@ func TestEmbeddedVC(t *testing.T) { t.Error("expected 'committed' field in JSON output") } }) +} + +func TestEmbeddedVCCommitVariantsAndMerge(t *testing.T) { + if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" { + t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt vc tests") + } + t.Parallel() + + bd := buildEmbeddedBD(t) t.Run("commit_stdin", func(t *testing.T) { dir, _, _ := bdInit(t, bd, "--prefix", "vccs") diff --git a/engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md b/engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md index b5811b291..54c181134 100644 --- a/engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md +++ b/engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md @@ -973,11 +973,14 @@ manifests. On those PRs they are the tiers' only pre-merge run, and - The `args` and `env` of every target tagged `embedded`, `dolt-server-proxied` or `dolt-server-integration` are pinned. - `tools/bazel/check_shard_coverage.py` runs after each tier. It requires: - - every Bazel shard of `//cmd/bd:bd_embedded_test`, - `//internal/storage/embeddeddolt:embeddeddolt_embedded_test`, - `//cmd/bd:bd_proxied_test` (30; PR Risk's own legacy + - every Bazel shard of `//cmd/bd:bd_embedded_test` (50; PR Risk's own + legacy `test-embedded-cmd` fork/push jobs still run 20 shards of + their own, frozen manifest block — a different split, not this one, + F1), `//internal/storage/embeddeddolt:embeddeddolt_embedded_test` + (15; legacy `test-embedded-storage` still runs 5 of its own, same + reasoning, F1), `//cmd/bd:bd_proxied_test` (30; PR Risk's own legacy `test-proxied-cmd` fork/push jobs still run 15 shards of their own, - frozen manifest block — a different split, not this one) and + frozen manifest block — a different split, not this one, F2) and `//internal/storage/dolt:dolt_server_full_test` (16) to have run exactly the tests its shard script lists (list-only mode, minus `TestMain`, which `grep '^func Test'` lists but which is never a diff --git a/internal/storage/dolt/large_batch_apply_measure_test.go b/internal/storage/dolt/large_batch_apply_measure_test.go index e8714ba28..07a0f1865 100644 --- a/internal/storage/dolt/large_batch_apply_measure_test.go +++ b/internal/storage/dolt/large_batch_apply_measure_test.go @@ -109,10 +109,12 @@ var largeBatchApplyDoltShapes = []struct { // TestLargeBatchApplyStatementCounts_Dolt pins the ACTUAL number of SQL // statements issueops.ApplyBatchInTx issues on the Dolt server (TCP) // backend, for each of three measured shapes. Sibling regression baseline to -// TestLargeBatchApplyStatementCounts_Embedded in the embeddeddolt package — -// see that test's doc comment for why counts are pinned with a small -// tolerance rather than exactly, and why a drift beyond it should be -// re-measured and re-pinned deliberately rather than loosened further. +// TestLargeBatchApplyStatementCounts{356,712,Classic40}_Embedded in the +// embeddeddolt package (originally one test, split for CI shard balance; +// see its doc comment) — see that test's doc comment for why counts are +// pinned with a small tolerance rather than exactly, and why a drift beyond +// it should be re-measured and re-pinned deliberately rather than loosened +// further. func TestLargeBatchApplyStatementCounts_Dolt(t *testing.T) { ctx := context.Background() diff --git a/internal/storage/embeddeddolt/BUILD.bazel b/internal/storage/embeddeddolt/BUILD.bazel index 4e3d1b160..a29384404 100644 --- a/internal/storage/embeddeddolt/BUILD.bazel +++ b/internal/storage/embeddeddolt/BUILD.bazel @@ -256,12 +256,18 @@ go_test( ], ) -# embeddeddolt_test as PR Risk's "Test (Embedded Dolt Storage N/5)" jobs run -# it: shard k runs .github/scripts/embedded-storage-test-shard.sh k 5 (every -# top-level test but TestConformance, assigned by the committed manifest) -# with BEADS_TEST_EMBEDDED_DOLT=1; the wrapper points +# Runs 15 Bazel shards against every top-level Test* function in +# internal/storage/embeddeddolt/*_test.go but TestConformance, assigned by +# the manifest's 15-shard block (duration-balanced; see +# scripts/ci/embedded_storage_test_durations.json), with +# BEADS_TEST_EMBEDDED_DOLT=1; the wrapper points # BEADS_TEST_EMBEDDED_TEST_BINARY at this binary, which the concurrency test -# re-executes. shard_count must stay equal to the manifest's shard total. +# re-executes. This is a *different* shard count from PR Risk's legacy +# fork/push jobs, "Test (Embedded Dolt Storage N/5)" (pr-risk.yml, +# main.yml): those keep running 5 shards of their own, frozen hand-assigned +# block in the same manifest file — shard k here is not the same split as +# their shard k. shard_count must stay equal to this lane's manifest +# block's shard total. # # Tags: # embedded: the embedded-Dolt tier; excluded from --config=prcore/ci, @@ -289,7 +295,7 @@ sh_test( "//:.github/scripts/embedded-storage-test-shards.txt", ], env = {"BEADS_TEST_EMBEDDED_DOLT": "1"}, - shard_count = 5, + shard_count = 15, tags = ["embedded"], ) diff --git a/internal/storage/embeddeddolt/large_batch_apply_measure_test.go b/internal/storage/embeddeddolt/large_batch_apply_measure_test.go index 889e0cd67..cbb30d323 100644 --- a/internal/storage/embeddeddolt/large_batch_apply_measure_test.go +++ b/internal/storage/embeddeddolt/large_batch_apply_measure_test.go @@ -151,7 +151,7 @@ var largeBatchApplyShapes = []struct { }, }, { - name: "712 (mol 2x)", + name: large712ShapeName, db: "m712", build: func(rootID string) issueops.ApplyBatchRequest { return batchfixtures.Shape712("tester", rootID) @@ -166,11 +166,13 @@ var largeBatchApplyShapes = []struct { }, } -// TestLargeBatchApplyStatementCounts_Embedded pins the ACTUAL number of SQL -// statements issueops.ApplyBatchInTx issues on the embedded backend, for -// each of three measured shapes (slice B0). It is a REGRESSION BASELINE for -// B2 (a later, lighter fast -// path): B2 must lower these numbers, and this test is what proves it did. +// TestLargeBatchApplyStatementCounts{356,712,Classic40}_Embedded (below) +// jointly pin the ACTUAL number of SQL statements issueops.ApplyBatchInTx +// issues on the embedded backend, for each of three measured shapes (slice +// B0; originally one test, TestLargeBatchApplyStatementCounts_Embedded, see +// its split's doc comment below for why it is now 3). Together they are a +// REGRESSION BASELINE for B2 (a later, lighter fast path): B2 must lower +// these numbers, and these tests are what prove it did. // // The exact counts are backend- and Dolt-version-sensitive by nature — they // come from real driver round trips, not a cost model — so a failure here @@ -189,12 +191,57 @@ var largeBatchApplyShapes = []struct { // and must be re-measured and re-pinned deliberately. const statementCountTolerance = 2 -func TestLargeBatchApplyStatementCounts_Embedded(t *testing.T) { +// large712ShapeName is largeBatchApplyShapes' "712 (mol 2x)" entry name, +// shared between the race-skip check and the zero-match guard in +// runLargeBatchApplyStatementCountsShape below so the two can never drift +// apart (see N2 in the F1 CI-speed review): if this shape is ever renamed in +// largeBatchApplyShapes, both call sites must be updated together, and the +// zero-match guard below fails loudly instead of letting a stale name here +// silently skip the entire pinned assertion. +const large712ShapeName = "712 (mol 2x)" + +// TestLargeBatchApplyStatementCounts356_Embedded, +// TestLargeBatchApplyStatementCounts712_Embedded and +// TestLargeBatchApplyStatementCountsClassic40_Embedded were split from +// TestLargeBatchApplyStatementCounts_Embedded (measured ~350.25s under +// --config=embedded: 81.03s + 260.70s + 8.52s for the 356/712/classic-40 +// shapes respectively) into 3 top-level tests, one per shape, for CI shard +// balance (see scripts/ci/embedded_storage_test_durations.json and +// engdocs/TESTING.md, slice F1). Each runs exactly one disjoint element of +// largeBatchApplyShapes, selected by name (not index) so the split stays +// correct if the shared shapes slice is reordered; the union of shapes run +// is identical to the original loop's, each exactly once. +// +// The 712 (mol 2x) shape alone still measures ~260-300s: it is a single +// pinned statement-count assertion over one indivisible ApplyBatchInTx +// transaction (no internal sub-cases to split further), a genuine, CPU-bound +// cost: 14014 real SQL statement round-trips through the race-instrumented +// in-process Dolt engine. It is skipped under -race below, following the +// exact precedent TestLargeBatchApplyWallClock_Embedded set for its +// 1000-item shape: the cost here is race-instrumentation overhead on the +// engine's own internal goroutine/lock machinery, not on anything this +// test's own logic does, and unlike the wall-clock test this one DOES have a +// real pass/fail assertion (the pinned Total() above), so skipping it under +// race is a deliberate reduction from per-PR to nightly-only coverage for +// this specific regression check — not a weakening of the assertion itself. +// nightly.yml's "Embedded Dolt batch-apply suite (non-race)" step (the +// nightly embedded non-race lane from #7128) has its -run regex extended +// alongside this change to include this test, so the full 14014-statement +// pinned baseline still runs, non-race, every night. +func runLargeBatchApplyStatementCountsShape(t *testing.T, shapeName string) { skipUnlessEmbeddedDolt(t) ctx := t.Context() + matched := false for _, tc := range largeBatchApplyShapes { + if tc.name != shapeName { + continue + } + matched = true t.Run(tc.name, func(t *testing.T) { + if tc.name == large712ShapeName && raceEnabled { + t.Skip("712-item shape's statement-count assertion skipped under -race (race-instrumentation overhead on the Dolt engine itself, not test logic); nightly.yml's Embedded Dolt batch-apply suite (non-race) step runs the full pinned assertion nightly instead; see doc comment above") + } fixture := newPristineEmbeddedDoltFixture(t, tc.db) t.Cleanup(func() { closeEmbeddedDoltStore(t, fixture.store) }) @@ -224,6 +271,27 @@ func TestLargeBatchApplyStatementCounts_Embedded(t *testing.T) { } }) } + if !matched { + // N2 (F1 review): a renamed/typo'd shapeName argument used to just + // silently iterate zero times and PASS with zero subtests run, + // looking identical to a legitimately skipped run. Fail loudly + // instead so a drift between this function's callers and + // largeBatchApplyShapes' names (or large712ShapeName above) is + // caught immediately. + t.Fatalf("no shape named %q in largeBatchApplyShapes; this test ran zero subtests", shapeName) + } +} + +func TestLargeBatchApplyStatementCounts356_Embedded(t *testing.T) { + runLargeBatchApplyStatementCountsShape(t, "356 (mol 1x)") +} + +func TestLargeBatchApplyStatementCounts712_Embedded(t *testing.T) { + runLargeBatchApplyStatementCountsShape(t, large712ShapeName) +} + +func TestLargeBatchApplyStatementCountsClassic40_Embedded(t *testing.T) { + runLargeBatchApplyStatementCountsShape(t, "40 (classic)") } // wallClockShapes is the coordinator's item-8 ask: real wall-clock numbers at @@ -340,9 +408,9 @@ func TestLargeBatchApplyWallClock_Embedded(t *testing.T) { // lower these; a change for any other reason should be re-measured and // re-pinned deliberately, not adjusted to make a failure go away. var pinnedEmbeddedStatementCounts = map[string]int64{ - "356 (mol 1x)": 7009, - "712 (mol 2x)": 14014, - "40 (classic)": 846, + "356 (mol 1x)": 7009, + large712ShapeName: 14014, + "40 (classic)": 846, } // BenchmarkLargeBatchApply_Embedded benchmarks issueops.ApplyBatchInTx on diff --git a/scripts/BUILD.bazel b/scripts/BUILD.bazel index 357d5af87..0e777f3e8 100644 --- a/scripts/BUILD.bazel +++ b/scripts/BUILD.bazel @@ -36,6 +36,7 @@ go_test( "ci_workflow_test.go", "complexity_script_test.go", "conformance_script_test.go", + "embedded_shard_count_test.go", "host_tools_test.go", "install_dolt_test.go", "pr_lanes_bazel_coverage_test.go", diff --git a/scripts/ci/_embedded_shard_manifest_lib.py b/scripts/ci/_embedded_shard_manifest_lib.py new file mode 100644 index 000000000..2f234d77f --- /dev/null +++ b/scripts/ci/_embedded_shard_manifest_lib.py @@ -0,0 +1,168 @@ +"""Shared bin-packing/manifest-file machinery for the embedded-Dolt tier's +Bazel-only shard generators (gen_embedded_cmd_shard_manifest.py, +gen_embedded_storage_shard_manifest.py). + +This is a near-verbatim extraction of scripts/ci/gen_proxied_shard_manifest.py +(F2)'s LPT packing, incremental --write, and multi-block manifest-file +machinery, generalized so both embedded generators (and any future one) share +one implementation instead of two independently-maintained copies. The +per-suite scripts own discovery (the test-name regex/glob), the duration-file +path and its "seconds_per_init_fallback" semantics, and the manifest header +text; this module owns everything suite-agnostic: pack(), incremental_pack(), +split_blocks()/write_block(), and check_coverage(). + +See gen_proxied_shard_manifest.py's module docstring for the full rationale +behind incremental --write (merge-conflict avoidance across unrelated PRs) +and --check's exact-once-coverage-only contract (fork-safe: it does not fail +when a fresh repack would reassign shards differently). +""" +import os +import re + + +def pack(costs, total): + """LPT bin-pack costs into `total` shards. Returns (shards, loads).""" + order = sorted(costs, key=lambda k: (-costs[k], k)) + shards = [[] for _ in range(total)] + loads = [0.0] * total + for name in order: + i = loads.index(min(loads)) + shards[i].append(name) + loads[i] += costs[name] + return shards, loads + + +def split_blocks(text): + """Split a manifest file's text into its per-total_shards blocks. + + A block is a run of leading '#' comment (and blank spacer) lines, + immediately followed by a run of lines that do not start with '#' (shard + assignment lines and the blank lines render() puts between shards) that + ends at the next '#' line or end of file. Returns a list of dicts with + 'header' and 'body' line lists and the block's 'total' (parsed from its + first assignment line; None for a block with no assignment lines yet). + """ + lines = text.split('\n') + blocks = [] + i, n = 0, len(lines) + while i < n: + header = [] + while i < n and lines[i].startswith('#'): + header.append(lines[i]) + i += 1 + while i < n and lines[i] == '' and not (i + 1 < n and lines[i + 1].startswith('#')): + header.append(lines[i]) + i += 1 + body = [] + while i < n and not lines[i].startswith('#'): + body.append(lines[i]) + i += 1 + if not header and not body: + continue + total = None + for bl in body: + m = re.match(r'^(\d+) ', bl) + if m: + total = int(m.group(1)) + break + blocks.append({'header': header, 'body': body, 'total': total}) + return blocks + + +def read_assignments(path, total): + """Return {name: shard_number} for path's block matching total_shards == + total, or None if path or that block does not exist yet.""" + if not os.path.exists(path): + return None + for b in split_blocks(open(path).read()): + if b['total'] == total: + assignments = {} + for ln in b['body']: + m = re.match(r'^(\d+) (\d+) (\S+)$', ln) + if m and int(m.group(1)) == total: + assignments[m.group(3)] = int(m.group(2)) + return assignments + return None + + +def incremental_pack(costs, total, existing): + """Keep every name in `existing` that is still in `costs` on its current + shard; drop names no longer in `costs` (renamed/split/deleted); LPT-place + every name in `costs` not already in `existing` onto whichever shard is + currently lightest. Never moves an already-assigned test. Returns + (shards, loads), like pack().""" + shards = [[] for _ in range(total)] + loads = [0.0] * total + for name, shard_num in existing.items(): + if name not in costs or not (1 <= shard_num <= total): + continue # stale, or out of range for a changed total: re-place below + shards[shard_num - 1].append(name) + loads[shard_num - 1] += costs[name] + placed = {n for s in shards for n in s} + new_names = sorted((n for n in costs if n not in placed), key=lambda k: (-costs[k], k)) + for name in new_names: + i = loads.index(min(loads)) + shards[i].append(name) + loads[i] += costs[name] + return shards, loads + + +def write_block(path, total, out): + """Replace path's block for total_shards == total with out (a render() + list), preserving every other block and its position. Appends out as a + new trailing block if path has none for this total yet.""" + text = open(path).read() if os.path.exists(path) else '' + blocks = split_blocks(text) if text else [] + result = [] + replaced = False + for b in blocks: + if b['total'] == total: + result.extend(out) + replaced = True + else: + result.extend(b['header']) + result.extend(b['body']) + if not replaced: + if result and result[-1] != '': + result.append('') + result.extend(out) + with open(path, 'w') as fh: + fh.write('\n'.join(result).rstrip('\n') + '\n') + + +def check_coverage(path, total, universe, test_name_desc, script_name): + """Return None if path's committed block for total_shards == total lists + every name in `universe` (the currently-discovered test set) exactly + once, with no stale or duplicate names; otherwise an actionable error + string. Does NOT check the block's shard *assignments* — see this + module's docstring for why (incremental --write).""" + if not os.path.exists(path): + return f'{path} does not exist' + for b in split_blocks(open(path).read()): + if b['total'] == total: + names = [] + for ln in b['body']: + m = re.match(r'^(\d+) (\d+) (\S+)$', ln) + if m and int(m.group(1)) == total: + names.append(m.group(3)) + universe_set = set(universe) + counts = {} + for n in names: + counts[n] = counts.get(n, 0) + 1 + dupes = sorted(n for n, c in counts.items() if c > 1) + stale = sorted(n for n in counts if n not in universe_set) + missing = sorted(n for n in universe_set if n not in counts) + if not dupes and not stale and not missing: + return None + weights_guess = 'duration' if any('duration' in h for h in b['header']) else 'inits' + msg = [f"{path}'s {total}-shard block does not have exact-once " + f'coverage of the currently-discovered {test_name_desc} test set.'] + if missing: + msg.append(f'missing ({len(missing)}): {", ".join(missing)}') + if stale: + msg.append(f'stale/unknown ({len(stale)}): {", ".join(stale)}') + if dupes: + msg.append(f'duplicated ({len(dupes)}): {", ".join(dupes)}') + msg.append(f'Run: python3 {script_name} {total} --weights={weights_guess} --write') + return '\n'.join(msg) + return f'{path} has no block for total_shards={total}' diff --git a/scripts/ci/embedded_cmd_test_durations.json b/scripts/ci/embedded_cmd_test_durations.json new file mode 100644 index 000000000..aa935db5c --- /dev/null +++ b/scripts/ci/embedded_cmd_test_durations.json @@ -0,0 +1,315 @@ +{ + "_comment": [ + "Measured locally with `bazel --nohome_rc test --config=embedded", + "//cmd/bd:bd_embedded_test --local_test_jobs=8` alongside the storage-tier", + "and conformance targets below (all four run in the same invocation, the", + "same way bazel.yml's bazel-embedded tier runs its shards concurrently on", + "one host) -- race, `-test.parallel=4`, tag filter `embedded`, run with the", + "real committed 40-shard manifest (.github/scripts/embedded-cmd-test-shards.txt)", + "so every one of the 242 TestEmbedded* functions that existed at capture time", + "ran exactly once (local sandboxed execution only; no remote executor or", + "cache flags were configured for this measurement run).", + "", + "Each value is that top-level test's own from Bazel's", + "generated test.xml, NOT the enclosing . For this Go test", + "binary / Bazel version, was found empirically to be a", + "wall-clock-since-binary-start timestamp (it does not isolate a concurrently", + "-run top-level test's own duration under -test.parallel=4), whereas", + " matched the test binary's own `--- PASS: TestX (Ys)` line", + "exactly. Do not switch this capture to without", + "re-verifying that relationship for the toolchain in use at the time.", + "", + "F1 review correction (S1): an earlier pass of this file explained four", + "outliers (TestEmbeddedUpdateConcurrent, TestEmbeddedCloseConcurrent,", + "TestEmbeddedInitA, TestEmbeddedEpicCloseEligibleReason) as a", + "\"queueing-delay\" artifact of Go's t.Parallel() clock starting before a", + "test's parallel slot is granted, and shipped corrected-down weights on", + "that theory. That explanation was WRONG and has been retracted: Go's", + "testing package (testing.go's (*T).Parallel) adds the elapsed time to", + "t.duration BEFORE blocking on the parallel-slot barrier, so queued-for-a-", + "slot wait time is never counted in a test's own reported duration in the", + "first place. The original large numbers were genuine CPU-bound running", + "time under real resource contention (dozens of Bazel shards running", + "concurrently on one host, each internally up to -test.parallel=4) -- this", + "capture re-measures every test under exactly that condition (all 54", + "cmd+storage+conformance shards running together, not solo) instead of", + "mixing solo-measured corrections into a contended-capture baseline.", + "", + "Two of those four outliers (TestEmbeddedUpdateConcurrent,", + "TestEmbeddedCloseConcurrent) are synthetic many-worker concurrency stress", + "tests whose cost is dominated by serialized `bd` subprocess invocations", + "behind the embedded engine's single-writer lock, not by -test.parallel", + "contention; their recorded values below are post-fix estimates (measured", + "value x 0.4) after cutting issuesPerWorker from 5 to 2 in both tests (see", + "their doc comments in update_embedded_test.go / close_embedded_test.go) --", + "pending a fresh capture to confirm. TestEmbeddedEpicCloseEligibleReason", + "was split into TestEmbeddedEpicCloseEligibleReasonFlag (3 subtests) and", + "TestEmbeddedEpicCloseEligibleReasonOutput (2 subtests); their recorded", + "values are the measured combined total apportioned 3:2 by subtest count,", + "also pending a fresh capture. TestEmbeddedInitA needed no further action:", + "at ~196s under this consistent methodology it is close to but under the", + "~200s single-test isolation threshold.", + "", + "Every test measuring at or above ~200s under this consistent load is", + "pinned alone on its own shard in the generated manifest below (see that", + "file's header); LPT bin-packing does this automatically as long as the", + "weight is accurate, since no combination of lighter tests sums close to", + "a ~200s+ single item without the packer preferring to isolate it.", + "", + "`seconds_per_init_fallback` is the weighted average seconds-per-init over", + "every measured test in this file (sum of all durations / sum of all inits,", + "where inits is discover_inits_cost()'s t.Run( count proxy), used only for a", + "brand-new test with no entry yet (see", + "scripts/ci/gen_embedded_cmd_shard_manifest.py's discover_inits_cost()).", + "", + "As with the proxied tier's duration file, these are a relative weight for", + "LPT bin-packing, not an absolute SLA: within-shard -test.parallel=4", + "overlap, farm noisy-neighbor effects, and future code changes all mean a", + "shard's total packed cost is a planning estimate, not a guarantee." + ], + "seconds_per_init_fallback": 19.4828, + "tests": { + "TestEmbeddedAdminBlocked": 20.33, + "TestEmbeddedAdminCompactReadOnlyAllowed": 21.99, + "TestEmbeddedAdminCompactTier2Honesty": 25.2, + "TestEmbeddedAutoImportFallback_ConflictSkip_DoesNotClobber": 28.23, + "TestEmbeddedAutoImportFallback_ConflictSkip_EmptyDBImportsAll": 23.63, + "TestEmbeddedAutoImportJSONLConcurrentWriter": 34.41, + "TestEmbeddedAutoImportJSONLNoExplicitCommit": 64.27, + "TestEmbeddedAutoImportJSONLSkipsNonEmpty": 27.53, + "TestEmbeddedBackup": 162.68, + "TestEmbeddedBackupConcurrent": 45.4, + "TestEmbeddedBlocked": 35.34, + "TestEmbeddedBlockedConcurrent": 51.71, + "TestEmbeddedBlockedLabelFilters": 66.12, + "TestEmbeddedBootstrap": 133.62, + "TestEmbeddedBootstrapConcurrent": 23.4, + "TestEmbeddedBranch": 128.64, + "TestEmbeddedBranchConcurrent": 36.84, + "TestEmbeddedChangeDirOverridesInheritedBeadsDir": 62.33, + "TestEmbeddedChildren": 86.18, + "TestEmbeddedChildrenConcurrent": 104.16, + "TestEmbeddedClaimPools": 78.4, + "TestEmbeddedCloseAlreadyClosed": 179.15, + "TestEmbeddedCloseBasic": 158.81, + "TestEmbeddedCloseConcurrent": 129.7, + "TestEmbeddedCloseGuardsAndEpics": 105.86, + "TestEmbeddedComments": 30.42, + "TestEmbeddedCommentsCLI": 89.97, + "TestEmbeddedCommentsCLIConcurrent": 38.85, + "TestEmbeddedCompact": 166.61, + "TestEmbeddedCompactConcurrent": 28.81, + "TestEmbeddedCompactMisdatedChild": 33.71, + "TestEmbeddedConfig": 83.45, + "TestEmbeddedConfigConcurrent": 96.47, + "TestEmbeddedConfigNoWorkspaceStoreFreeCommands": 6.09, + "TestEmbeddedConfigValidateJSONNoWorkspaceWritesStdout": 0.35, + "TestEmbeddedContext": 68.26, + "TestEmbeddedContextConcurrent": 22.8, + "TestEmbeddedContributorCreate": 166.08, + "TestEmbeddedCount": 127.95, + "TestEmbeddedCountConcurrent": 122.44, + "TestEmbeddedCountIncludeEphemeral": 86.31, + "TestEmbeddedCountIncludeInfra": 80.89, + "TestEmbeddedCreateCommitPending": 41.89, + "TestEmbeddedCreateConcurrent": 39.83, + "TestEmbeddedCreateCrossRepo": 44.64, + "TestEmbeddedCreateCrossRepoDryRunWithParent": 62.24, + "TestEmbeddedCreateCrossRepoUninit": 45.69, + "TestEmbeddedCreateCrossRepoWithParent": 80.27, + "TestEmbeddedCreateDryRunCrossRepoDoesNotMigrateTarget": 48.57, + "TestEmbeddedCreateDryRunDoesNotMigrate": 28.63, + "TestEmbeddedCreateDryRunRepoDoesNotInitializeTarget": 24.56, + "TestEmbeddedCreateFormCommitsLabelOnlyCreate": 31.46, + "TestEmbeddedCreateGraphAndBulk": 114.91, + "TestEmbeddedCreateJSONEchoesSubSecondTimestamps": 32.3, + "TestEmbeddedCreateRepoFromNonBeadsCwd": 23.45, + "TestEmbeddedCreateRepoProxiedTargetError": 45.4, + "TestEmbeddedCreateRepoRelativeExistingWorkspaceUnaffected": 45.93, + "TestEmbeddedCreateRepoRelativeUninitRefused": 24.13, + "TestEmbeddedCreateScalarAndRelationships": 191.35, + "TestEmbeddedCreateStorageClassBasic": 111.77, + "TestEmbeddedCreateStorageClassBatchAndMisc": 163.68, + "TestEmbeddedCreateStorageClassConflicts": 93.79, + "TestEmbeddedCreateWithGitRemote": 25.17, + "TestEmbeddedDefer": 92.03, + "TestEmbeddedDeferAutoWake": 79.42, + "TestEmbeddedDeferConcurrent": 121.28, + "TestEmbeddedDelete": 122.43, + "TestEmbeddedDeleteJSONDependencyErrorContract": 58.17, + "TestEmbeddedDepA": 175.73, + "TestEmbeddedDepAddWispSource": 58.14, + "TestEmbeddedDepAndLinkBatchAutoCommitDoesNotAdvanceHead": 132.31, + "TestEmbeddedDepB": 176.75, + "TestEmbeddedDepBulkNoCycleCheckKeepsWholeGraphGate": 38.51, + "TestEmbeddedDepConcurrent": 150.67, + "TestEmbeddedDepListSingleIDWarnsOnDroppedExternalEdge": 66.98, + "TestEmbeddedDepNoCycleCheck": 88.95, + "TestEmbeddedDiff": 56.44, + "TestEmbeddedDiffConcurrent": 68.69, + "TestEmbeddedDirectCommitPathsBatchAutoCommitDoesNotAdvanceHead": 174.02, + "TestEmbeddedDoltBlockedAndCommit": 59.12, + "TestEmbeddedDoltConcurrent": 68.7, + "TestEmbeddedDoltRemoteAndPush": 136.04, + "TestEmbeddedDuplicate": 72.32, + "TestEmbeddedDuplicateConcurrent": 69.38, + "TestEmbeddedDuplicates": 173.68, + "TestEmbeddedDuplicatesConcurrent": 59.24, + "TestEmbeddedEpicCloseEligible": 149.47, + "TestEmbeddedEpicCloseEligibleReasonFlag": 169.79, + "TestEmbeddedEpicCloseEligibleReasonOutput": 113.2, + "TestEmbeddedEpicConcurrent": 142.99, + "TestEmbeddedEpicStatus": 95.11, + "TestEmbeddedExplicitHierarchicalParentEdges": 65.84, + "TestEmbeddedExportConcurrent": 44.93, + "TestEmbeddedExportFlags": 130.38, + "TestEmbeddedExportFormats": 115.36, + "TestEmbeddedExternalCapabilityReadyAfterShip": 63.55, + "TestEmbeddedExternalMutationPolicy": 116.06, + "TestEmbeddedFederationConcurrent": 31.34, + "TestEmbeddedFederationPeers": 146.53, + "TestEmbeddedFederationStatus": 133.93, + "TestEmbeddedFindDuplicates": 124.62, + "TestEmbeddedFindDuplicatesConcurrent": 70.27, + "TestEmbeddedFlatten": 161.6, + "TestEmbeddedFlattenConcurrent": 32.13, + "TestEmbeddedGC": 185.65, + "TestEmbeddedGCConcurrent": 40.17, + "TestEmbeddedGate": 181.85, + "TestEmbeddedGateConcurrent": 66.94, + "TestEmbeddedGateCreate": 123.16, + "TestEmbeddedGateListScoped": 38.87, + "TestEmbeddedGetDependencies": 59.85, + "TestEmbeddedGraph": 81.16, + "TestEmbeddedGraphConcurrent": 88.99, + "TestEmbeddedHistory": 109.52, + "TestEmbeddedHistoryConcurrent": 90.07, + "TestEmbeddedHooks": 27.53, + "TestEmbeddedHooksConcurrent": 23.45, + "TestEmbeddedHuman": 97.66, + "TestEmbeddedHumanConcurrent": 30.64, + "TestEmbeddedImportClosedChildWhoseParentWasDeleted": 21.26, + "TestEmbeddedImportConcurrent": 31.4, + "TestEmbeddedImportFromSources": 124.38, + "TestEmbeddedImportPromotedWispRoundtrip": 137.68, + "TestEmbeddedImportUpsertAndPrefixSync": 111.37, + "TestEmbeddedInfo": 41.38, + "TestEmbeddedInfoConcurrent": 39.05, + "TestEmbeddedInfoSuppressGitHooks": 33.35, + "TestEmbeddedInitA": 196.25, + "TestEmbeddedInitArtifactRouting": 22.67, + "TestEmbeddedInitB": 145.22, + "TestEmbeddedInitC": 23.58, + "TestEmbeddedInitConcurrent": 47.95, + "TestEmbeddedInitExplicitBeadsDirIsAuthoritative": 53.4, + "TestEmbeddedInitGitBootstrapRouting": 158.28, + "TestEmbeddedInitHooksRouting": 22.24, + "TestEmbeddedInitRoleRouting": 125.08, + "TestEmbeddedInitSelectedExcludeRouting": 153.06, + "TestEmbeddedKV": 48.98, + "TestEmbeddedKVConcurrent": 108.72, + "TestEmbeddedLabelAddRemove": 138.68, + "TestEmbeddedLabelConcurrent": 84.16, + "TestEmbeddedLabelEditReports": 138.9, + "TestEmbeddedLabelPropagateRename": 120.78, + "TestEmbeddedLint": 82.71, + "TestEmbeddedLintConcurrent": 75.72, + "TestEmbeddedListComments": 51.81, + "TestEmbeddedListConcurrent": 33.41, + "TestEmbeddedListFiltering": 239.7, + "TestEmbeddedListSearchAndOutput": 147.31, + "TestEmbeddedMaxRowsDepTreeAndDuplicates": 97.69, + "TestEmbeddedMaxRowsDoctorAndConfig": 90.88, + "TestEmbeddedMaxRowsGraph": 159.26, + "TestEmbeddedMaxRowsListBasic": 107.6, + "TestEmbeddedMaxRowsListLimitInteraction": 192.0, + "TestEmbeddedMaxRowsReady": 168.09, + "TestEmbeddedMaxRowsReadyMerge": 78.54, + "TestEmbeddedMemory": 130.17, + "TestEmbeddedMemoryConcurrent": 95.43, + "TestEmbeddedMigrate": 155.64, + "TestEmbeddedMigrateConcurrent": 32.07, + "TestEmbeddedOnboard": 25.75, + "TestEmbeddedOnboardConcurrent": 22.91, + "TestEmbeddedOpen_EmptyDatabaseRejected": 0.0, + "TestEmbeddedOrphans": 40.45, + "TestEmbeddedOrphansConcurrent": 65.99, + "TestEmbeddedPreviewDoesNotConsumeVersionMarker": 29.99, + "TestEmbeddedPrime": 32.69, + "TestEmbeddedPrimeChangeDirDefaultTemplateMatchesTarget": 4.13, + "TestEmbeddedPrimeChangeDirPrimesTarget": 6.83, + "TestEmbeddedPrimeChangeDirRedirectNoticeFollowsTarget": 5.69, + "TestEmbeddedPrimeConcurrent": 55.99, + "TestEmbeddedPromote": 24.64, + "TestEmbeddedPromoteCLI": 64.87, + "TestEmbeddedPromoteCLIConcurrent": 67.99, + "TestEmbeddedPruneDryRunAndFilters": 155.68, + "TestEmbeddedPruneSafetyAndScope": 163.85, + "TestEmbeddedPurgeBasics": 89.31, + "TestEmbeddedPurgeConcurrent": 38.42, + "TestEmbeddedPurgeFiltered": 131.05, + "TestEmbeddedPurgeWispsPlaneRetention": 119.54, + "TestEmbeddedQuery": 69.55, + "TestEmbeddedQueryConcurrent": 79.23, + "TestEmbeddedQuick": 108.95, + "TestEmbeddedQuickConcurrent": 52.9, + "TestEmbeddedQuickstart": 21.7, + "TestEmbeddedQuickstartConcurrent": 24.95, + "TestEmbeddedQuietFlagSuppressesSuccessOutput": 37.91, + "TestEmbeddedReady": 148.56, + "TestEmbeddedReadyClaimConcurrent": 37.22, + "TestEmbeddedReadyConcurrent": 39.82, + "TestEmbeddedRenamePrefixBasic": 111.8, + "TestEmbeddedRenamePrefixConcurrent": 30.67, + "TestEmbeddedRenamePrefixEdgeCases": 81.14, + "TestEmbeddedReopen": 105.91, + "TestEmbeddedReopenConcurrent": 112.14, + "TestEmbeddedRestore": 137.93, + "TestEmbeddedRestoreConcurrent": 0.0, + "TestEmbeddedRoutedMutatingSiblingAssignTagDep": 156.88, + "TestEmbeddedRoutedMutatingSiblingDeleteClose": 101.7, + "TestEmbeddedRoutedSiblingWritesCommitTargetHead": 191.64, + "TestEmbeddedSearch": 118.78, + "TestEmbeddedSearchConcurrent": 98.05, + "TestEmbeddedSetup": 52.64, + "TestEmbeddedSetupConcurrent": 22.64, + "TestEmbeddedShip": 71.62, + "TestEmbeddedShipConcurrent": 101.01, + "TestEmbeddedShowBasicsAndJSON": 118.13, + "TestEmbeddedShowConcurrent": 150.5, + "TestEmbeddedShowDetailAndCurrent": 135.15, + "TestEmbeddedShowUnresolvableDeps": 59.52, + "TestEmbeddedSquashWispMoleculeClearsRootEphemeral": 23.37, + "TestEmbeddedStale": 90.45, + "TestEmbeddedStaleConcurrent": 79.59, + "TestEmbeddedStaleLabelFilters": 84.1, + "TestEmbeddedState": 66.22, + "TestEmbeddedStateConcurrent": 45.78, + "TestEmbeddedStatus": 47.37, + "TestEmbeddedStatusConcurrent": 79.04, + "TestEmbeddedSupersede": 76.14, + "TestEmbeddedSupersedeConcurrent": 72.31, + "TestEmbeddedSwarm": 172.12, + "TestEmbeddedSwarmConcurrent": 71.02, + "TestEmbeddedTodo": 74.74, + "TestEmbeddedTodoConcurrent": 30.6, + "TestEmbeddedTypes": 46.4, + "TestEmbeddedTypesAgreeWithCreate": 109.4, + "TestEmbeddedTypesConcurrent": 29.96, + "TestEmbeddedUnclaim": 130.59, + "TestEmbeddedUndefer": 122.4, + "TestEmbeddedUndeferConcurrent": 121.9, + "TestEmbeddedUpdateBatchAutoCommitDoesNotAdvanceHead": 28.26, + "TestEmbeddedUpdateConcurrent": 180.79, + "TestEmbeddedUpdateFields": 142.3, + "TestEmbeddedUpdateLifecycle": 150.21, + "TestEmbeddedUpdateRoutedStoreCommitsTargetHead": 51.5, + "TestEmbeddedVCCommitVariantsAndMerge": 105.56, + "TestEmbeddedVCConcurrent": 56.84, + "TestEmbeddedVCStatusAndCommit": 126.19, + "TestEmbeddedVersion": 1.4, + "TestEmbeddedVersionConcurrent": 1.48, + "TestEmbeddedWhere": 55.16, + "TestEmbeddedWhereConcurrent": 27.1 + } +} diff --git a/scripts/ci/embedded_storage_test_durations.json b/scripts/ci/embedded_storage_test_durations.json new file mode 100644 index 000000000..92c4c4ede --- /dev/null +++ b/scripts/ci/embedded_storage_test_durations.json @@ -0,0 +1,469 @@ +{ + "_comment": [ + "Measured locally with `bazel --nohome_rc test --config=embedded", + "//internal/storage/embeddeddolt:embeddeddolt_embedded_test", + "--local_test_jobs=8` alongside the cmd-tier and conformance targets (all", + "four run in the same invocation, the same way bazel.yml's bazel-embedded", + "tier runs its shards concurrently on one host) -- race,", + "`-test.parallel=4`, tag filter `embedded`, run with the real committed", + "14-shard manifest (.github/scripts/embedded-storage-test-shards.txt) so", + "every one of the 387 top-level Test* functions (excluding TestConformance,", + "sharded separately, and TestMain, which is not dispatched through", + "-test.run and carries no measurable duration of its own) that existed at", + "capture time ran exactly once (local sandboxed execution only; no remote", + "executor or cache flags were configured for this measurement run).", + "", + "Each value is that top-level test's own from Bazel's", + "generated test.xml, NOT the enclosing , for the same", + "reason documented in scripts/ci/embedded_cmd_test_durations.json's header", + "(empirically, is a wall-clock-since-binary-start", + "timestamp that does not isolate a concurrently-run top-level test's own", + "duration under -test.parallel=4, while matches the test", + "binary's own `--- PASS: TestX (Ys)` line exactly). Do not switch this", + "capture to without re-verifying that relationship for the", + "toolchain in use at the time.", + "", + "One originally-measured top-level function,", + "TestLargeBatchApplyStatementCounts_Embedded (350.25s: 81.03s + 260.70s +", + "8.52s for its 356/712/classic-40 subtests), was subsequently split (same", + "F1 CI-speed pass, see this file's git history) into 3 smaller", + "top-level functions, one per shape:", + "TestLargeBatchApplyStatementCounts356_Embedded,", + "TestLargeBatchApplyStatementCounts712_Embedded and", + "TestLargeBatchApplyStatementCountsClassic40_Embedded. Unlike the cmd", + "tier's reconstructed split durations, these 3 have always been exact, not", + "summed estimates: each runs exactly the one subtest the original combined", + "function used to, and this file's values are now a direct, fresh", + "per-function measurement from the same concurrent-load sweep as every", + "other test in this file (not a reconstruction from an older, solo-measured", + "capture). TestLargeBatchApplyStatementCounts712_Embedded genuinely still", + "costs ~260-300s to actually run outside of -race (re-confirmed by a solo", + "non-race re-measurement in an isolated clone: 298.60s) -- it is a single", + "pinned statement-count assertion over one indivisible ApplyBatchInTx", + "transaction (no internal sub-cases), so it cannot be split or shown to be", + "a measurement artifact the way some of the cmd tier's flagged outliers", + "were (see that file's header for the retraction of that theory -- it never", + "applied here; this test's cost is real work, not scheduling).", + "", + "F1 follow-up (coordinator item 2): rather than leave this as the tier's", + "irreducible ~260.7s floor, it is now skipped under raceEnabled (the exact", + "precedent TestLargeBatchApplyWallClock_Embedded set for its 1000-item", + "shape -- the cost is race-instrumentation overhead on the Dolt engine's", + "own internal goroutine/lock machinery, not this test's own logic), with", + "nightly.yml's \"Embedded Dolt batch-apply suite (non-race)\" step's -run", + "regex extended to still run the full pinned assertion every night (and, as", + "of this capture, given an explicit -timeout and `if: ${{ !cancelled() }}`", + "so it still runs and reports even if an earlier nightly step fails). Its", + "weight below is therefore set to 1.0 (negligible: a skipped subtest's own", + " was observed at 0.00s in this capture too, confirming the", + "skip is still in effect; 1.0 is a small nonzero placeholder so LPT packing", + "still orders it after truly-free slots), reflecting what this test", + "actually costs the per-PR -race lane bazel-embedded packs for -- NOT its", + "real non-race cost, which is still ~260-300s and is what nightly.yml", + "exercises. Do not revert this weight to 260.7 without also reverting the", + "raceEnabled skip in large_batch_apply_measure_test.go -- the two must stay", + "in sync.", + "", + "`seconds_per_init_fallback` is the weighted average seconds-per-init over", + "every measured test in this file (sum of all durations / sum of all", + "inits, where inits is discover_inits_cost()'s t.Run( count proxy), used", + "only for a brand-new test with no entry yet (see", + "scripts/ci/gen_embedded_storage_shard_manifest.py's discover_inits_cost()).", + "", + "As with the proxied and cmd tiers' duration files, these are a relative", + "weight for LPT bin-packing, not an absolute SLA: within-shard", + "-test.parallel=4 overlap, farm noisy-neighbor effects, and future code", + "changes all mean a shard's total packed cost is a planning estimate, not", + "a guarantee." + ], + "seconds_per_init_fallback": 3.8877, + "tests": { + "TestActiveDatabaseSizeFailsForMissingEmbeddedDatabase": 0.0, + "TestActiveDatabaseSizeScopesToActiveEmbeddedDatabase": 0.0, + "TestAddDependency": 33.47, + "TestAddLabel": 1.12, + "TestBatchApplyContract": 151.73, + "TestBatchCreatorContract": 11.81, + "TestBlockingAnnotatorContract": 17.98, + "TestBootstrapperContract": 3.79, + "TestBuildDSN_NoDatabase": 0.0, + "TestBuildDSN_SpacesInPath": 0.0, + "TestBuildDSN_WithDatabase": 0.0, + "TestCleanGitRemoteCacheGarbage": 0.0, + "TestCleanGitRemoteCacheGarbage_NoCacheDir": 0.0, + "TestCleanGitRemoteCacheGarbage_Throttled": 0.0, + "TestCommenterContract": 25.32, + "TestCommitAllOutOfBandChangeAndHonestNoOp": 1.1, + "TestConcurrencyMultiProcess": 6.48, + "TestConcurrentOpenReturnsCached": 18.03, + "TestCounterContract": 10.53, + "TestCountsIncludeWispDependencies": 1.73, + "TestCountsWithMissingWispPlaneTables": 3.29, + "TestCreateIssue": 35.89, + "TestCreateIssues": 19.14, + "TestCreateIssuesRejectStaleUpserts": 9.47, + "TestCreateRejectsCrossTableIDCollision": 2.63, + "TestCrossRecordInvariantContract": 0.0, + "TestCycleDetectorContract": 14.23, + "TestDecryptPassword_KeyMismatchNamesTheLocalKey": 18.0, + "TestDecryptPassword_ShortCiphertextClassifiesAsKeyMismatch": 17.16, + "TestDeleterContract": 61.52, + "TestDepRekeyMarkerStillInIgnoredSeries": 0.0, + "TestDescendantWalkReachesEveryTargetKindAndStopsOnCycles": 4.78, + "TestDoltCommitResponseLossIsIndeterminate": 0.0, + "TestDualWriteContract": 2.41, + "TestDualWriteContractFlagOff": 0.74, + "TestDualWriteFixtureKitIsWired": 1.06, + "TestEdgeReaderContract": 10.02, + "TestEmbeddedBatchCloserAllIdempotentBatchLandsNothing": 1.9, + "TestEmbeddedBatchCloserAllRefusedBatchRecordsNoHistory": 19.12, + "TestEmbeddedBatchCloserBackendFailureReturnsNoOutcomes": 20.46, + "TestEmbeddedBatchCloserClaimFilterValueFailureIsARequestValidationFailure": 19.63, + "TestEmbeddedBatchCloserClaimNextHydratesWhenSomethingClosed": 2.3, + "TestEmbeddedBatchCloserClaimNextIsNilWhenNothingClosed": 0.79, + "TestEmbeddedBatchCloserClaimNextIsNilWhenTheFrontIsEmpty": 1.36, + "TestEmbeddedBatchCloserClaimNextSeesAnUnblockingFromItsOwnBatch": 22.31, + "TestEmbeddedBatchCloserDoesNotMutateTheCallerRequest": 19.41, + "TestEmbeddedBatchCloserDuplicateItemRecloseAtItsOwnIndex": 1.64, + "TestEmbeddedBatchCloserDurableHistoryNeverNamesAWisp": 20.52, + "TestEmbeddedBatchCloserForceBypassesOnlyClosePolicy": 4.15, + "TestEmbeddedBatchCloserIdempotentRecloseIsAPerItemSuccess": 3.81, + "TestEmbeddedBatchCloserOutcomeSnapshotIsTheDocumentedShape": 21.59, + "TestEmbeddedBatchCloserOutcomesMirrorItemsIndexForIndex": 22.63, + "TestEmbeddedBatchCloserPerItemRefusalIsAResultAndSurvivorsCommit": 22.49, + "TestEmbeddedBatchCloserRecordsOneHistoryEntryForWhatLanded": 1.99, + "TestEmbeddedBatchCloserRequestValidationReturnsZeroResultAndChangesNothing": 18.68, + "TestEmbeddedBatchCloserSettlesTheDependersOfWhatItClosed": 9.26, + "TestEmbeddedBatchCloserWispItemClosesAndEarnsTheClaim": 1.89, + "TestEmbeddedBatchCreateFirstVersionCarriesCreationTimeEdges": 1.79, + "TestEmbeddedBlockedRecheckFailureKeepsTheWrite": 20.58, + "TestEmbeddedBlockedRecheckRecordsUnblockingWrites": 21.27, + "TestEmbeddedBlockedRecheckSettlesWhatTheWriteRecorded": 22.27, + "TestEmbeddedCheckedWrappersRouteTheirPreconditions": 22.28, + "TestEmbeddedClaimIssueWrapperClaimsTheWispPlane": 1.35, + "TestEmbeddedClaimerAcceptsAConfiguredActiveStatusAndRefusesAConfiguredWipOne": 1.81, + "TestEmbeddedClaimerClaimsAnUnassignedOpenIssueAndAnswersTheBareRow": 1.13, + "TestEmbeddedClaimerGrantsALiveLeaseOnTheRowItWonAndLeavesARefusedOneAlone": 2.21, + "TestEmbeddedClaimerReclaimsAcrossSpellingWithoutWriting": 1.6, + "TestEmbeddedClaimerReclaimsItsOwnInProgressIssueWithoutWriting": 1.43, + "TestEmbeddedClaimerRefusalMessagesCarryTheirFragments": 1.7, + "TestEmbeddedClaimerRefusesABuiltInIneligibleStatusWithTheStateThatRefusedIt": 1.43, + "TestEmbeddedClaimerRefusesAWispIDAsNotFound": 1.02, + "TestEmbeddedClaimerRefusesIncompleteRequestsAndAnAbsentIDWithoutTouchingState": 1.07, + "TestEmbeddedClaimerStampsStartedAtOnceAcrossTheTwoWritesItChoosesBetween": 1.81, + "TestEmbeddedClaimerTakesAPoolAssignedIssueButRefusesAForeignHolder": 2.5, + "TestEmbeddedClaimerTakesAnOpenIssueTheActorAlreadyHolds": 18.89, + "TestEmbeddedCommitToleratesNothingToCommit": 1.12, + "TestEmbeddedConfigCommitStagesConfig": 0.55, + "TestEmbeddedDepRekeyAbortLeavesMarkerUnrecorded": 14.53, + "TestEmbeddedDepRekeyConvergesRenameChain": 14.34, + "TestEmbeddedDepRekeyMergesFreshDuplicateEdges": 29.13, + "TestEmbeddedDepRekeyRefusesToWriteIntoDirtyDependencies": 15.17, + "TestEmbeddedDepRekeyRepairsHalfRekeyedDuplicateEdge": 14.62, + "TestEmbeddedDependencyEditorAcceptsADiamond": 3.43, + "TestEmbeddedDependencyEditorAcceptsAForeignRepoTarget": 1.1, + "TestEmbeddedDependencyEditorAcceptsATypeOutsideTheConstants": 1.53, + "TestEmbeddedDependencyEditorAcceptsAnExternalTarget": 0.93, + "TestEmbeddedDependencyEditorAcceptsBlockingAcrossIssueTypes": 3.07, + "TestEmbeddedDependencyEditorAddMarksItsSourceInTheSameVerb": 3.21, + "TestEmbeddedDependencyEditorAddedEchoesTheRequestOrder": 2.1, + "TestEmbeddedDependencyEditorAppliesParentChildBeforeBlockingEdges": 2.3, + "TestEmbeddedDependencyEditorAttributesItsEventsToTheActor": 2.62, + "TestEmbeddedDependencyEditorClosedChildAddSatisfiesAnAnyChildrenGate": 5.12, + "TestEmbeddedDependencyEditorGateScopeFollowsTheEdgeType": 2.82, + "TestEmbeddedDependencyEditorMaintainsBlockedStateAcrossPlanes": 6.74, + "TestEmbeddedDependencyEditorMixedBatchRefusalRollsBackBothPlanes": 1.92, + "TestEmbeddedDependencyEditorMixedBatchWritesBothPlanes": 1.75, + "TestEmbeddedDependencyEditorRecordsNoHistoryForAnAllEphemeralRequest": 1.62, + "TestEmbeddedDependencyEditorRecordsOneHistoryEntryForAMixedPlaneRequest": 1.83, + "TestEmbeddedDependencyEditorRecordsOneHistoryEntryPerLandedRequest": 3.76, + "TestEmbeddedDependencyEditorRefusalWritesNothing": 2.44, + "TestEmbeddedDependencyEditorRefusesACycleThroughAParentChildHop": 9.58, + "TestEmbeddedDependencyEditorRefusesAGhostSource": 1.87, + "TestEmbeddedDependencyEditorRefusesAMissingLocalTarget": 1.41, + "TestEmbeddedDependencyEditorRefusesASamePlaneEdgeClosingACrossPlaneCycle": 3.51, + "TestEmbeddedDependencyEditorRefusesBlockingEdgeAcrossAWispHierarchy": 2.37, + "TestEmbeddedDependencyEditorRefusesBlockingEdgeAcrossItsOwnHierarchy": 2.44, + "TestEmbeddedDependencyEditorRefusesCrossPlaneCycle": 1.64, + "TestEmbeddedDependencyEditorRefusesSelfDependencyWithTheProbeSkipped": 1.18, + "TestEmbeddedDependencyEditorRelatesToAddLeavesItsSourceUnblocked": 3.62, + "TestEmbeddedDependencyEditorRemoveIsIdempotent": 3.04, + "TestEmbeddedDependencyEditorRemoveUnmarksItsSourceAndDescendants": 6.72, + "TestEmbeddedDependencyEditorRemovesOnlyTheNamedEdge": 1.88, + "TestEmbeddedDependencyEditorRepeatsWithinOneRequestCollapse": 2.47, + "TestEmbeddedDependencyEditorRetypeRefusalLeavesTheOriginalEdge": 1.63, + "TestEmbeddedDependencyEditorRoutesWispSourcedEdgeToTheWispPlane": 1.47, + "TestEmbeddedDependencyEditorRoutesWispSourcedRemovalToTheWispPlane": 2.85, + "TestEmbeddedDependencyEditorSameTypeReAddIsIdempotent": 2.06, + "TestEmbeddedDependencyEditorSameTypeReAddWithChangedMetadataMintsOneVersion": 2.15, + "TestEmbeddedDependencyEditorSameTypeReAddWithIdenticalMetadataIsANoOp": 3.79, + "TestEmbeddedDependencyEditorSkipPerEdgeCycleCheckDropsOnlyTheProbe": 1.77, + "TestEmbeddedDependencyEditorSnapshotsTheRequest": 1.19, + "TestEmbeddedDependencyEditorValidationRefusalsWriteNothing": 1.01, + "TestEmbeddedDependencyEditorVersionsOnlyARealAddOrRemove": 3.61, + "TestEmbeddedDependencyEditorWritesTheTargetIntoItsTypedColumn": 3.14, + "TestEmbeddedDirtyTablesGate_BlocksReopenThenWorkingSetReconcileRecovers": 25.82, + "TestEmbeddedDoltStoreIssueLifecycleBuildsOperations": 0.0, + "TestEmbeddedDoltStoreIssueLifecycleRejectsNilStore": 0.0, + "TestEmbeddedEveryAcceptedLifecycleMutationMintsOneVersion": 5.86, + "TestEmbeddedGetMergeBlockersOnAQuietStore": 0.52, + "TestEmbeddedGuardedUpdateMintsOnceAfterItsPatches": 2.24, + "TestEmbeddedIgnoredMigration0019CreatesEventsForFreshClones": 12.11, + "TestEmbeddedIgnoredMigration0023NoopsOnHealthyWorkspace": 15.91, + "TestEmbeddedIgnoredMigration0023NoopsWithoutItsTables": 13.02, + "TestEmbeddedIgnoredMigration0023RepairsForkLineageJournalShape": 16.07, + "TestEmbeddedIgnoredSeriesConvergesWithFreshInitShape": 31.85, + "TestEmbeddedIssueOperationsClaimLeavesBlockedStateAlone": 4.09, + "TestEmbeddedIssueOperationsCloseAndReopenAWisp": 1.72, + "TestEmbeddedIssueOperationsCreateAggregatesRelations": 2.22, + "TestEmbeddedIssueOperationsCreateClosedDerivesTheClosedStamp": 2.84, + "TestEmbeddedIssueOperationsCreateEmptyIDPreservesParentAndReverseDependencies": 3.31, + "TestEmbeddedIssueOperationsCreateParentChildRecomputesWaitsForClosure": 3.64, + "TestEmbeddedIssueOperationsCreateReverseNonBlockingStagesConcreteTables": 2.25, + "TestEmbeddedIssueOperationsCreateRoutesInfraTypesToWisps": 4.22, + "TestEmbeddedIssueOperationsCreateUnderAParentMintsTheNextChildID": 5.14, + "TestEmbeddedIssueOperationsCreateWithDependenciesSettlesInTheCreatingTransaction": 4.32, + "TestEmbeddedIssueOperationsPersistenceRollbackWispAndDetachedResults": 5.07, + "TestEmbeddedIssueOperationsRequestValuesAreNotMutated": 2.9, + "TestEmbeddedIssueOperationsUpdateAllScalarAndPointerFieldsReportChanged": 15.17, + "TestEmbeddedIssueOperationsUpdateClaimCASAndTransferMatrix": 2.6, + "TestEmbeddedIssueOperationsUpdateClaimConflictCarriesTheLosingState": 4.89, + "TestEmbeddedIssueOperationsUpdateClaimHonorsConfiguredActiveStatuses": 2.93, + "TestEmbeddedIssueOperationsUpdateClaimPatchAppliesAfterClaim": 1.03, + "TestEmbeddedIssueOperationsUpdateClosedFieldsMatchClose": 17.14, + "TestEmbeddedIssueOperationsUpdateFoldsMetadataIntoOneEvent": 2.85, + "TestEmbeddedIssueOperationsUpdateInvalidIssueTypePreservesValidationSentinel": 0.73, + "TestEmbeddedIssueOperationsUpdateIssuePlaneOnlyRefusesWisps": 2.44, + "TestEmbeddedIssueOperationsUpdateLabelPatchOrdering": 3.02, + "TestEmbeddedIssueOperationsUpdateLabelPatchValueRules": 2.64, + "TestEmbeddedIssueOperationsUpdateMetadataReplaceClearsAndValidates": 2.62, + "TestEmbeddedIssueOperationsUpdateNotesConflictPreservesValidationSentinel": 0.76, + "TestEmbeddedIssueOperationsUpdateParentStagesBlockedStateAndRollsBackFailure": 5.78, + "TestEmbeddedIssueOperationsUpdateRawMetadataTakesTheFunnelsValueShapes": 1.5, + "TestEmbeddedIssueOperationsUpdateRefusesATypeOutsideTheWorkspaceVocabulary": 2.34, + "TestEmbeddedIssueOperationsUpdateStampsStartedAtOnceOnTheFirstInProgress": 1.77, + "TestEmbeddedIssueOperationsUpdateStatusCrossingSettlesAConditionalBlocksDepender": 7.63, + "TestEmbeddedIssueOperationsUpdateStatusCrossingSettlesDependers": 6.92, + "TestEmbeddedIssueOperationsUpdateWritesEveryScalarPatchField": 1.74, + "TestEmbeddedLenientOpenWarning_DataBehind": 73.08, + "TestEmbeddedLenientOpenWarning_OtherRefusalsKeepBluntGuidance": 17.96, + "TestEmbeddedMergeAndSettleDirtyWorkingSetSurvivesRefusedMerge": 0.76, + "TestEmbeddedMergeAndSettleFKCascadeRepair": 2.52, + "TestEmbeddedMergeAndSettleMemoryConfigConflict": 0.68, + "TestEmbeddedMergeAndSettleMetadataConflict": 0.74, + "TestEmbeddedMergeAndSettleRefusesUnknownTable": 0.91, + "TestEmbeddedMergeAndSettleReportsOperatorConflicts": 1.14, + "TestEmbeddedMergeAndSettleSkipsNonMemoryConfigConflict": 0.82, + "TestEmbeddedMigrateConvergesUnderConcurrentInterruptedRetries_4566": 31.96, + "TestEmbeddedMigrateRepairedDependenciesIDColumnCommitsAtomicallyWithVersion53_4690": 15.34, + "TestEmbeddedMigrateSelfHealsAfterMidPassFault_4566": 15.45, + "TestEmbeddedMigrateStepCommitLeavesUntouchedUserDirt_4566": 15.77, + "TestEmbeddedMigration0047DelegatesToSplitTargetRepairAndConvergesThroughIgnoredChain": 15.59, + "TestEmbeddedMigration0057ConvertsTextEventsColumnsToLongtext": 20.01, + "TestEmbeddedMigration0058AddsOnlyMissingConstraintInMixedPopulation": 9.28, + "TestEmbeddedMigration0058CleansOrphanedAndInvalidWispDependenciesRowsBeforeAddingConstraints": 10.02, + "TestEmbeddedMigration0058HealsAlreadyAffectedDatabase": 10.23, + "TestEmbeddedMigration0062FlipsEventsToIgnoredPlane": 8.38, + "TestEmbeddedMoveIssuePersistenceEphemeralNoHistorySmoke": 17.42, + "TestEmbeddedOpenForReadOnlyCommand_LenientGate": 18.99, + "TestEmbeddedOpenForRemoteSync_DataBehind": 35.97, + "TestEmbeddedOpenForRemoteSync_OtherRefusalsStayFatal": 18.37, + "TestEmbeddedOpenReadOnly_FreshDatabaseNoWrites": 0.0, + "TestEmbeddedOpenReadOnly_SkipsGateAndMigrations": 18.77, + "TestEmbeddedPre0058RepairMarchesDriftedWispDependenciesToFinalShape": 57.06, + "TestEmbeddedReaderDoesNotMutateTheCallerRequest": 0.91, + "TestEmbeddedReaderGetBriefDepsProjectsTheDependencyRows": 1.84, + "TestEmbeddedReaderGetDetailShapeMatchesTheSeededIssue": 1.46, + "TestEmbeddedReaderGetMissIsNotFoundAndBackendFailureDoesNotDecay": 0.48, + "TestEmbeddedReaderGetOptionalRowListsAreOffByDefault": 2.13, + "TestEmbeddedReaderGetResolvesTheExactIDAcrossBothPlanes": 1.81, + "TestEmbeddedReaderListBriefDropsTheFreeFormTextAndNothingElse": 2.8, + "TestEmbeddedReaderListCountsAreBlocksOnlyWhereGetCountsEveryEdge": 5.14, + "TestEmbeddedReaderListDefaultExclusionsAndTheirOverrides": 7.58, + "TestEmbeddedReaderListEmptyPageIsWellFormed": 0.79, + "TestEmbeddedReaderListIncludeEphemeralMergesThePlanesIntoOneOrder": 6.4, + "TestEmbeddedReaderListKeysetPositionNarrowsWithoutReplacingTheOtherPredicates": 3.76, + "TestEmbeddedReaderListKeysetPositionResumesTheCreatedDescIDAscOrder": 1.56, + "TestEmbeddedReaderListKeysetWalkOverAnOversizedGroupLosesNothingAndRepeatsNothing": 4.24, + "TestEmbeddedReaderListLimitBoundaryUnderASortTheDatabaseCanExpress": 2.66, + "TestEmbeddedReaderListMaxRowsBoundaryIsLimitPlusOffset": 4.12, + "TestEmbeddedReaderListMaxRowsIsHonored": 2.77, + "TestEmbeddedReaderListNaturalNumericIDSortTrimsAfterTheFetch": 1.84, + "TestEmbeddedReaderListParentReachesEveryDescendantAndOnlyItsOwn": 2.03, + "TestEmbeddedReaderListPriorityKeysetWalkOverAnOversizedEqualKeyRunLosesNothingAndRepeatsNothing": 4.86, + "TestEmbeddedReaderListReadyFlagAnswersTheBlockerAwareSet": 2.64, + "TestEmbeddedReaderListReadyFlagCarriesTheAssigneeAndPriorityFilters": 2.71, + "TestEmbeddedReaderListReadyFlagRefusesAFilterItCannotCarry": 2.87, + "TestEmbeddedReaderListRejectsATypeOutsideTheWorkspaceVocabulary": 1.04, + "TestEmbeddedReaderListSkipCountsDropsTheCardinalitiesAndNothingElse": 2.97, + "TestEmbeddedReaderListStatusAcceptsACommaSeparatedORSet": 1.89, + "TestEmbeddedReaderListWispTypeNarrowsTheAdmittedPlaneRatherThanAdmittingIt": 2.31, + "TestEmbeddedReaderOffsetSkipsTheRowsBeforeThePage": 3.16, + "TestEmbeddedReaderReadyBriefDropsTheFreeFormTextAndNothingElse": 1.62, + "TestEmbeddedReaderReadyDefaultTypeExclusionsYieldToAnExplicitType": 1.92, + "TestEmbeddedReaderReadyDeferredAndEphemeralGates": 1.87, + "TestEmbeddedReaderReadyEphemeralPageKeepsBothPlanesCountsAtItsBoundary": 5.32, + "TestEmbeddedReaderReadyLimitBoundary": 1.97, + "TestEmbeddedReaderReadyPageIsThePrefixOfTheUnboundedAnswerCountsIncluded": 6.4, + "TestEmbeddedReaderReadyPageWiderThanTheHydrationBatchIsStillThatPrefix": 71.65, + "TestEmbeddedReaderReadyParentScopesToItsTransitiveDescendants": 3.93, + "TestEmbeddedReaderReadySetOwnsItsStatusPinnedAndTemplateDecisions": 3.2, + "TestEmbeddedReaderReadySortPoliciesOrderTheSameRows": 1.64, + "TestEmbeddedReadyClaimerAnswersTheQuestionReaderReadyLists": 1.82, + "TestEmbeddedReadyClaimerClaimsAnEphemeralRowTheFilterAdmits": 1.62, + "TestEmbeddedReadyClaimerClaimsTheFrontRowAndReturnsThePostClaimState": 2.4, + "TestEmbeddedReadyClaimerDoesNotMutateTheCallerRequest": 0.64, + "TestEmbeddedReadyClaimerEmptyFrontIsNormal": 1.2, + "TestEmbeddedReadyClaimerFencesTheClaimByEveryLabelSetAndTheParentItWasGiven": 5.45, + "TestEmbeddedReadyClaimerHydratesOnlyItsBlocksEdgesIntoTheCardinalities": 4.27, + "TestEmbeddedReadyClaimerLeasesADurableWinButNotAnEphemeralOne": 2.09, + "TestEmbeddedReadyClaimerLeavesEphemeralRowsOutOfTheDefaultReadySet": 1.15, + "TestEmbeddedReadyClaimerRecordsOneHistoryEntryForAWin": 1.45, + "TestEmbeddedReadyClaimerRejectsLimitOffsetBriefAndEmptyActor": 0.89, + "TestEmbeddedReadyClaimerSkipsIneligibleFrontRows": 1.83, + "TestEmbeddedRecomputeAllBlockedDirtyGraphIsTyped": 0.72, + "TestEmbeddedRecomputeAllBlockedWiring": 1.9, + "TestEmbeddedRelationsAnswersAWispTargetInTheOutDirection": 2.24, + "TestEmbeddedRelationsAnswersInThePinnedOrder": 2.28, + "TestEmbeddedRelationsDirectionSelectsTheInverseGraph": 1.71, + "TestEmbeddedRelationsFiltersByAnOpenTypeVocabulary": 2.03, + "TestEmbeddedRelationsLeavesAnExternalTargetOutOfTheAnswer": 1.81, + "TestEmbeddedRelationsLeavesTheCallersRequestAlone": 1.41, + "TestEmbeddedRelationsOrdersNeighborsFromBothPlanesTogether": 2.73, + "TestEmbeddedRelationsRefusesATypeFilterOverTheColumnLength": 0.59, + "TestEmbeddedRelationsRefusesAnUnusableTypeFilter": 0.54, + "TestEmbeddedRelationsRefusesTheZeroDirection": 0.59, + "TestEmbeddedRelationsResolvesAWispAnchor": 1.56, + "TestEmbeddedRelationsResolvesTheAnchorIDExactly": 1.45, + "TestEmbeddedRelationsSeparatesNoNeighborsFromNoSuchIssue": 0.72, + "TestEmbeddedRemoteMigrateGate_BlocksReopen": 18.03, + "TestEmbeddedRenameKeepsDependencyIDsDeterministic": 13.78, + "TestEmbeddedReopenCustomDoneFallsBackWhenNormalizedTableMissing": 1.36, + "TestEmbeddedReopenIssueCategorySemantics": 5.51, + "TestEmbeddedRoleFixtureKitCountHistoryMovesWithACommit": 0.81, + "TestEmbeddedRoleFixtureKitHooksAreUsable": 1.85, + "TestEmbeddedSQLCommitResponseLossIsIndeterminate": 0.0, + "TestEmbeddedSyncCommitsConfigBeforeMerge": 1.4, + "TestEmbeddedTempRebuildIntermediatesNeverReachHistory": 21.52, + "TestEmbeddedTransactionSameStatusDoesNotPublishUnrelatedDurableChanges": 1.63, + "TestEmbeddedTransactionStandaloneWispLifecyclePersistsWithoutDoltCommit": 3.09, + "TestEmbeddedTransactionStandaloneWispMutationDoesNotPublishUnrelatedDurableChanges": 2.64, + "TestEmbeddedTransactionWispLifecyclePersistsDurableDependent": 8.69, + "TestEpochContract": 0.0, + "TestEventsAutoPruneBoundsTheJournalToTheFloors": 2.74, + "TestEventsAutoPruneKeepsAnUnboundedJournalUnbounded": 1.51, + "TestEventsAutoPruneThrottleSkipsTheSecondPass": 3.34, + "TestEventsJournalAccessor": 2.42, + "TestEventsJournalPruneNeverSplitsASkewedPair": 2.51, + "TestEventsJournalPruneWhenEveryRowIsOlderThanTheCutoff": 2.02, + "TestEventsJournalReadRefusesAnInteriorHole": 1.83, + "TestEventsJournalRestartAndRetentionBoundary": 4.32, + "TestEventsJournal_ActorMatrixOnEmbedded": 5.04, + "TestEventsJournal_EmbeddedConcurrentGaplessNoDup": 4.15, + "TestEventsSinceEmbedded": 2.2, + "TestExpectedRevisionContract": 0.0, + "TestExternalCapabilityBlocksEmbeddedReadyAndAppearsInTree": 38.8, + "TestForeignSkipDetailIsBounded": 9.16, + "TestGetDependentRecordsEmbedded": 2.76, + "TestGetDependentRecordsForIssuesEmbedded": 4.19, + "TestGetIssue": 3.8, + "TestGetIssueCommentsPageEmbedded": 5.56, + "TestGetLabels": 1.66, + "TestGetNextChildID": 5.99, + "TestGetReadyWorkIncludeEphemeralAppliesWispFilters": 8.29, + "TestGetReadyWorkIncludeEphemeralExcludesNonReadyWisps": 3.58, + "TestGetReadyWorkMoleculeFilter": 2.41, + "TestGitRefPinsMatchDolt": 0.0, + "TestGraphCounterContract": 15.92, + "TestHasRepository": 0.0, + "TestHeartbeatMatchesActorAcrossSpellingEmbedded": 1.01, + "TestHeartbeatRejectsWispEmbedded": 0.69, + "TestHelperProcess": 0.0, + "TestHistoryPresenceContract": 3.26, + "TestHistory_NullTextColumns": 1.66, + "TestHookFiringStoreCreateIssuesFiresDependencyUpdatesFromEmbeddedStore": 3.06, + "TestIgnoredCursorHealConvergesFromEveryCrashState": 4.9, + "TestIgnoredCursorHealIsIdempotent": 2.0, + "TestIgnoredCursorHealNeverResurrectsRolledBackVersions": 1.95, + "TestIgnoredCursorHealRespectsOperatorOverride": 1.02, + "TestIgnoredCursorHealRespectsOverrideUnderBroaderPattern": 0.95, + "TestIgnoredCursorHealResumeRespectsOperatorOverride": 1.14, + "TestIsBlockedBatchEmbedded": 2.64, + "TestJournalContract": 9.95, + "TestLargeBatchApplyStatementCounts356_Embedded": 79.51, + "TestLargeBatchApplyStatementCounts712_Embedded": 1.0, + "TestLargeBatchApplyStatementCountsClassic40_Embedded": 8.73, + "TestLargeBatchApplyWallClock_Embedded": 80.7, + "TestLeaseLifecycleEmbedded": 3.76, + "TestLegacyTrackedIgnoredCursorHealsOnOpen": 1.39, + "TestLegacyTrackedIgnoredCursorUnwedgesPull": 7.0, + "TestLenientGateWarningBody_BluntRefusalKeepsSharedGuidance": 0.0, + "TestLenientGateWarningBody_ShapedDecisions": 0.0, + "TestLenientGateWarning_ComposedOutput": 0.0, + "TestLifecycleCloseReopenContract": 76.34, + "TestLifecycleCreateContract": 12.17, + "TestLifecycleUpdateContract": 39.87, + "TestListWisps": 1.63, + "TestMemoriesContract": 13.94, + "TestMergeAndSettleWithStrategy_EmptyStrategyMatchesMergeAndSettle": 1.32, + "TestMergeAndSettleWithStrategy_ResolvesDeclinedConflict": 1.22, + "TestMergeWithStrategy_OursKeepsLocal": 1.19, + "TestMergeWithStrategy_TheirsResolvesAndCommits": 1.19, + "TestMerge_NoStrategyStillErrorsWithEscapeHatchHint": 1.13, + "TestMetadataCASContract": 14.63, + "TestOpenAfterCloseCreatesNewStore": 17.64, + "TestOpenDifferentDirsReturnsDifferentStores": 36.11, + "TestOpenReturnsCachedStore": 19.64, + "TestPartitionWispIDs": 1.2, + "TestPreviewStoreRefusesLocalMetadataWrite": 17.18, + "TestPristineEmbeddedDoltFixtureRelocatesPrefixes": 2.42, + "TestPromoteClearsNoHistory": 3.01, + "TestPromoteStillWorksWithCollisionGuard": 1.36, + "TestPruneEventsBatchDeletesTheLowestSeqsOnly": 1.97, + "TestPushCommitsPendingChangesFirst": 89.08, + "TestQuerierContract": 15.49, + "TestReadEventsAutoPruneStateResolvesBothHalvesInOneQuery": 1.38, + "TestReadyCounterContract": 8.16, + "TestReadyWorkPageTotal": 30.66, + "TestReadyWorkPageTotalFilterMatrix": 23.68, + "TestReadyWorkStatementBudget": 22.78, + "TestReadyWorkWithMissingDeferredParentTables": 5.52, + "TestReclaimAnyReplicaOverride": 0.89, + "TestReclaimExpiredLeaseSurvivesRestartEmbedded": 20.55, + "TestReclaimRefusesForeignReplicaLease": 1.28, + "TestReleaserContract": 8.65, + "TestRemoteEntryPointsUseVersionControlOps": 0.0, + "TestRemoteGitDataRefOnEngine": 2.82, + "TestRemoteVerbPlainRemoteKeepsAmbientEnv": 17.91, + "TestRemoteVerbPlainRemoteWithoutAuthPresentsNone": 17.51, + "TestRemoteVerbsPresentStoredPeerCredentials": 20.79, + "TestRenameLabel": 3.42, + "TestRenameLabelMintsOneVersionPerTouchedBead": 4.17, + "TestReopenIssue": 1.7, + "TestRetentionContract": 0.0, + "TestRunInTransactionPostCommitStageFailureIsIndeterminate": 0.62, + "TestSchemaAfterInit": 17.05, + "TestSearchWithMissingWispPlaneTables": 3.33, + "TestStageAndCommitAfterSQLCommitResponseLossIsIndeterminateAndNotReplayed": 0.0, + "TestStatsReporterContract": 14.39, + "TestSweeperContract": 25.68, + "TestTransactionCleanupAfterSQLCommitIsIndeterminate": 0.0, + "TestTransactionCleanupBeforeSQLCommitRemainsDefinite": 0.0, + "TestTreeWalkerContract": 22.54, + "TestUnknownProvenanceLeaseStaysReclaimable": 1.83, + "TestUpdateIssueType": 0.83, + "TestUpdateMergeOps_DoltEngine": 0.92, + "TestUpdateMergeOps_DoltEngine_TwoIssues": 1.16, + "TestVersionReconcilerContract": 4.02, + "TestWithPeerAuth": 87.8, + "TestWithPeerAuth_ConcurrentPeersDoNotMixCredentials": 20.0, + "TestWithPeerAuth_DivergedRemoteURLFailsClosed": 17.14, + "TestWithPeerAuth_EnvFallbackDoesNotObservePeerCredentials": 19.57, + "TestWithPeerAuth_KeyMismatchFailsClosed": 16.51, + "TestWithPeerAuth_MissingRemoteFailsClosed": 17.76, + "TestWithPeerAuth_RestoresAbsentEnv": 17.9, + "TestWithPeerAuth_RestoresEnvWhenCallbackFails": 16.84, + "TestWithPeerAuth_WarnsWhenStoredPeerSuppressesAmbientPassword": 121.04, + "TestWorkspaceConfigContract": 7.42 + } +} diff --git a/scripts/ci/gen_embedded_cmd_shard_manifest.py b/scripts/ci/gen_embedded_cmd_shard_manifest.py new file mode 100755 index 000000000..a93255eff --- /dev/null +++ b/scripts/ci/gen_embedded_cmd_shard_manifest.py @@ -0,0 +1,210 @@ +#!/usr/bin/env python3 +"""Regenerate .github/scripts/embedded-cmd-test-shards.txt by cost-balancing. + +Discovery must stay in step with embedded-test-shard.sh: both select every +top-level TestEmbedded* function from cmd/bd/*_embedded_test.go. + +Two cost models, selected by --weights (default: inits, matching +gen_proxied_shard_manifest.py's convention — see that script's module +docstring for the full rationale this one shares): + + --weights=inits (default): a cheap static proxy for wall-time — the + function's subtest count (t.Run( occurrences), floored at 1. Unlike the + proxied generator's bd-init count, embedded cmd tests do not share one + dominant per-call cost center, so subtest count is the simplest proxy + that is still monotonic in "more scenarios this function exercises." + + --weights=duration: measured wall-time from + scripts/ci/embedded_cmd_test_durations.json (see that file's header for + provenance). A function missing from that file — a test added since it + was last captured — falls back to its inits cost times the file's + "seconds_per_init_fallback" ratio. + +This file's existing 20-shard block (hand-assigned, round-robin; see its own +header) is the FROZEN block PR Risk's and main.yml's legacy fork/push jobs +read — see engdocs/TESTING.md. Do NOT regenerate it. This generator's +--weights=duration path targets a *separate*, Bazel-only shard total (the +bazel-embedded job in .github/workflows/bazel.yml): a different total_shards +value, written as its own block in the same manifest file (see +split_blocks()/write_block() in _embedded_shard_manifest_lib.py for how one +file holds more than one independent block). + +--write is INCREMENTAL by default (see --repack to force a full rebalance); +--check verifies exact-once NAME coverage only, not shard assignments. Both +behave exactly like gen_proxied_shard_manifest.py's flags of the same name; +see that script's module docstring for the complete explanation (merge- +conflict avoidance, fork-safety, etc.) — it is not repeated here. + +Usage: gen_embedded_cmd_shard_manifest.py [total_shards] [--weights=inits|duration] + [--write [--repack] | --check] [--manifest PATH] +""" +import argparse +import glob +import json +import os +import re +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _embedded_shard_manifest_lib import ( # noqa: E402 + check_coverage, incremental_pack, pack, read_assignments, write_block, +) + +default_manifest_path = '.github/scripts/embedded-cmd-test-shards.txt' +durations_filename = 'embedded_cmd_test_durations.json' +default_total_shards = 20 + +func_re = re.compile(r'^func (TestEmbedded[A-Za-z0-9_]+)\(') + + +def discover_inits_cost(): + """Return {test_name: subtest count}, floored at 1.""" + costs = {} + for path in glob.glob('cmd/bd/*_embedded_test.go'): + with open(path) as fh: + lines = fh.readlines() + cur = None + for ln in lines: + m = func_re.match(ln) + if m: + cur = m.group(1) + costs.setdefault(cur, 0) + elif ln.startswith('func '): + cur = None + if cur: + costs[cur] += ln.count('t.Run(') + for k in costs: + costs[k] = max(costs[k], 1) + return costs + + +def duration_cost(inits): + """Return {test_name: seconds}, from the committed duration capture, + falling back to inits * seconds_per_init_fallback for an undated test.""" + durations_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), durations_filename) + with open(durations_path) as fh: + data = json.load(fh) + measured = data['tests'] + fallback_ratio = data['seconds_per_init_fallback'] + stale = sorted(name for name in measured if name not in inits) + if stale: + sys.stderr.write( + f'warning: {durations_filename} has {len(stale)} capture(s) for ' + 'tests that no longer exist in cmd/bd/*_embedded_test.go (renamed, split or ' + f'deleted since the last capture): {", ".join(stale)}\n') + costs = {} + for name, cost in inits.items(): + costs[name] = measured[name] if name in measured else cost * fallback_ratio + costs[name] = max(costs[name], 0.001) + return costs + + +def render(total, shards, weights): + out = [] + out.append('# Embedded-Dolt cmd/bd test shard manifest.') + out.append('#') + out.append('# Format: ') + out.append('#') + if total == default_total_shards and weights == 'inits': + out.append(f'# {total}-shard split. This block is hand-assigned (round-robin,') + out.append('# not cost-balanced) and FROZEN for pr-risk.yml/main.yml\'s legacy') + out.append('# fork/push jobs (see engdocs/TESTING.md): do not regenerate it. Newly-') + out.append('# added tests not listed here hash-distribute via embedded-test-shard.sh:') + out.append('# cksum(name) % total. --check does not cover this block; only the') + out.append('# Bazel-only --weights=duration block is checked in CI.') + else: + out.append(f'# {total}-shard split for the Bazel-only embedded-Dolt cmd tier') + out.append('# (bazel-embedded in .github/workflows/bazel.yml), bin-packed') + out.append('# longest-processing-time-first by measured wall-time from') + out.append(f'# scripts/ci/{durations_filename} (see that file for provenance') + out.append('# and its "relative weight, not absolute SLA" caveat).') + out.append(f'# Regenerate with scripts/ci/gen_embedded_cmd_shard_manifest.py {total}') + out.append('# --weights=duration --write after adding, splitting or removing') + out.append('# TestEmbedded* functions in cmd/bd/*_embedded_test.go. --write is') + out.append('# incremental: it keeps every already-assigned test on its current shard') + out.append('# and only places newly-discovered names, onto whichever shard is') + out.append('# currently lightest, so two unrelated PRs that each add one test do not') + out.append('# textually conflict or silently invalidate each other\'s packing. Pass') + out.append('# --repack to force a full from-scratch rebalance instead (a deliberate,') + out.append('# separate change — expect a large diff). --check only verifies every') + out.append('# discovered name is listed here exactly once (no stale or duplicate') + out.append('# entries); it does NOT verify the packing is still well-balanced.') + out.append('#') + out.append('# This file is generated: notes added here are erased by the next') + out.append('# regeneration. Add them to the generator or the durations file instead.') + out.append('') + for i in range(total): + for name in sorted(shards[i]): + out.append(f'{total} {i + 1} {name}') + out.append('') + return out + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split('\n\n')[0]) + ap.add_argument('total_shards', nargs='?', type=int, default=None) + ap.add_argument('--weights', choices=['inits', 'duration'], default='inits') + ap.add_argument('--manifest', default=default_manifest_path, + help=f'manifest file to read/write (default: {default_manifest_path})') + mode = ap.add_mutually_exclusive_group() + mode.add_argument('--write', action='store_true', + help='rewrite only this total_shards block in --manifest in place ' + '(incremental by default; see --repack)') + mode.add_argument('--check', action='store_true', + help="exit non-zero unless --manifest's block for this total_shards has " + 'exact-once coverage of the discovered test set') + ap.add_argument('--repack', action='store_true', + help='ignore any existing block for this total_shards and do a full ' + 'from-scratch LPT pack') + args = ap.parse_args() + if args.total_shards is None: + # N3 (F1 review): omitting total_shards used to default to 20, the + # FROZEN legacy block's own total, with --weights=inits also + # defaulting on -- so a bare `--write` silently rewrote the frozen + # legacy 20-shard block instead of erroring or targeting the + # Bazel-only block. Require the caller to say which total they mean + # whenever they're about to write; --check/dry-run keep the old + # default_total_shards=20 fallback since reading the legacy block is + # harmless and already how engdocs/TESTING.md documents inspecting it. + if args.write: + ap.error(f'total_shards is required with --write (the default, {default_total_shards}, ' + 'is the FROZEN legacy block -- see its header in embedded-cmd-test-shards.txt; ' + 'it must not be regenerated). Pass the Bazel-only total explicitly instead, e.g. ' + '"50 --weights=duration" for bazel-embedded\'s block (see cmd/bd/BUILD.bazel\'s ' + 'bd_embedded_test shard_count for the current value)') + args.total_shards = default_total_shards + + inits = discover_inits_cost() + costs = inits if args.weights == 'inits' else duration_cost(inits) + + existing = None if args.repack else read_assignments(args.manifest, args.total_shards) + if existing: + shards, loads = incremental_pack(costs, args.total_shards, existing) + else: + shards, loads = pack(costs, args.total_shards) + out = render(args.total_shards, shards, args.weights) + + unit = 'inits' if args.weights == 'inits' else 's' + fmt = (lambda v: str(v)) if args.weights == 'inits' else (lambda v: f'{v:.1f}') + mode_label = 'repack' if (args.repack or not existing) else 'incremental' + sys.stderr.write(f'shard loads ({mode_label}, est. {unit}, weights={args.weights}): ' + + ', '.join(f'{i + 1}:{fmt(loads[i])}' for i in range(args.total_shards)) + '\n') + sys.stderr.write(f'heaviest shard: {fmt(max(loads))} {unit}\n') + + if args.check: + err = check_coverage(args.manifest, args.total_shards, costs.keys(), + 'TestEmbedded*', sys.argv[0]) + if err: + sys.stderr.write('error: ' + err + '\n') + sys.exit(1) + return + if args.write: + write_block(args.manifest, args.total_shards, out) + sys.stderr.write(f'wrote {args.manifest}\'s {args.total_shards}-shard block ' + f'({mode_label})\n') + return + print('\n'.join(out).rstrip() + '\n', end='') + + +if __name__ == '__main__': + main() diff --git a/scripts/ci/gen_embedded_storage_shard_manifest.py b/scripts/ci/gen_embedded_storage_shard_manifest.py new file mode 100755 index 000000000..3df3eba97 --- /dev/null +++ b/scripts/ci/gen_embedded_storage_shard_manifest.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +"""Regenerate .github/scripts/embedded-storage-test-shards.txt by cost-balancing. + +Discovery must stay in step with embedded-storage-test-shard.sh: both select +every top-level Test* function from internal/storage/embeddeddolt/*_test.go, +excluding TestConformance (sharded separately, by sub-test path rather than +top-level function name; see that shell script's header and +engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md). + +Two cost models, selected by --weights (default: inits, matching +gen_proxied_shard_manifest.py's convention — see that script's module +docstring for the full rationale this one shares): + + --weights=inits (default): a cheap static proxy for wall-time — the + function's subtest count (t.Run( occurrences), floored at 1. + + --weights=duration: measured wall-time from + scripts/ci/embedded_storage_test_durations.json (see that file's header + for provenance). A function missing from that file falls back to its + inits cost times the file's "seconds_per_init_fallback" ratio. + +This file's existing 5-shard block (hand-assigned round-robin, with three +tests manually pinned by measured duration; see its own header) is the +FROZEN block PR Risk's and main.yml's legacy fork/push jobs read — see +engdocs/TESTING.md. Do NOT regenerate it. This generator's --weights=duration +path targets a *separate*, Bazel-only shard total (the bazel-embedded job in +.github/workflows/bazel.yml): a different total_shards value, written as its +own block in the same manifest file. + +--write is INCREMENTAL by default (see --repack to force a full rebalance); +--check verifies exact-once NAME coverage only, not shard assignments. Both +behave exactly like gen_proxied_shard_manifest.py's flags of the same name; +see that script's module docstring for the complete explanation (merge- +conflict avoidance, fork-safety, etc.) — it is not repeated here. + +Usage: gen_embedded_storage_shard_manifest.py [total_shards] [--weights=inits|duration] + [--write [--repack] | --check] [--manifest PATH] +""" +import argparse +import glob +import json +import os +import re +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from _embedded_shard_manifest_lib import ( # noqa: E402 + check_coverage, incremental_pack, pack, read_assignments, write_block, +) + +default_manifest_path = '.github/scripts/embedded-storage-test-shards.txt' +durations_filename = 'embedded_storage_test_durations.json' +default_total_shards = 5 + +func_re = re.compile(r'^func (Test[A-Za-z0-9_]+)\(') + + +def discover_inits_cost(): + """Return {test_name: subtest count}, floored at 1. Excludes + TestConformance (sharded separately).""" + costs = {} + for path in glob.glob('internal/storage/embeddeddolt/*_test.go'): + with open(path) as fh: + lines = fh.readlines() + cur = None + for ln in lines: + m = func_re.match(ln) + if m: + cur = m.group(1) + costs.setdefault(cur, 0) + elif ln.startswith('func '): + cur = None + if cur: + costs[cur] += ln.count('t.Run(') + costs.pop('TestConformance', None) + for k in costs: + costs[k] = max(costs[k], 1) + return costs + + +def duration_cost(inits): + """Return {test_name: seconds}, from the committed duration capture, + falling back to inits * seconds_per_init_fallback for an undated test.""" + durations_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), durations_filename) + with open(durations_path) as fh: + data = json.load(fh) + measured = data['tests'] + fallback_ratio = data['seconds_per_init_fallback'] + stale = sorted(name for name in measured if name not in inits) + if stale: + sys.stderr.write( + f'warning: {durations_filename} has {len(stale)} capture(s) for ' + 'tests that no longer exist in internal/storage/embeddeddolt/*_test.go ' + f'(renamed, split or deleted since the last capture): {", ".join(stale)}\n') + costs = {} + for name, cost in inits.items(): + costs[name] = measured[name] if name in measured else cost * fallback_ratio + costs[name] = max(costs[name], 0.001) + return costs + + +def render(total, shards, weights): + out = [] + out.append('# Embedded-Dolt storage test shard manifest.') + out.append('#') + out.append('# Format: ') + out.append('#') + if total == default_total_shards and weights == 'inits': + out.append(f'# {total}-shard split. This block is hand-assigned (round-robin,') + out.append('# with three tests manually pinned by measured duration; see git') + out.append('# history) and FROZEN for pr-risk.yml/main.yml\'s legacy fork/push') + out.append('# jobs (see engdocs/TESTING.md): do not regenerate it. Newly-added') + out.append('# tests not listed here hash-distribute via') + out.append('# embedded-storage-test-shard.sh: cksum(name) % total. --check does') + out.append('# not cover this block; only the Bazel-only --weights=duration block') + out.append('# is checked in CI.') + else: + out.append(f'# {total}-shard split for the Bazel-only embedded-Dolt storage tier') + out.append('# (bazel-embedded in .github/workflows/bazel.yml), bin-packed') + out.append('# longest-processing-time-first by measured wall-time from') + out.append(f'# scripts/ci/{durations_filename} (see that file for provenance') + out.append('# and its "relative weight, not absolute SLA" caveat).') + out.append(f'# Regenerate with scripts/ci/gen_embedded_storage_shard_manifest.py {total}') + out.append('# --weights=duration --write after adding or removing Test* functions in') + out.append('# internal/storage/embeddeddolt/*_test.go. --write is incremental: it') + out.append('# keeps every already-assigned test on its current shard and only places') + out.append('# newly-discovered names, onto whichever shard is currently lightest, so') + out.append('# two unrelated PRs that each add one test do not textually conflict or') + out.append('# silently invalidate each other\'s packing. Pass --repack to force a full') + out.append('# from-scratch rebalance instead (a deliberate, separate change — expect a') + out.append('# large diff). --check only verifies every discovered name is listed here') + out.append('# exactly once (no stale or duplicate entries); it does NOT verify the') + out.append('# packing is still well-balanced.') + out.append('#') + out.append('# This file is generated: notes added here are erased by the next') + out.append('# regeneration. Add them to the generator or the durations file instead.') + out.append('') + for i in range(total): + for name in sorted(shards[i]): + out.append(f'{total} {i + 1} {name}') + out.append('') + return out + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.split('\n\n')[0]) + ap.add_argument('total_shards', nargs='?', type=int, default=None) + ap.add_argument('--weights', choices=['inits', 'duration'], default='inits') + ap.add_argument('--manifest', default=default_manifest_path, + help=f'manifest file to read/write (default: {default_manifest_path})') + mode = ap.add_mutually_exclusive_group() + mode.add_argument('--write', action='store_true', + help='rewrite only this total_shards block in --manifest in place ' + '(incremental by default; see --repack)') + mode.add_argument('--check', action='store_true', + help="exit non-zero unless --manifest's block for this total_shards has " + 'exact-once coverage of the discovered test set') + ap.add_argument('--repack', action='store_true', + help='ignore any existing block for this total_shards and do a full ' + 'from-scratch LPT pack') + args = ap.parse_args() + if args.total_shards is None: + # N3 (F1 review): see gen_embedded_cmd_shard_manifest.py's identical + # guard for the full rationale -- a bare `--write` used to silently + # rewrite the FROZEN legacy 5-shard block via the (20->5-shard- + # equivalent) default_total_shards/--weights=inits default pairing. + if args.write: + ap.error(f'total_shards is required with --write (the default, {default_total_shards}, ' + 'is the FROZEN legacy block -- see its header in embedded-storage-test-shards.txt; ' + 'it must not be regenerated). Pass the Bazel-only total explicitly instead, e.g. ' + '"15 --weights=duration" for bazel-embedded\'s block (see ' + 'internal/storage/embeddeddolt/BUILD.bazel\'s embeddeddolt_embedded_test ' + 'shard_count for the current value)') + args.total_shards = default_total_shards + + inits = discover_inits_cost() + costs = inits if args.weights == 'inits' else duration_cost(inits) + + existing = None if args.repack else read_assignments(args.manifest, args.total_shards) + if existing: + shards, loads = incremental_pack(costs, args.total_shards, existing) + else: + shards, loads = pack(costs, args.total_shards) + out = render(args.total_shards, shards, args.weights) + + unit = 'inits' if args.weights == 'inits' else 's' + fmt = (lambda v: str(v)) if args.weights == 'inits' else (lambda v: f'{v:.1f}') + mode_label = 'repack' if (args.repack or not existing) else 'incremental' + sys.stderr.write(f'shard loads ({mode_label}, est. {unit}, weights={args.weights}): ' + + ', '.join(f'{i + 1}:{fmt(loads[i])}' for i in range(args.total_shards)) + '\n') + sys.stderr.write(f'heaviest shard: {fmt(max(loads))} {unit}\n') + + if args.check: + err = check_coverage(args.manifest, args.total_shards, costs.keys(), + 'Test* (excluding TestConformance)', sys.argv[0]) + if err: + sys.stderr.write('error: ' + err + '\n') + sys.exit(1) + return + if args.write: + write_block(args.manifest, args.total_shards, out) + sys.stderr.write(f'wrote {args.manifest}\'s {args.total_shards}-shard block ' + f'({mode_label})\n') + return + print('\n'.join(out).rstrip() + '\n', end='') + + +if __name__ == '__main__': + main() diff --git a/scripts/ci_workflow_test.go b/scripts/ci_workflow_test.go index 68de8ef6f..475eca4b0 100644 --- a/scripts/ci_workflow_test.go +++ b/scripts/ci_workflow_test.go @@ -1060,22 +1060,29 @@ func TestDoltTestcontainerStepsDisableRyuk(t *testing.T) { // TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite pins the nightly // full-test job's embedded-Dolt step: it must set BEADS_TEST_EMBEDDED_DOLT=1 // (the gate skipUnlessEmbeddedDolt checks) and run exactly -// TestBatchApplyContract and TestLargeBatchApplyWallClock_Embedded, non-race, -// after the main "Full Test Suite" step. That main step never sets -// BEADS_TEST_EMBEDDED_DOLT, so without this step the nightly job would never -// exercise a real 1000-item apply through the embedded backend at all — see -// the step's own comment in nightly.yml for why non-race and why these two -// tests specifically. +// TestBatchApplyContract, TestLargeBatchApplyWallClock_Embedded and (F1) +// TestLargeBatchApplyStatementCounts712_Embedded, non-race, after the main +// "Full Test Suite" step. That main step never sets BEADS_TEST_EMBEDDED_DOLT, +// so without this step the nightly job would never exercise a real +// 1000-item apply through the embedded backend, nor the 712-item shape's +// pinned statement-count baseline, at all — see the step's own comment in +// nightly.yml for why non-race and why these tests specifically. func TestNightlyFullTestRunsEmbeddedDoltBatchApplySuite(t *testing.T) { job := readCIWorkflow(t, "nightly.yml").job(t, "full-test") const stepName = "Embedded Dolt batch-apply suite (non-race)" - const wantRun = "go test -tags gms_pure_go -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded)$' ./internal/storage/embeddeddolt" + const wantRun = "go test -tags gms_pure_go -timeout 20m -run '^(TestBatchApplyContract|TestLargeBatchApplyWallClock_Embedded|TestLargeBatchApplyStatementCounts712_Embedded)$' ./internal/storage/embeddeddolt" assertStepRunsExactly(t, job, stepName, wantRun) assertStepEnvValue(t, job, stepName, "BEADS_TEST_EMBEDDED_DOLT", "1") assertStepsBefore(t, job, []string{"Full Test Suite (including integration tests)"}, []string{stepName}) if strings.Contains(wantRun, "-race") { t.Errorf("embedded-dolt nightly step run = %q, must stay non-race (race dramatically inflates this backend's own wall-clock)", wantRun) } + // N4 (F1 review): this step must still run (and report) even if an + // earlier nightly step failed, and must carry its own explicit Go + // timeout independent of the job-level timeout-minutes. + if got := job.step(t, stepName).If; got != "${{ !cancelled() }}" { + t.Errorf("embedded-dolt nightly step if = %q, want ${{ !cancelled() }}", got) + } } func TestPRPreflightPlatformsRunsTestScriptPrebuiltBinaryContract(t *testing.T) { @@ -4958,10 +4965,20 @@ func TestBazelEmbeddedJobMirrorsEmbeddedTier(t *testing.T) { } root := sourceRepoRoot(t) rule := bazelRuleBlock(readPolicyFile(t, root, c.pkg+"/BUILD.bazel"), c.target) + // c.target's own shard_count is deliberately NOT asserted to equal + // shards (pr-risk.yml's matrix size) here: since slice F1, the + // Bazel-only bazel-embedded job (bazel.yml) shards both of these + // targets on its own duration-balanced manifest block, a different + // total from PR Risk's/main.yml's legacy "Test (Embedded Dolt {Cmd + // N/20,Storage N/5})" jobs' frozen block (same reasoning as + // cmd/bd:bd_proxied_test's noBuildShardPin above, F2). The real + // cross-file pin for c.target's shard_count lives in + // pr_risk_bazel_coverage_test.go's TestBazelRetiredLanesCheckListedTestsRan, + // which ties it to bazel.yml's check_shard_coverage.py argument via + // bazelEmbeddedCmdShardCount/bazelEmbeddedStorageShardCount. for _, want := range []string{ `srcs = ["//tools/bazel:go_test_manifest_shard.sh"],`, `"$(rootpath //:` + c.script + `)",`, - "shard_count = " + strconv.Itoa(shards) + ",", `"BEADS_TEST_EMBEDDED_DOLT": "1"`, `"embedded"`, } { diff --git a/scripts/embedded_shard_count_test.go b/scripts/embedded_shard_count_test.go new file mode 100644 index 000000000..ebe26c5b4 --- /dev/null +++ b/scripts/embedded_shard_count_test.go @@ -0,0 +1,123 @@ +package scripts_test + +import ( + "os" + "os/exec" + "regexp" + "strconv" + "testing" +) + +// shardCountPattern matches a rule block's ` shard_count = N,` line, the +// same pattern bazelProxiedShardCount (scripts/ci_workflow_test.go) inlines +// for its own single use; shared here since both embedded accessors below +// need it. +var shardCountPattern = regexp.MustCompile(`(?m)^ shard_count = (\d+),$`) + +// bazelEmbeddedCmdShardCount returns cmd/bd:bd_embedded_test's own +// shard_count from cmd/bd/BUILD.bazel: the single source of truth for the +// Bazel-only bazel-embedded lane's cmd/bd shard split (slice F1), which no +// longer has to equal PR Risk's/main.yml's legacy "Test (Embedded Dolt Cmd +// N/20)" fork/push jobs' matrix size — mirrors bazelProxiedShardCount (F2; +// see that function's doc comment in scripts/ci_workflow_test.go for the +// shared rationale, not repeated here). Under `bazel test`, scripts_test's +// runfiles hold no other package's BUILD file, so this falls back to the +// literal TestBazelRetiredLanesCheckListedTestsRan pins under plain `go +// test` for //cmd/bd:bd_embedded_test; that test fails if cmd/bd/BUILD.bazel's +// shard_count ever drifts from this fallback. +func bazelEmbeddedCmdShardCount(t *testing.T) int { + t.Helper() + const bazelTestFallback = 50 + if os.Getenv("TEST_SRCDIR") != "" { + return bazelTestFallback + } + root := sourceRepoRoot(t) + rule := bazelRuleBlock(readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "bd_embedded_test") + m := shardCountPattern.FindStringSubmatch(rule) + if m == nil { + t.Fatalf("cmd/bd:bd_embedded_test has no `shard_count = N,` in cmd/bd/BUILD.bazel:\n%s", rule) + } + n, err := strconv.Atoi(m[1]) + if err != nil { + t.Fatalf("cmd/bd:bd_embedded_test shard_count: %v", err) + } + return n +} + +// bazelEmbeddedStorageShardCount returns +// embeddeddolt:embeddeddolt_embedded_test's own shard_count from +// internal/storage/embeddeddolt/BUILD.bazel: the single source of truth for +// the Bazel-only bazel-embedded lane's storage shard split (slice F1), which +// no longer has to equal PR Risk's/main.yml's legacy "Test (Embedded Dolt +// Storage N/5)" fork/push jobs' matrix size. Mirrors +// bazelEmbeddedCmdShardCount above; see its doc comment. +func bazelEmbeddedStorageShardCount(t *testing.T) int { + t.Helper() + const bazelTestFallback = 15 + if os.Getenv("TEST_SRCDIR") != "" { + return bazelTestFallback + } + root := sourceRepoRoot(t) + rule := bazelRuleBlock(readPolicyFile(t, root, "internal/storage/embeddeddolt/BUILD.bazel"), "embeddeddolt_embedded_test") + m := shardCountPattern.FindStringSubmatch(rule) + if m == nil { + t.Fatalf("embeddeddolt:embeddeddolt_embedded_test has no `shard_count = N,` in internal/storage/embeddeddolt/BUILD.bazel:\n%s", rule) + } + n, err := strconv.Atoi(m[1]) + if err != nil { + t.Fatalf("embeddeddolt:embeddeddolt_embedded_test shard_count: %v", err) + } + return n +} + +// S3 (F1, mirroring F2's TestProxiedShardManifestGeneratorNotStale in +// scripts/pr_risk_bazel_coverage_test.go — see that test's doc comment for +// the full --check rationale, not repeated here): the Bazel-only 50-shard +// cmd block and 15-shard storage block are not frozen like their files' +// legacy 20- and 5-shard blocks. gen_embedded_{cmd,storage}_shard_manifest.py +// --check verifies only that the committed block names every discovered +// test exactly once, failing with the exact command to fix it when a name +// is missing, stale, or duplicated — run here so a drifted block fails `go +// test ./scripts/...` (and scripts-go-checks on fork PRs) instead of only +// surfacing as a test silently never running in any shard. The legacy +// blocks are deliberately excluded: both files document that their 20- and +// 5-shard blocks are frozen (see .github/scripts/embedded-{cmd,storage}- +// test-shards.txt and engdocs/TESTING.md), so a --check against them is +// expected to report "missing" entries by design (see those generators' +// module docstrings) and is not what this test runs. +func TestCmdEmbeddedShardManifestGeneratorNotStale(t *testing.T) { + if os.Getenv("TEST_SRCDIR") != "" { + t.Skip("scripts_test's runfiles hold neither the generator's sources nor cmd/bd") + } + python, err := exec.LookPath("python3") + if err != nil { + t.Skip("python3 not available") + } + root := sourceRepoRoot(t) + shards := strconv.Itoa(bazelEmbeddedCmdShardCount(t)) + cmd := exec.Command(python, "scripts/ci/gen_embedded_cmd_shard_manifest.py", shards, "--weights=duration", "--check") + cmd.Dir = root + if out, err := cmd.CombinedOutput(); err != nil { + t.Errorf("gen_embedded_cmd_shard_manifest.py %s --weights=duration --check: %v\n%s", shards, err, out) + } +} + +// TestStorageEmbeddedShardManifestGeneratorNotStale mirrors +// TestCmdEmbeddedShardManifestGeneratorNotStale above for the storage tier's +// Bazel-only 15-shard block; see that test's doc comment. +func TestStorageEmbeddedShardManifestGeneratorNotStale(t *testing.T) { + if os.Getenv("TEST_SRCDIR") != "" { + t.Skip("scripts_test's runfiles hold neither the generator's sources nor cmd/bd") + } + python, err := exec.LookPath("python3") + if err != nil { + t.Skip("python3 not available") + } + root := sourceRepoRoot(t) + shards := strconv.Itoa(bazelEmbeddedStorageShardCount(t)) + cmd := exec.Command(python, "scripts/ci/gen_embedded_storage_shard_manifest.py", shards, "--weights=duration", "--check") + cmd.Dir = root + if out, err := cmd.CombinedOutput(); err != nil { + t.Errorf("gen_embedded_storage_shard_manifest.py %s --weights=duration --check: %v\n%s", shards, err, out) + } +} diff --git a/scripts/pr_risk_bazel_coverage_test.go b/scripts/pr_risk_bazel_coverage_test.go index a21afb897..54c08748c 100644 --- a/scripts/pr_risk_bazel_coverage_test.go +++ b/scripts/pr_risk_bazel_coverage_test.go @@ -1081,20 +1081,44 @@ func TestBazelRetiredLanesCheckListedTestsRan(t *testing.T) { // a different number of shards than this PR Risk job's matrix. Zero // means "same as this suite's PR Risk job matrix size" (the common // case for a lane that is a drop-in retirement of the legacy job). - // -1 means "read cmd/bd:bd_proxied_test's own shard_count from - // cmd/bd/BUILD.bazel" (bazelProxiedShardCount); only label == - // "//cmd/bd:bd_proxied_test" may use -1 (enforced below), so - // copying this onto another suite needs a reviewed change there. + // -1 means "read this suite's label's own shard_count live from its + // BUILD.bazel rule", via the liveShardCount dispatch table below + // (bazelProxiedShardCount, bazelEmbeddedCmdShardCount, + // bazelEmbeddedStorageShardCount); only a label present in that + // table may use -1 (enforced below), so copying this onto another + // suite needs a reviewed change there. shardCount int } + // liveShardCount dispatches a suite's -1 shardCount to the accessor that + // reads its target's own shard_count from its BUILD.bazel rule — the + // single source of truth for a Bazel-only lane's shard split, which no + // longer has to equal the retired PR Risk job's matrix size (F2's + // bd_proxied_test; F1's bd_embedded_test and + // embeddeddolt_embedded_test). See bazelProxiedShardCount's doc comment + // (scripts/ci_workflow_test.go) for the shared rationale. + liveShardCount := map[string]func(*testing.T) int{ + "//cmd/bd:bd_proxied_test": bazelProxiedShardCount, + "//cmd/bd:bd_embedded_test": bazelEmbeddedCmdShardCount, + "//internal/storage/embeddeddolt:embeddeddolt_embedded_test": bazelEmbeddedStorageShardCount, + } for _, c := range []struct { lane, config string suites []suite whole []string }{ {bazelEmbedJobName, "embedded", []suite{ - {"test-embedded-cmd", "Test", "//cmd/bd:bd_embedded_test", ".github/scripts/embedded-test-shard.sh", 0}, - {"test-embedded-storage", "Test", "//internal/storage/embeddeddolt:embeddeddolt_embedded_test", ".github/scripts/embedded-storage-test-shard.sh", 0}, + // bazel-embedded runs its own duration-balanced manifest blocks + // (scripts/ci/embedded_{cmd,storage}_test_durations.json), not + // PR Risk's frozen 20- and 5-shard blocks (slice F1): neither is + // a drop-in retirement of its legacy job's shard count, just its + // tests. bazelEmbeddedCmdShardCount/bazelEmbeddedStorageShardCount + // read the real counts from BUILD.bazel, making these suites' + // `want` (below) genuine cross-file pins against bazel.yml's own + // check_shard_coverage.py arguments, not independently + // hard-coded literals that could drift from BUILD.bazel + // unnoticed (S1). + {"test-embedded-cmd", "Test", "//cmd/bd:bd_embedded_test", ".github/scripts/embedded-test-shard.sh", -1}, + {"test-embedded-storage", "Test", "//internal/storage/embeddeddolt:embeddeddolt_embedded_test", ".github/scripts/embedded-storage-test-shard.sh", -1}, }, []string{"//internal/storage/embeddeddolt:embeddeddolt_conformance_core_test", "//internal/storage/embeddeddolt:embeddeddolt_conformance_audit_test"}}, {bazelProxiedJobName, "doltserver-proxied", []suite{ // bazel-proxied runs its own duration-balanced manifest block @@ -1106,9 +1130,7 @@ func TestBazelRetiredLanesCheckListedTestsRan(t *testing.T) { // genuine cross-file pin against bazel.yml's own // check_shard_coverage.py argument, not an independently // hard-coded literal that could drift from BUILD.bazel unnoticed - // (S1). This -1 sentinel is deliberately restricted to - // bd_proxied_test below: copying it onto another suite requires - // touching that guard, not just this literal. + // (S1). {"test-proxied-cmd", "Test proxied-server cmd shard", "//cmd/bd:bd_proxied_test", ".github/scripts/proxied-test-shard.sh", -1}, }, nil}, {bazelServerJobName, "doltserver-integration", []suite{ @@ -1122,10 +1144,11 @@ func TestBazelRetiredLanesCheckListedTestsRan(t *testing.T) { shards := s.shardCount switch { case shards < 0: - if s.label != "//cmd/bd:bd_proxied_test" { - t.Fatalf("%s: shardCount<0 (read live from BUILD.bazel) is only allowed for //cmd/bd:bd_proxied_test, not %s", s.job, s.label) + fn, ok := liveShardCount[s.label] + if !ok { + t.Fatalf("%s: shardCount<0 (read live from BUILD.bazel) is not configured for %s", s.job, s.label) } - shards = bazelProxiedShardCount(t) + shards = fn(t) case shards == 0: shards = len(risk.job(t, s.job).Strategy.Matrix.Shard) } @@ -1394,8 +1417,21 @@ func TestShardScriptsListOnlyRealTests(t *testing.T) { totalsSet[n] = true } } - if c.script == ".github/scripts/proxied-test-shard.sh" { + switch c.script { + case ".github/scripts/proxied-test-shard.sh": totalsSet[bazelProxiedShardCount(t)] = true + case ".github/scripts/embedded-test-shard.sh": + // F1: the Bazel-only bazel-embedded lane reads a 50-shard cmd + // block that no PR-Risk-matrix-only (20-shard) check exercises; + // without this, "BUILD.bazel's shard_count and bazel.yml's + // check_shard_coverage.py arg both drift to a new total with no + // manifest block" passes every policy test (see review S3) -- + // the lane then silently goes 100% hash fallback and loses its + // duration balancing. + totalsSet[bazelEmbeddedCmdShardCount(t)] = true + case ".github/scripts/embedded-storage-test-shard.sh": + // F1: mirrors the cmd case above for the 15-shard storage block. + totalsSet[bazelEmbeddedStorageShardCount(t)] = true } totals := make([]int, 0, len(totalsSet)) for n := range totalsSet { diff --git a/tests/regression/discovery_test.go b/tests/regression/discovery_test.go index 101ad9c42..3286be267 100644 --- a/tests/regression/discovery_test.go +++ b/tests/regression/discovery_test.go @@ -2792,7 +2792,7 @@ func TestDiscovery_BlockedNonexistentParentSilentEmpty(t *testing.T) { // test's assertions describe the fixed behavior and stay as a discovery-suite // regression check — this file is `//go:build regression && discovery` and no // CI lane passes the discovery tag, so the CI lock for the fix is -// TestEmbeddedLabel/TestProxiedServerLabel in cmd/bd, not this test. Same +// TestEmbeddedLabelAddRemove/TestProxiedServerLabel in cmd/bd, not this test. Same // class as BUG-42 (dep rm nonexistent says "Removed"). // // Classification: BUG (fixed) — false positive confirmation. @@ -2835,7 +2835,7 @@ func TestDiscovery_LabelRemoveNonexistentSilentSuccess(t *testing.T) { // duplicate row); the message now says " already has label 'x'" and the // command still exits 0. Like the test above this stays as a discovery-suite // regression check and is not a CI gate; the CI lock is -// TestEmbeddedLabel/TestProxiedServerLabel in cmd/bd. +// TestEmbeddedLabelAddRemove/TestProxiedServerLabel in cmd/bd. // // Classification: BUG (fixed) — misleading success message. func TestDiscovery_LabelAddDuplicateReportsAdded(t *testing.T) { diff --git a/tools/bazel/equivalence_allowlist.txt b/tools/bazel/equivalence_allowlist.txt index 56b0a64d2..b2c7e4e70 100644 --- a/tools/bazel/equivalence_allowlist.txt +++ b/tools/bazel/equivalence_allowlist.txt @@ -38,6 +38,8 @@ scripts TestBazelGatedLanesNeverRetryFlakyTests skip # reads every BUILD and .b scripts TestCheckShardCoverageScript skip # runs tools/bazel/check_shard_coverage.py, not in the runfiles; runs under go test scripts TestShardScriptsListOnlyRealTests skip # runs the real .github/scripts shard scripts over the source tree; runs under go test scripts TestProxiedShardManifestGeneratorNotStale skip # runs scripts/ci/gen_proxied_shard_manifest.py --check, not in the runfiles; runs under go test +scripts TestCmdEmbeddedShardManifestGeneratorNotStale skip # runs scripts/ci/gen_embedded_cmd_shard_manifest.py --check, not in the runfiles; runs under go test +scripts TestStorageEmbeddedShardManifestGeneratorNotStale skip # runs scripts/ci/gen_embedded_storage_shard_manifest.py --check, not in the runfiles; runs under go test scripts TestShardScriptNarrowingCheck skip # reads the real .github/scripts shard scripts; runs under go test scripts TestBazelOnlySkipsAreAllowlisted skip # parses every _test.go file in the checkout; runs under go test scripts TestRunAllowlistedGoTestsScript skip # runs tools/bazel/run_allowlisted_go_tests.py, not in the runfiles; runs under go test