beads/scripts/ci/embedded_cmd_test_durations.json
Julian Knutsen 0d865460b1
ci(bazel): split the slow embedded cmd tests and run the cmd block as 2x50 shards (#7438)
//cmd/bd:bd_embedded_test was the embedded lane's floor: at Bazel's 50-shard
cap per rule, each shard's wall time was its slowest top-level test, and
~40 tests ran 65-130s on rbe-west.

- 34 slow TestEmbedded* functions are split into 89 smaller top-level
  functions. Subtests and table cases are moved verbatim: no assertion,
  flag or data changes, the same t.Run set per file, the same skip guard and
  t.Parallel. Subtests that read state earlier ones left stay together, in
  order; each new function repeats the shared setup it needs.
- The manifest's cmd/bd block is now 100 shards, run by two targets:
  bd_embedded_test (shards 1-50) and the new bd_embedded_part2_test
  (51-100), through go_test_manifest_shard.sh --shard-offset/--shard-total.
  check_shard_coverage.py takes SHARDS@OFFSET/TOTAL and fails unless the
  targets' ranges tile the block exactly once, so every test runs in exactly
  one shard of one target; the policy tests read both rules' ranges from
  cmd/bd/BUILD.bazel.
- Durations re-measured on rbe-west with the split tests and repacked. The
  frozen legacy 20-shard block drops the renamed names (no workflow reads
  it; new names hash-distribute as before).

Not split: the *Concurrent stress tests (their bd subprocesses queue on the
workspace's single writer lock: TestEmbeddedUpdateConcurrent is ~100
serialized bd commands), TestEmbeddedPurgeWispsPlaneRetention (one stateful
sequence), and the MaxRowsList* tests (setup is 21 sequential creates every
subtest reads).

rbe-west, both targets, --nocache_test_results: slowest shard 138-163s ->
109s (TestEmbeddedUpdateConcurrent alone; every other shard <=86s), median
80-84s -> 40s, shard-seconds 4.1-4.4k -> 4.2k, critical path 213-306s ->
113s.

Bead: ga-vnycm2.17

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 20:30:52 +00:00

342 lines
16 KiB
JSON

{
"_comment": [
"Measured on rbe-west, the lane's own remote executors (instance oss), on",
"2026-10-09: one `bazel test --config=embedded --nocache_test_results",
"//cmd/bd:bd_embedded_test //cmd/bd:bd_embedded_part2_test` run (race,",
"`-test.parallel=4`, the 100-shard block split 50+50 over the two targets),",
"so every top-level TestEmbedded* function ran exactly once. It was taken",
"after the slowest functions were split into smaller top-level functions",
"(subtests moved verbatim): the sum of test-seconds fell from ~10.3k",
"(50 shards, the previous capture) to ~8.1k, partly because each shard now",
"runs fewer tests at once. The remaining >60s functions are the *Concurrent",
"stress tests, whose bd subprocesses queue on the workspace's single writer",
"lock, and fixtures that seed many issues one `bd create` at a time.",
"",
"Each value is that top-level test's own <testcase time=...> from Bazel's",
"generated test.xml, NOT the enclosing <testsuite time=...>: <testsuite",
"time> is a wall-clock-since-binary-start timestamp that does not isolate a",
"concurrently-run top-level test's own duration under -test.parallel=4,",
"while <testcase time> matches the test binary's own `--- PASS: TestX (Ys)`",
"line. Go adds a parallel test's elapsed time to its duration before it",
"waits for a parallel slot, so slot wait is not counted.",
"",
"The generator packs these with a model of the shard process",
"(_embedded_shard_manifest_lib.py's pack_concurrent): functions without a",
"top-level t.Parallel() run one after another, before the parallel ones,",
"which share -test.parallel=4 slots.",
"",
"`seconds_per_init_fallback` is sum(durations) / sum(inits) over every test",
"here (inits: discover_inits_cost()'s t.Run( count proxy), used only for a",
"brand-new test with no entry yet.",
"",
"These are a relative weight for packing, not an absolute SLA."
],
"seconds_per_init_fallback": 7.4818,
"tests": {
"TestEmbeddedAdminBlocked": 1.42,
"TestEmbeddedAdminCompactReadOnlyAllowed": 1.66,
"TestEmbeddedAdminCompactTier2Honesty": 2.62,
"TestEmbeddedAutoImportFallback_ConflictSkip_DoesNotClobber": 4.51,
"TestEmbeddedAutoImportFallback_ConflictSkip_EmptyDBImportsAll": 2.19,
"TestEmbeddedAutoImportJSONLConcurrentWriter": 24.46,
"TestEmbeddedAutoImportJSONLNoExplicitCommit": 22.47,
"TestEmbeddedAutoImportJSONLSkipsNonEmpty": 7.28,
"TestEmbeddedBackup": 23.1,
"TestEmbeddedBackupConcurrent": 26.77,
"TestEmbeddedBlocked": 9.54,
"TestEmbeddedBlockedConcurrent": 17.22,
"TestEmbeddedBlockedLabelFilters": 31.99,
"TestEmbeddedBootstrapConcurrent": 4.77,
"TestEmbeddedBootstrapExistingDryRunInit": 25.26,
"TestEmbeddedBootstrapJSONLAndGitOrigin": 46.5,
"TestEmbeddedBranch": 17.51,
"TestEmbeddedBranchConcurrent": 20.27,
"TestEmbeddedChangeDirOverridesInheritedBeadsDir": 9.53,
"TestEmbeddedChildrenBasicAndEmpty": 25.81,
"TestEmbeddedChildrenConcurrent": 67.76,
"TestEmbeddedChildrenJSONAndAllStatuses": 29.32,
"TestEmbeddedClaimPools": 35.44,
"TestEmbeddedCloseAlreadyClosedClaimNextAndMolecule": 41.27,
"TestEmbeddedCloseAlreadyClosedContinue": 19.24,
"TestEmbeddedCloseAlreadyClosedMultipleIDs": 19.19,
"TestEmbeddedCloseBasicIsolatedStores": 37.04,
"TestEmbeddedCloseBasicSharedStore": 30.42,
"TestEmbeddedCloseConcurrent": 84.21,
"TestEmbeddedCloseGuardsAndEpics": 47.46,
"TestEmbeddedComments": 9.58,
"TestEmbeddedCommentsCLI": 38.59,
"TestEmbeddedCommentsCLIConcurrent": 17.99,
"TestEmbeddedCompact": 36.04,
"TestEmbeddedCompactConcurrent": 8.04,
"TestEmbeddedCompactMisdatedChild": 10.32,
"TestEmbeddedConfig": 23.12,
"TestEmbeddedConfigConcurrent": 73.0,
"TestEmbeddedConfigNoWorkspaceStoreFreeCommands": 1.32,
"TestEmbeddedConfigValidateJSONNoWorkspaceWritesStdout": 0.27,
"TestEmbeddedContext": 10.13,
"TestEmbeddedContextConcurrent": 2.6,
"TestEmbeddedContributorCreateMultiple": 35.87,
"TestEmbeddedContributorCreateRoutesToPlanning": 32.15,
"TestEmbeddedContributorCreateShowReadsRouted": 31.17,
"TestEmbeddedCount": 54.32,
"TestEmbeddedCountConcurrent": 53.87,
"TestEmbeddedCountIncludeEphemeral": 17.46,
"TestEmbeddedCountIncludeInfra": 36.42,
"TestEmbeddedCreateCommitPending": 5.38,
"TestEmbeddedCreateConcurrent": 11.03,
"TestEmbeddedCreateCrossRepo": 31.6,
"TestEmbeddedCreateCrossRepoDryRunWithParent": 24.04,
"TestEmbeddedCreateCrossRepoUninit": 20.91,
"TestEmbeddedCreateCrossRepoWithParent": 34.75,
"TestEmbeddedCreateDryRunCrossRepoDoesNotMigrateTarget": 6.44,
"TestEmbeddedCreateDryRunDoesNotMigrate": 9.42,
"TestEmbeddedCreateDryRunRepoDoesNotInitializeTarget": 6.2,
"TestEmbeddedCreateFormCommitsLabelOnlyCreate": 4.26,
"TestEmbeddedCreateGraphAndBulk": 25.37,
"TestEmbeddedCreateJSONEchoesSubSecondTimestamps": 8.77,
"TestEmbeddedCreateRepoFromNonBeadsCwd": 4.36,
"TestEmbeddedCreateRepoProxiedTargetError": 11.3,
"TestEmbeddedCreateRepoRelativeExistingWorkspaceUnaffected": 25.79,
"TestEmbeddedCreateRepoRelativeUninitRefused": 1.95,
"TestEmbeddedCreateScalarAndRelationships": 66.4,
"TestEmbeddedCreateStorageClassBasic": 22.65,
"TestEmbeddedCreateStorageClassBatchAndMisc": 21.69,
"TestEmbeddedCreateStorageClassConflicts": 19.86,
"TestEmbeddedCreateWithGitRemote": 6.98,
"TestEmbeddedDefer": 27.86,
"TestEmbeddedDeferAutoWake": 39.08,
"TestEmbeddedDeferConcurrent": 40.97,
"TestEmbeddedDeleteBasic": 24.18,
"TestEmbeddedDeleteForceAndBatch": 23.09,
"TestEmbeddedDeleteJSONDependencyErrorContract": 21.24,
"TestEmbeddedDeletePreviewAndGuards": 27.96,
"TestEmbeddedDepAAddAndList": 33.09,
"TestEmbeddedDepAAddRejectionsAndBulk": 39.27,
"TestEmbeddedDepAAddTypesAndFlags": 32.71,
"TestEmbeddedDepAddWispSource": 17.18,
"TestEmbeddedDepAndLinkBatchAutoCommitDoesNotAdvanceHead": 37.58,
"TestEmbeddedDepBRemove": 19.99,
"TestEmbeddedDepBRemoveOutputAndRelatesTo": 32.44,
"TestEmbeddedDepBTreeAndCycles": 34.46,
"TestEmbeddedDepBulkNoCycleCheckKeepsWholeGraphGate": 10.71,
"TestEmbeddedDepConcurrent": 68.24,
"TestEmbeddedDepListSingleIDWarnsOnDroppedExternalEdge": 29.15,
"TestEmbeddedDepNoCycleCheck": 39.5,
"TestEmbeddedDiff": 29.84,
"TestEmbeddedDiffConcurrent": 40.71,
"TestEmbeddedDirectCommitPathsBatchAutoCommitDoesNotAdvanceHead": 23.64,
"TestEmbeddedDoltBlockedAndCommit": 9.95,
"TestEmbeddedDoltConcurrent": 9.47,
"TestEmbeddedDoltRemoteAndPush": 30.56,
"TestEmbeddedDuplicate": 27.59,
"TestEmbeddedDuplicateConcurrent": 42.26,
"TestEmbeddedDuplicatesConcurrent": 29.81,
"TestEmbeddedDuplicatesOwnWorkspaces": 30.03,
"TestEmbeddedDuplicatesSharedFixture": 8.63,
"TestEmbeddedEpicCloseEligibleClosesEpics": 27.95,
"TestEmbeddedEpicCloseEligibleNotEligible": 21.64,
"TestEmbeddedEpicCloseEligibleReasonFlag": 47.66,
"TestEmbeddedEpicCloseEligibleReasonOutput": 32.31,
"TestEmbeddedEpicConcurrent": 76.55,
"TestEmbeddedEpicStatus": 30.62,
"TestEmbeddedExplicitHierarchicalParentEdges": 33.58,
"TestEmbeddedExportConcurrent": 13.39,
"TestEmbeddedExportFlags": 15.66,
"TestEmbeddedExportFormats": 19.19,
"TestEmbeddedExternalCapabilityReadyAfterShip": 19.28,
"TestEmbeddedExternalMutationPolicy": 51.66,
"TestEmbeddedFederationConcurrent": 14.03,
"TestEmbeddedFederationPeers": 17.04,
"TestEmbeddedFederationStatus": 28.64,
"TestEmbeddedFindDuplicates": 29.41,
"TestEmbeddedFindDuplicatesConcurrent": 32.84,
"TestEmbeddedFlatten": 18.02,
"TestEmbeddedFlattenConcurrent": 6.31,
"TestEmbeddedGCConcurrent": 20.43,
"TestEmbeddedGCConditionalMatcherDecode": 29.32,
"TestEmbeddedGCDryRunAndSkips": 12.61,
"TestEmbeddedGCForceWithDecay": 12.32,
"TestEmbeddedGCFullAndNoForcePrompts": 18.95,
"TestEmbeddedGateConcurrent": 45.35,
"TestEmbeddedGateCreateBeadCheck": 15.83,
"TestEmbeddedGateCreateBeadRename": 45.63,
"TestEmbeddedGateCreateFlags": 16.48,
"TestEmbeddedGateCreateReady": 17.01,
"TestEmbeddedGateCreateRepoErrorsList": 15.24,
"TestEmbeddedGateInheritsDownParentChain": 26.11,
"TestEmbeddedGateList": 23.01,
"TestEmbeddedGateListScoped": 11.29,
"TestEmbeddedGateShowResolve": 23.68,
"TestEmbeddedGateWaiterCheckLifecycle": 37.46,
"TestEmbeddedGatedRendering": 63.79,
"TestEmbeddedGetDependencies": 10.67,
"TestEmbeddedGraph": 41.56,
"TestEmbeddedGraphConcurrent": 44.9,
"TestEmbeddedHistory": 32.22,
"TestEmbeddedHistoryConcurrent": 47.68,
"TestEmbeddedHooks": 3.05,
"TestEmbeddedHooksConcurrent": 2.08,
"TestEmbeddedHumanConcurrent": 14.83,
"TestEmbeddedHumanListStatsRespondDismiss": 19.75,
"TestEmbeddedHumanPositionalFileAndGuards": 32.16,
"TestEmbeddedIfRevisionReopenGuardDirect": 33.7,
"TestEmbeddedImportClosedChildWhoseParentWasDeleted": 20.54,
"TestEmbeddedImportConcurrent": 12.88,
"TestEmbeddedImportFromSources": 13.16,
"TestEmbeddedImportPromotedWispRoundtrip": 14.82,
"TestEmbeddedImportUpsertAndPrefixSync": 13.54,
"TestEmbeddedInfo": 10.01,
"TestEmbeddedInfoConcurrent": 11.69,
"TestEmbeddedInfoSuppressGitHooks": 7.53,
"TestEmbeddedInitABasic": 5.98,
"TestEmbeddedInitAForkAutoContributor": 42.22,
"TestEmbeddedInitAGitOriginRemote": 18.95,
"TestEmbeddedInitAPushConsentAndRemoteBootstrap": 40.9,
"TestEmbeddedInitArtifactRouting": 14.88,
"TestEmbeddedInitBAutoCommitAndJSONLRemote": 34.21,
"TestEmbeddedInitBRemoteSchemaAndCloneFailure": 20.41,
"TestEmbeddedInitBStealthAndForceReinit": 26.93,
"TestEmbeddedInitC": 17.76,
"TestEmbeddedInitConcurrent": 32.28,
"TestEmbeddedInitExplicitBeadsDirIsAuthoritative": 41.88,
"TestEmbeddedInitGitBootstrapRoutingDecoyInvalid": 33.8,
"TestEmbeddedInitGitBootstrapRoutingExistingExplicit": 30.63,
"TestEmbeddedInitGitBootstrapRoutingFreshQuiet": 40.37,
"TestEmbeddedInitHooksRouting": 16.2,
"TestEmbeddedInitRoleRoutingExplicitDefault": 35.88,
"TestEmbeddedInitRoleRoutingRetainedFork": 58.03,
"TestEmbeddedInitSelectedExcludeRoutingForkDecoyAuto": 29.54,
"TestEmbeddedInitSelectedExcludeRoutingForkInvalidQuiet": 46.94,
"TestEmbeddedInitSelectedExcludeRoutingStealthDecoyQuiet": 40.12,
"TestEmbeddedInitSelectedExcludeRoutingStealthInvalid": 16.07,
"TestEmbeddedKV": 26.57,
"TestEmbeddedKVConcurrent": 65.01,
"TestEmbeddedLabelAddRemoveAdd": 20.02,
"TestEmbeddedLabelAddRemoveMultiAndDuplicate": 11.76,
"TestEmbeddedLabelAddRemoveRemove": 35.57,
"TestEmbeddedLabelConcurrent": 59.05,
"TestEmbeddedLabelEditReportsListAndTemplate": 27.39,
"TestEmbeddedLabelEditReportsNoop": 27.14,
"TestEmbeddedLabelEditReportsPrefix": 25.05,
"TestEmbeddedLabelPropagateRename": 59.22,
"TestEmbeddedLint": 31.87,
"TestEmbeddedLintConcurrent": 27.16,
"TestEmbeddedListComments": 18.25,
"TestEmbeddedListConcurrent": 6.51,
"TestEmbeddedListFilteringBasicAndLabels": 72.7,
"TestEmbeddedListFilteringStatusAndHierarchy": 62.55,
"TestEmbeddedListSearchAndOutputFormats": 46.59,
"TestEmbeddedListSearchAndOutputSearchAndSort": 41.23,
"TestEmbeddedMaxRowsDepTreeAndDuplicates": 45.01,
"TestEmbeddedMaxRowsDoctorAndConfig": 24.99,
"TestEmbeddedMaxRowsGraph": 43.56,
"TestEmbeddedMaxRowsListBasic": 51.98,
"TestEmbeddedMaxRowsListLimitInteraction": 68.22,
"TestEmbeddedMaxRowsReady": 56.15,
"TestEmbeddedMaxRowsReadyMerge": 24.36,
"TestEmbeddedMemory": 35.89,
"TestEmbeddedMemoryConcurrent": 61.73,
"TestEmbeddedMemoryCorpusBudget": 19.62,
"TestEmbeddedMigrate": 17.36,
"TestEmbeddedMigrateConcurrent": 11.99,
"TestEmbeddedOnboard": 1.66,
"TestEmbeddedOnboardConcurrent": 2.25,
"TestEmbeddedOpen_EmptyDatabaseRejected": 0.0,
"TestEmbeddedOrphans": 9.01,
"TestEmbeddedOrphansConcurrent": 10.6,
"TestEmbeddedPreviewDoesNotConsumeVersionMarker": 6.7,
"TestEmbeddedPrime": 5.07,
"TestEmbeddedPrimeChangeDirDefaultTemplateMatchesTarget": 0.75,
"TestEmbeddedPrimeChangeDirPrimesTarget": 1.28,
"TestEmbeddedPrimeChangeDirRedirectNoticeFollowsTarget": 1.51,
"TestEmbeddedPrimeConcurrent": 5.89,
"TestEmbeddedPromote": 4.72,
"TestEmbeddedPromoteCLI": 26.32,
"TestEmbeddedPromoteCLIConcurrent": 31.41,
"TestEmbeddedPruneDryRunAndFilters": 27.53,
"TestEmbeddedPruneSafetyAndScope": 28.64,
"TestEmbeddedPurgeBasics": 30.34,
"TestEmbeddedPurgeConcurrent": 13.6,
"TestEmbeddedPurgeFiltered": 24.79,
"TestEmbeddedPurgeWispsPlaneRetention": 61.83,
"TestEmbeddedQuery": 30.45,
"TestEmbeddedQueryConcurrent": 39.48,
"TestEmbeddedQuick": 29.71,
"TestEmbeddedQuickConcurrent": 35.71,
"TestEmbeddedQuickstart": 5.34,
"TestEmbeddedQuickstartConcurrent": 2.05,
"TestEmbeddedQuietFlagSuppressesSuccessOutput": 26.01,
"TestEmbeddedReadOnlyCommandsSpawnNoGit": 27.76,
"TestEmbeddedReadyBlockersAndLabels": 37.82,
"TestEmbeddedReadyClaimConcurrent": 25.11,
"TestEmbeddedReadyConcurrent": 10.61,
"TestEmbeddedReadyDefaultJSONAndFlags": 29.09,
"TestEmbeddedReclaimRunsTheLeaseReclaimerRole": 16.3,
"TestEmbeddedRenamePrefixBasic": 16.2,
"TestEmbeddedRenamePrefixConcurrent": 11.31,
"TestEmbeddedRenamePrefixEdgeCases": 20.33,
"TestEmbeddedReopen": 61.04,
"TestEmbeddedReopenConcurrent": 62.19,
"TestEmbeddedRestore": 17.16,
"TestEmbeddedRestoreConcurrent": 0.0,
"TestEmbeddedRoutedMutatingSiblingAssignTagDepAssign": 21.91,
"TestEmbeddedRoutedMutatingSiblingAssignTagDepDepAdd": 16.67,
"TestEmbeddedRoutedMutatingSiblingAssignTagDepTag": 13.47,
"TestEmbeddedRoutedMutatingSiblingDeleteCloseClose": 14.03,
"TestEmbeddedRoutedMutatingSiblingDeleteCloseDelete": 36.33,
"TestEmbeddedRoutedSiblingWritesCommitTargetHeadComment": 35.58,
"TestEmbeddedRoutedSiblingWritesCommitTargetHeadNote": 11.65,
"TestEmbeddedRoutedSiblingWritesCommitTargetHeadReopen": 13.7,
"TestEmbeddedSearchConcurrent": 39.9,
"TestEmbeddedSearchQueryStatusAssigneeLabels": 31.01,
"TestEmbeddedSearchStatusLimitSortDatesMetadata": 30.41,
"TestEmbeddedSetup": 6.55,
"TestEmbeddedSetupConcurrent": 7.66,
"TestEmbeddedShip": 19.99,
"TestEmbeddedShipConcurrent": 50.72,
"TestEmbeddedShowBasicsAndJSON": 33.03,
"TestEmbeddedShowConcurrent": 86.17,
"TestEmbeddedShowDetailAndCurrentFlags": 28.1,
"TestEmbeddedShowDetailAndCurrentHistoryAndCurrent": 29.86,
"TestEmbeddedShowUnresolvableDeps": 22.88,
"TestEmbeddedSquashWispMoleculeClearsRootEphemeral": 23.14,
"TestEmbeddedStale": 24.08,
"TestEmbeddedStaleConcurrent": 41.57,
"TestEmbeddedStaleLabelFilters": 23.0,
"TestEmbeddedState": 25.83,
"TestEmbeddedStateConcurrent": 34.66,
"TestEmbeddedStatus": 19.75,
"TestEmbeddedStatusConcurrent": 22.8,
"TestEmbeddedSupersede": 37.32,
"TestEmbeddedSupersedeConcurrent": 33.14,
"TestEmbeddedSwarmConcurrent": 40.14,
"TestEmbeddedSwarmCreate": 22.83,
"TestEmbeddedSwarmCreateForceAndErrors": 26.05,
"TestEmbeddedSwarmStatusList": 27.84,
"TestEmbeddedSwarmValidate": 17.55,
"TestEmbeddedTodo": 30.49,
"TestEmbeddedTodoConcurrent": 7.62,
"TestEmbeddedTypes": 9.38,
"TestEmbeddedTypesAgreeWithCreate": 32.79,
"TestEmbeddedTypesConcurrent": 11.21,
"TestEmbeddedUnclaimBatchAndRefusals": 45.89,
"TestEmbeddedUnclaimSuccess": 26.64,
"TestEmbeddedUndefer": 39.57,
"TestEmbeddedUndeferConcurrent": 53.14,
"TestEmbeddedUpdateBatchAutoCommitDoesNotAdvanceHead": 11.06,
"TestEmbeddedUpdateConcurrent": 145.04,
"TestEmbeddedUpdateFieldsMappingAndRefs": 15.54,
"TestEmbeddedUpdateFieldsNotesClearAndDefer": 35.0,
"TestEmbeddedUpdateFieldsNotesOverwrite": 35.18,
"TestEmbeddedUpdateLifecycleClaimParentCommit": 43.24,
"TestEmbeddedUpdateLifecycleDefer": 27.33,
"TestEmbeddedUpdateRoutedStoreCommitsTargetHead": 38.07,
"TestEmbeddedVCCommitVariantsAndMerge": 20.16,
"TestEmbeddedVCConcurrent": 35.67,
"TestEmbeddedVCStatusAndCommit": 18.81,
"TestEmbeddedVersion": 0.53,
"TestEmbeddedVersionConcurrent": 0.35,
"TestEmbeddedWhere": 6.18,
"TestEmbeddedWhereConcurrent": 5.25
}
}