beads/scripts/ci/proxied_test_durations.json
Julian Knutsen 7896abbfa8
ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 34 shards) (#7436)
* ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 28 shards)

//cmd/bd:bd_proxied_test spends nearly all its time in `bd init
--proxied-server` against the shard's shared dolt sql-server. Measured
where that goes (dolt query log): of ~2,031 statements per init, the ~220
INFORMATION_SCHEMA queries (migration guards and pre-repair probes) are
~80% of server time at a handful of databases, the 81 DOLT_COMMITs only
~0.5s. Dolt evaluates INFORMATION_SCHEMA by walking every database the
session can see, so as root each init cost the server ~1.2 CPU-seconds
more per database already on it (locally 6.7 CPU-s with 1 database, 21
with 11): a shard's k-th init paid for the k-1 projects before it.

- proxied_shared_harness_test.go: newSharedProxiedProject now provisions
  the database server-side (root creates it plus a passwordless user of
  the same name with ALL on it and nothing else) and inits with
  --proxied-server-external-user, so each project's sessions see one
  database. newSharedProxiedRootProject keeps the old root/bd-creates-it
  shape for the four tests that need a server-wide privilege (DOLT_REMOTE
  remove, CREATE DATABASE, one compact statement); a run with the server
  log in test outputs showed no other denied statement.
- Dropping finished databases instead was tried and rejected: a DROP
  DATABASE fails other sessions' in-flight INFORMATION_SCHEMA queries
  ("no root value found in session"), ~35 init failures per run.
- No assertion changed. The bd-creates-the-database path stays covered
  by the root variant, the managed-local and non-shared init tests.
- proxied_test_durations.json re-measured (two rbe-west runs): 19,618 ->
  6,345 cost-seconds. 44 -> 28 shards, LPT-packed to ~227 per shard.

rbe-west, --nocache_test_results (BEP attempt durations): before 44
shards, sum 6,063s, max 186s; scoped users at 44 shards, sum 2,337s, max
68s; at 20 shards, sum 1,950s, max 136s.

Bead: ga-vnycm2.16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): proxied lane 28 -> 34 shards for a <=100s slowest shard

PR CI at 28 shards (~227 packed cost each): 2,285 shard-seconds, median
78s, but max 113s; equally packed shards spread up to 1.45x the median.
34 shards (~187 each) targets a slowest shard under 100s.

Bead: ga-vnycm2.16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 18:22:08 +00:00

253 lines
12 KiB
JSON

{
"_comment": [
"Measured on rbe-west, the lane's own remote executors (instance oss), on",
"2026-10-09: two `bazel test //cmd/bd:bd_proxied_test",
"--config=doltserver-proxied --nocache_test_results` runs, one against the",
"then-committed 44-shard block and one against a 20-shard repack, both",
"with the shared harness's database-scoped users",
"(cmd/bd/proxied_shared_harness_test.go), so every discovered test ran",
"exactly once per run, with the lane's -test.parallel=4 and GOMAXPROCS=4.",
"Both passed.",
"",
"Each value is a top-level test's serial-equivalent cost: the sum of the",
"<testcase time> of every leaf test under it (a subtest with no subtests of",
"its own; the test itself when it has none), each leaf averaged over the",
"two runs. It is NOT <testsuite time>: for a t.Parallel test that",
"attribute runs from the test's first RUN line to its last PASS line, so it",
"also counts the time the test sat paused behind its shard's other tests.",
"Leaves, not direct subtests, because a parallel group's own <testcase",
"time> is 0.",
"",
"Nearly all of a test's cost is `bd init --proxied-server` against the",
"shard's dolt sql-server, ~one per subtest. Before the scoped users each",
"init also paid for every database already on the server (Dolt walks every",
"database the session can see for each of the ~220 INFORMATION_SCHEMA",
"queries in the migration chain), so a test's cost depended on how many",
"projects its shard had created before it; the previous capture totalled",
"19,618 cost-seconds over the same tests, this one 6,345.",
"",
"A shard's remote wall time came out at about 0.33x its packed cost (four",
"subtests run at once), with a wide spread between equally packed shards:",
"a 20-shard repack (~335 per shard) ran median 113s, max 136s; a 28-shard",
"repack (~227) ran median 78s, max 113s in PR CI. Size the shard count for",
"the slowest shard, not the median.",
"",
"`seconds_per_init_fallback` is the sum of all costs over the sum of all",
"inits (discover_inits_cost()), used only for a test with no entry yet.",
"",
"Regenerate the same way: fresh remote runs of the target, then sum each",
"top-level test's leaf <testcase time>s from every shard's",
"bazel-testlogs/cmd/bd/bd_proxied_test/shard_*/test.xml."
],
"seconds_per_init_fallback": 9.9294,
"tests": {
"TestProxiedServerAssign": 16.45,
"TestProxiedServerAutoPrune": 8.6,
"TestProxiedServerBackupRefusedOnExternalTopology": 0.38,
"TestProxiedServerBatchDefersThenDoltCommitAdvancesHeadOnce": 8.01,
"TestProxiedServerBatchParity": 30.55,
"TestProxiedServerBlocked": 29.31,
"TestProxiedServerCascadeParity": 48.81,
"TestProxiedServerChildren": 9.53,
"TestProxiedServerCleanDatabases": 0.81,
"TestProxiedServerCleanDatabasesPurgeDropped": 2.22,
"TestProxiedServerClose": 68.94,
"TestProxiedServerClose2": 104.58,
"TestProxiedServerClose2B": 113.92,
"TestProxiedServerClose3": 50.47,
"TestProxiedServerClose4": 69.47,
"TestProxiedServerCloseB": 48.63,
"TestProxiedServerCloseBeadGate": 16.14,
"TestProxiedServerCloseConcurrent": 36.83,
"TestProxiedServerCloseSiblingBlockersFanIn": 25.27,
"TestProxiedServerComment": 24.83,
"TestProxiedServerCompactDolt": 0.2,
"TestProxiedServerCompactHistory": 1.86,
"TestProxiedServerConfig": 85.93,
"TestProxiedServerConfigB": 67.56,
"TestProxiedServerConfigConcurrent": 14.05,
"TestProxiedServerConfigSetMany": 12.34,
"TestProxiedServerContext": 9.81,
"TestProxiedServerCount": 8.05,
"TestProxiedServerCountIncludeInfra": 3.19,
"TestProxiedServerCreate": 22.17,
"TestProxiedServerCreate2": 94.47,
"TestProxiedServerCreate2B": 98.11,
"TestProxiedServerCreate2C": 70.34,
"TestProxiedServerCreateInfraTypeRoutesToWisps": 22.11,
"TestProxiedServerDeferAutoWake": 3.47,
"TestProxiedServerDeferUndefer": 9.21,
"TestProxiedServerDelete": 10.21,
"TestProxiedServerDeleteBlockedJSONSingleDoc": 9.34,
"TestProxiedServerDeleteConcurrent": 11.56,
"TestProxiedServerDeleteForceOrphans": 6.2,
"TestProxiedServerDeleteWisp": 4.71,
"TestProxiedServerDep": 91.86,
"TestProxiedServerDep2": 131.93,
"TestProxiedServerDep2B": 57.41,
"TestProxiedServerDep2C": 75.41,
"TestProxiedServerDepB": 67.92,
"TestProxiedServerDepConcurrent": 19.74,
"TestProxiedServerDepListSingleIDWarnsOnDroppedExternalEdge": 8.32,
"TestProxiedServerDoltLifecycle": 0.13,
"TestProxiedServerDoltRemoteRemove": 90.8,
"TestProxiedServerDuplicates": 22.36,
"TestProxiedServerEdit": 9.97,
"TestProxiedServerEpic": 8.17,
"TestProxiedServerEventsJournal": 4.62,
"TestProxiedServerEventsJournalOffByDefault": 5.95,
"TestProxiedServerExport": 14.47,
"TestProxiedServerExternalCreate": 4.62,
"TestProxiedServerExternalMutationPolicy": 8.86,
"TestProxiedServerExternalPolicyDirectParity": 6.31,
"TestProxiedServerFindDuplicates": 22.0,
"TestProxiedServerGC": 2.77,
"TestProxiedServerGateCheck": 38.27,
"TestProxiedServerGateCheckB": 41.55,
"TestProxiedServerGateCreateInheritsRepoMetadata": 2.72,
"TestProxiedServerGateFrontDoorDirectServerProxyParity": 17.87,
"TestProxiedServerGateLifecycle": 40.73,
"TestProxiedServerGateList": 20.14,
"TestProxiedServerGatedRendering": 4.95,
"TestProxiedServerGraph": 4.6,
"TestProxiedServerHeartbeat": 4.61,
"TestProxiedServerHistory": 3.5,
"TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix": 21.43,
"TestProxiedServerHistoryRemoteSupportedFrontDoorParity": 6.74,
"TestProxiedServerHuman": 79.25,
"TestProxiedServerIfRevisionGuardAssign": 0.47,
"TestProxiedServerIfRevisionGuardClose": 0.67,
"TestProxiedServerIfRevisionGuardDelete": 0.59,
"TestProxiedServerIfRevisionGuardReopen": 1.39,
"TestProxiedServerIfRevisionGuardUpdate": 0.79,
"TestProxiedServerImport": 92.69,
"TestProxiedServerInfo": 2.71,
"TestProxiedServerInitExplicitBeadsDirIsAuthoritative": 9.07,
"TestProxiedServerInitIfMissing": 0.39,
"TestProxiedServerKV": 91.89,
"TestProxiedServerLabel": 75.59,
"TestProxiedServerLabelB": 60.23,
"TestProxiedServerLabelC": 105.9,
"TestProxiedServerLink": 26.12,
"TestProxiedServerLint": 2.66,
"TestProxiedServerList": 35.23,
"TestProxiedServerListComments": 0.81,
"TestProxiedServerListSortRetiresTheWalk": 0.08,
"TestProxiedServerListWatchCountsTheOffsetIntoMaxRows": 0.44,
"TestProxiedServerMemory": 26.25,
"TestProxiedServerMolBond": 72.46,
"TestProxiedServerMolBurn": 29.07,
"TestProxiedServerMolCurrent": 89.43,
"TestProxiedServerMolDistill": 23.19,
"TestProxiedServerMolLastActivity": 35.41,
"TestProxiedServerMolProgress": 34.63,
"TestProxiedServerMolReadyGated": 93.82,
"TestProxiedServerMolShow": 24.26,
"TestProxiedServerMolSquash": 45.59,
"TestProxiedServerMolStale": 36.09,
"TestProxiedServerMoleculeFrontDoorDirectServerProxyParity": 1.62,
"TestProxiedServerMoleculeFrontDoorWriteParity": 23.11,
"TestProxiedServerMultiStatementSQL": 5.6,
"TestProxiedServerNote": 9.88,
"TestProxiedServerOrphans": 1.81,
"TestProxiedServerOutageReconnectAcceptanceMatrix": 36.12,
"TestProxiedServerPathHelpers": 0.0,
"TestProxiedServerPing": 0.27,
"TestProxiedServerPour": 42.98,
"TestProxiedServerPourRejectsEnumViolatingVar": 10.18,
"TestProxiedServerPrime": 49.46,
"TestProxiedServerPriority": 8.17,
"TestProxiedServerPromote": 24.66,
"TestProxiedServerPrune": 56.2,
"TestProxiedServerPurgeWispsPlaneRetention": 20.91,
"TestProxiedServerQuery": 9.29,
"TestProxiedServerQuick": 21.8,
"TestProxiedServerReady": 96.02,
"TestProxiedServerReady2": 61.85,
"TestProxiedServerReadyB": 43.87,
"TestProxiedServerReadyC": 74.0,
"TestProxiedServerReadyClaimConcurrent": 7.96,
"TestProxiedServerReadyConcurrent": 9.66,
"TestProxiedServerReadyWisp": 95.28,
"TestProxiedServerReassignFence": 31.82,
"TestProxiedServerReclaim": 0.52,
"TestProxiedServerRecomputeBlocked": 56.46,
"TestProxiedServerRelate": 70.72,
"TestProxiedServerReopen": 66.48,
"TestProxiedServerReopenB": 59.36,
"TestProxiedServerReopenConcurrent": 48.76,
"TestProxiedServerSQL": 2.47,
"TestProxiedServerSQLDatabaseFlag": 13.98,
"TestProxiedServerSQLStatementRouting": 1.43,
"TestProxiedServerSearch": 33.73,
"TestProxiedServerServeActivatesTheEventsJournal": 9.88,
"TestProxiedServerServeAddComment": 1.6,
"TestProxiedServerServeAddDependencies": 5.68,
"TestProxiedServerServeBatchClose": 4.48,
"TestProxiedServerServeClaim": 5.07,
"TestProxiedServerServeClaimNext": 9.38,
"TestProxiedServerServeClose": 13.13,
"TestProxiedServerServeCount": 7.19,
"TestProxiedServerServeCreate": 1.13,
"TestProxiedServerServeDelete": 3.95,
"TestProxiedServerServeEdgeCounts": 10.32,
"TestProxiedServerServeLifecycle": 0.05,
"TestProxiedServerServeModeGate": 23.65,
"TestProxiedServerServeReadParity": 3.51,
"TestProxiedServerServeReadRevision": 3.24,
"TestProxiedServerServeReadsTheJournalItJustWrote": 16.7,
"TestProxiedServerServeRefusesAStaleCheckpointWithGone": 16.72,
"TestProxiedServerServeRefusesAStreamOnADisabledJournal": 7.45,
"TestProxiedServerServeRefusesTheJournalWhenItIsDisabled": 12.84,
"TestProxiedServerServeRelatedIssues": 7.45,
"TestProxiedServerServeRelease": 1.96,
"TestProxiedServerServeRemoveDependency": 3.87,
"TestProxiedServerServeSettingsWrites": 1.94,
"TestProxiedServerServeStreamsTheJournalItIsWriting": 24.92,
"TestProxiedServerServeUpdate": 7.19,
"TestProxiedServerServeUpdateClaim": 5.33,
"TestProxiedServerSetState": 4.87,
"TestProxiedServerShow": 77.15,
"TestProxiedServerShow2": 110.3,
"TestProxiedServerShow2B": 88.13,
"TestProxiedServerShow3": 53.11,
"TestProxiedServerShow3B": 77.05,
"TestProxiedServerShowB": 59.27,
"TestProxiedServerShowWatch": 58.3,
"TestProxiedServerStale": 3.77,
"TestProxiedServerState": 74.91,
"TestProxiedServerStatus": 7.78,
"TestProxiedServerStatuses": 1.42,
"TestProxiedServerStorageClass": 72.09,
"TestProxiedServerSwarm": 4.54,
"TestProxiedServerTag": 24.55,
"TestProxiedServerTodo": 24.62,
"TestProxiedServerTypes": 2.06,
"TestProxiedServerTypesAgreeWithCreate": 16.18,
"TestProxiedServerUnclaim": 6.9,
"TestProxiedServerUnclaimIfAssigneeParity": 23.02,
"TestProxiedServerUpdate": 99.24,
"TestProxiedServerUpdate2": 66.73,
"TestProxiedServerUpdate2B": 106.27,
"TestProxiedServerUpdate2C": 27.48,
"TestProxiedServerUpdate3": 108.59,
"TestProxiedServerUpdateB": 84.72,
"TestProxiedServerUpdateC": 118.83,
"TestProxiedServerUpdateClosePolicy": 34.08,
"TestProxiedServerUpdateConcurrentAppendNotes": 6.24,
"TestProxiedServerUpdateConcurrentClaim": 9.59,
"TestProxiedServerUpdateConcurrentMetadata": 8.05,
"TestProxiedServerUpdateHooks": 24.63,
"TestProxiedServerUpdateIfGuards": 31.82,
"TestProxiedServerUpdateWisp": 81.79,
"TestProxiedServerWispCreate": 33.64,
"TestProxiedServerWispCreateRejectsEnumViolatingVar": 10.52,
"TestProxiedServerWispGC": 56.11,
"TestProxiedServerWispList": 36.71,
"TestServerModeDeferAutoWake": 8.1,
"TestServerModeServe": 0.03,
"TestServerModeServeSkipsPostRunMaintenance": 20.92,
"TestServerModeSharedServeGlobalReportsTheServedDatabase": 21.54,
"TestServerModeTypesAgreeWithCreate": 17.63
}
}