beads/scripts/ci
Julian Knutsen 7896abbfa8
ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 34 shards) (#7436)
* ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 28 shards)

//cmd/bd:bd_proxied_test spends nearly all its time in `bd init
--proxied-server` against the shard's shared dolt sql-server. Measured
where that goes (dolt query log): of ~2,031 statements per init, the ~220
INFORMATION_SCHEMA queries (migration guards and pre-repair probes) are
~80% of server time at a handful of databases, the 81 DOLT_COMMITs only
~0.5s. Dolt evaluates INFORMATION_SCHEMA by walking every database the
session can see, so as root each init cost the server ~1.2 CPU-seconds
more per database already on it (locally 6.7 CPU-s with 1 database, 21
with 11): a shard's k-th init paid for the k-1 projects before it.

- proxied_shared_harness_test.go: newSharedProxiedProject now provisions
  the database server-side (root creates it plus a passwordless user of
  the same name with ALL on it and nothing else) and inits with
  --proxied-server-external-user, so each project's sessions see one
  database. newSharedProxiedRootProject keeps the old root/bd-creates-it
  shape for the four tests that need a server-wide privilege (DOLT_REMOTE
  remove, CREATE DATABASE, one compact statement); a run with the server
  log in test outputs showed no other denied statement.
- Dropping finished databases instead was tried and rejected: a DROP
  DATABASE fails other sessions' in-flight INFORMATION_SCHEMA queries
  ("no root value found in session"), ~35 init failures per run.
- No assertion changed. The bd-creates-the-database path stays covered
  by the root variant, the managed-local and non-shared init tests.
- proxied_test_durations.json re-measured (two rbe-west runs): 19,618 ->
  6,345 cost-seconds. 44 -> 28 shards, LPT-packed to ~227 per shard.

rbe-west, --nocache_test_results (BEP attempt durations): before 44
shards, sum 6,063s, max 186s; scoped users at 44 shards, sum 2,337s, max
68s; at 20 shards, sum 1,950s, max 136s.

Bead: ga-vnycm2.16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): proxied lane 28 -> 34 shards for a <=100s slowest shard

PR CI at 28 shards (~227 packed cost each): 2,285 shard-seconds, median
78s, but max 113s; equally packed shards spread up to 1.45x the median.
34 shards (~187 each) targets a slowest shard under 100s.

Bead: ga-vnycm2.16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 18:22:08 +00:00
..
go-list-test-names Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
lib test(bazel): runfiles helpers + declared inputs for scan-guard tests (#6828) 2026-09-27 08:24:08 -07:00
_embedded_shard_manifest_lib.py ci(bazel): embedded-Dolt tier 20/5 → 50/15 duration-balanced Bazel shards (F1) (#7156) 2026-10-03 21:30:03 -07:00
bazel-release-cross-compile.sh ci(bazel): nogo replaces go vet and golangci-lint (ga-96smfk.18) (#7341) 2026-10-07 05:17:22 +00:00
bazel-sync-patch.sh ci: auto-push Bazel BUILD sync fixes to PRs; harden docs autofix (#6907) 2026-09-28 17:23:27 -07:00
complexity.sh ci: add advisory cyclomatic complexity reporting (#6081) 2026-08-30 19:05:17 -07:00
detect-package-gates.sh ci: merge-queue readiness for the required gates; cache test results (fresh nightly) (#7264) 2026-10-05 13:54:15 -07:00
embedded_cmd_test_durations.json ci(bazel): embedded-Dolt tier 20/5 → 50/15 duration-balanced Bazel shards (F1) (#7156) 2026-10-03 21:30:03 -07:00
embedded_storage_test_durations.json ci(bazel): cut embedded/test lane long shards (served template, bulk seed, rebalance) (#7416) 2026-10-09 08:02:40 +00:00
fmt-check.sh fix(ci): resolve gofmt from go.mod's toolchain, not PATH (be-gx8) (#6294) 2026-09-30 18:38:46 -07:00
gen_embedded_cmd_shard_manifest.py ci(bazel): embedded-Dolt tier 20/5 → 50/15 duration-balanced Bazel shards (F1) (#7156) 2026-10-03 21:30:03 -07:00
gen_embedded_storage_shard_manifest.py ci(bazel): embedded-Dolt tier 20/5 → 50/15 duration-balanced Bazel shards (F1) (#7156) 2026-10-03 21:30:03 -07:00
gen_proxied_shard_manifest.py ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 34 shards) (#7436) 2026-10-09 18:22:08 +00:00
go-test-shard-names.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
go-test-shard-packages.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
gofmt-bin.sh fix(ci): resolve gofmt from go.mod's toolchain, not PATH (be-gx8) (#6294) 2026-09-30 18:38:46 -07:00
install-dolt.sh test(bazel): run every Bazel test against a hermetic, pinned dolt CLI (#6853) 2026-09-27 14:29:49 -07:00
package-bazel-bd.sh ci: package gates start after rbe, MCP pytest -n 8, gate jobs on Blacksmith (F3) (#7126) 2026-10-03 00:03:43 -07:00
package-mcp.sh ci: bigger runners for the slowest cached-run jobs (MCP gate 8 vCPU -n16, Bazel test 4 vCPU) (#7381) 2026-10-08 05:16:20 +00:00
package-npm.sh Add PR CI wrapper command surface (#4211) 2026-05-29 18:52:53 -07:00
pr-core.sh ci(bazel): retire main.yml's Linux jobs, nightly's go test jobs and ci-measurements (#7354) 2026-10-07 10:06:59 +00:00
pr-lint.sh ci(bazel): nogo replaces go vet and golangci-lint (ga-96smfk.18) (#7341) 2026-10-07 05:17:22 +00:00
proxied_test_durations.json ci(bazel): scope the proxied lane's shared-server projects to their own database (44 -> 34 shards) (#7436) 2026-10-09 18:22:08 +00:00
pull-dolt-image.sh ci: retry Dolt image pulls (bd-m0o1z) (#5228) 2026-08-01 08:18:41 -07:00
release-targets.txt ci(bazel): cross-compile every release target with Bazel, retire the go build job (#7332) 2026-10-07 03:18:37 +00:00
run-go-test-binary.sh ci(bazel): run every Windows/macOS job on Blacksmith; retire the Linux preflight leg and the cross-compiled Windows pair (#7355) 2026-10-07 10:41:14 +00:00
test-init-gateway-credential.sh ci(pr): share restored GOCACHE with the native init credential step (#7227) 2026-10-05 01:01:23 -07:00
test-user-config-diagnostic.sh fix(config): identify native config environment in path errors (#6325) 2026-09-28 12:17:20 -07:00
test-windows-env-helpers.sh fix(gitenv): retain explicit null config suppression (#6461) 2026-09-27 15:59:08 -07:00
warm-non-race-cache.sh ci(pr): cut the Windows PR preflight leg's fixed costs (slice W) (#7378) 2026-10-08 02:42:27 +00:00