ci(bazel): proxied-server tier 15 → 30 duration-balanced shards; un-hide 12 proxied tests (F2) (#7129)
* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)
bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.
cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)
TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.
Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.
Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.
Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)
gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
leaving every other block's header/body text untouched) and --check
(regenerate in memory and diff against the committed block, exiting
1 with the exact --write command needed if stale). Both operate via
new split_blocks()/write_block()/check_block() helpers that round-trip
the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
split, or deleted since the last capture).
proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
not kept in sync with newly-added tests (those hash-fallback
instead), so regenerating it is not routine maintenance. Only the
header text changed; all 149 explicit assignment lines and the
"known blind spots" callouts are untouched (verified: a fresh
--write would pack all 164 currently-discovered tests explicitly,
silently changing which shard runs which test for this block, which
is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
`gen_proxied_shard_manifest.py 30 --weights=duration --write` to
describe --write/--check instead of the old manual-copy-paste
instructions. proxied_test_durations.json has not changed yet (S2
is still in progress), so this run only touched header text; the 164
assignment lines are unchanged.
pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
gen_proxied_shard_manifest.py 30 --weights=duration --check under
plain `go test` (skipped under `bazel test`, whose runfiles hold
neither the generator's sources nor cmd/bd) so a stale 30-block
(e.g. committed by hand, or left behind after adding tests without
regenerating) fails scripts-go-checks on every PR, including forks.
tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes
cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).
Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.
Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.
cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.
.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)
scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.
A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.
seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.
Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).
Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)
Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:
- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
tmp-root plus the (64-byte-truncated) test name deep enough to exceed
sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
helper (a short os.MkdirTemp("", "bds") directory) instead.
- runProxiedDeadline raced a deadline-watcher goroutine against
exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
with cmd.Cancel, which exec.CommandContext only invokes after Start, so
the read is ordered after Start's write.
With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.
TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.
The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)
The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.
Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.
Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.
Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)
The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.
Redesigned:
- --check now verifies exact-once NAME coverage of the currently
discovered TestProxiedServer*/TestServerMode* set against the
committed block -- missing, stale, or duplicated names fail with an
actionable message -- but does NOT compare shard assignments, so a
weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
keeps its current shard, stale (no longer discovered) names are
dropped, and only newly-discovered names are LPT-placed onto whichever
shard is currently lightest. A full from-scratch LPT pack now requires
the explicit --repack flag (used in the preceding commit for the
deliberate full reweight; ordinary new-test additions should use the
incremental default).
Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.
Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
ef988df84b
commit
6e9a15507e
21 changed files with 1243 additions and 195 deletions
7
.bazelrc
7
.bazelrc
|
|
@ -137,9 +137,10 @@ test:doltserver --remote_download_regex=.*/test\.(log|xml)$
|
|||
# Proxied-server tier: PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs, as
|
||||
# //cmd/bd:bd_proxied_test (dolt-server-proxied tag). Kept out of
|
||||
# --config=doltserver because bazel.yml runs that lane locally too (forks,
|
||||
# rbe=off), where 15 shards of ~10 minutes, each with a dolt sql-server busy
|
||||
# on ~4 cores, do not fit; bazel.yml runs this one only with remote
|
||||
# execution, in a job of its own (bazel-proxied). The build is
|
||||
# rbe=off), where 30 concurrent shards, each with a dolt sql-server busy on
|
||||
# ~4 cores, do not fit regardless of any single shard's wall time; bazel.yml
|
||||
# runs this one only with remote execution, in a job of its own
|
||||
# (bazel-proxied). The build is
|
||||
# --config=doltserver's: race, like PR Risk's /tmp/bd-cmd-test (main.yml's
|
||||
# twin job runs build-artifacts' non-race bd-cmd-test, so this is the
|
||||
# stricter of the two); the subprocess bd is the non-race bd_for_tests, like
|
||||
|
|
|
|||
253
.github/scripts/proxied-cmd-test-shards.txt
vendored
253
.github/scripts/proxied-cmd-test-shards.txt
vendored
|
|
@ -3,11 +3,17 @@
|
|||
# Format: <total_shards> <shard_number> <top_level_test_name>
|
||||
#
|
||||
# 15-shard split, bin-packed longest-processing-time-first by
|
||||
# estimated cost (bd-init count; per-init migration chains dominate).
|
||||
# Regenerate with scripts/ci/gen_proxied_shard_manifest.py after adding,
|
||||
# splitting, or reweighting TestProxiedServer*/TestServerMode*
|
||||
# functions. Newly-added tests not listed here hash-distribute via
|
||||
# proxied-test-shard.sh.
|
||||
# estimated cost (bd-init count; per-init migration chains dominate), as of
|
||||
# the test set that existed when this block was last generated.
|
||||
# Regenerating it (scripts/ci/gen_proxied_shard_manifest.py 15 --write) is
|
||||
# deliberately NOT part of normal maintenance: this block is frozen for
|
||||
# pr-risk.yml's/main.yml's legacy fork/push jobs (see engdocs/TESTING.md),
|
||||
# so every TestProxiedServer*/TestServerMode* function added since it was
|
||||
# last generated hash-distributes via proxied-test-shard.sh (cksum(name) %
|
||||
# total) instead of being explicitly listed here. Only reweight it on
|
||||
# purpose, and expect the shard assignments below to change substantially
|
||||
# if you do (a fresh run packs every currently-discovered test, not just
|
||||
# the ones already listed here).
|
||||
#
|
||||
# This file is generated: notes added here are erased by the next
|
||||
# regeneration. Add them to the generator instead. Known blind spots
|
||||
|
|
@ -188,3 +194,240 @@
|
|||
15 15 TestProxiedServerTodo
|
||||
15 15 TestProxiedServerUpdate3
|
||||
15 15 TestProxiedServerUpdateConcurrentAppendNotes
|
||||
|
||||
# Proxied-server cmd test shard manifest.
|
||||
#
|
||||
# Format: <total_shards> <shard_number> <top_level_test_name>
|
||||
#
|
||||
# 30-shard split for the Bazel-only proxied-server tier
|
||||
# (bazel-proxied in .github/workflows/bazel.yml), bin-packed
|
||||
# longest-processing-time-first by measured wall-time from
|
||||
# scripts/ci/proxied_test_durations.json (see that file for
|
||||
# provenance and its "relative weight, not absolute SLA" caveat).
|
||||
# Regenerate with scripts/ci/gen_proxied_shard_manifest.py 30
|
||||
# --weights=duration --write after adding or removing
|
||||
# TestProxiedServer*/TestServerMode* functions. --write is incremental:
|
||||
# it keeps every already-assigned test on its current shard and only
|
||||
# places newly-discovered names, onto whichever shard is currently
|
||||
# lightest, so two unrelated PRs that each add one test do not textually
|
||||
# conflict or silently invalidate each other's packing. Pass --repack to
|
||||
# force a full from-scratch rebalance instead (a deliberate, separate
|
||||
# change — expect a large diff). --check only verifies every discovered
|
||||
# name is listed here exactly once (no stale or duplicate entries); it
|
||||
# does NOT verify the packing is still well-balanced, so a reweighted
|
||||
# proxied_test_durations.json does not by itself fail --check — rerun
|
||||
# with --repack periodically (or whenever shard loads look uneven in CI)
|
||||
# to rebalance on purpose. Newly-added tests not listed here
|
||||
# hash-distribute via proxied-test-shard.sh: cksum(name) % total,
|
||||
# unrelated to proxied_test_durations.json (that file only feeds this
|
||||
# generator's own cost estimate for a test it has not measured yet).
|
||||
#
|
||||
# This file is generated: notes added here are erased by the next
|
||||
# regeneration. Add them to the generator or the durations file
|
||||
# instead.
|
||||
|
||||
30 1 TestProxiedServerGateList
|
||||
30 1 TestProxiedServerHuman
|
||||
30 1 TestProxiedServerReclaim
|
||||
30 1 TestProxiedServerSQLStatementRouting
|
||||
30 1 TestProxiedServerServeLifecycle
|
||||
|
||||
30 2 TestProxiedServerDelete
|
||||
30 2 TestProxiedServerDeleteForceOrphans
|
||||
30 2 TestProxiedServerListComments
|
||||
30 2 TestProxiedServerPromote
|
||||
30 2 TestProxiedServerTypesAgreeWithCreate
|
||||
|
||||
30 3 TestProxiedServerDuplicates
|
||||
30 3 TestProxiedServerReadyWisp
|
||||
30 3 TestProxiedServerServeDelete
|
||||
30 3 TestProxiedServerServeModeGate
|
||||
30 3 TestProxiedServerServeReadRevision
|
||||
|
||||
30 4 TestProxiedServerGateCreateInheritsRepoMetadata
|
||||
30 4 TestProxiedServerMolBurn
|
||||
30 4 TestProxiedServerReadyConcurrent
|
||||
30 4 TestProxiedServerServeAddDependencies
|
||||
30 4 TestProxiedServerServeRelatedIssues
|
||||
|
||||
30 5 TestProxiedServerCleanDatabases
|
||||
30 5 TestProxiedServerCreate
|
||||
30 5 TestProxiedServerEventsJournal
|
||||
30 5 TestProxiedServerPour
|
||||
30 5 TestProxiedServerServeUpdateClaim
|
||||
30 5 TestServerModeDeferAutoWake
|
||||
|
||||
30 6 TestProxiedServerBatchParity
|
||||
30 6 TestProxiedServerHistoryRemoteSupportedFrontDoorParity
|
||||
30 6 TestProxiedServerMolShow
|
||||
30 6 TestProxiedServerMultiStatementSQL
|
||||
30 6 TestProxiedServerReady
|
||||
30 6 TestProxiedServerServeRemoveDependency
|
||||
|
||||
30 7 TestProxiedServerCleanDatabasesPurgeDropped
|
||||
30 7 TestProxiedServerCount
|
||||
30 7 TestProxiedServerCountIncludeInfra
|
||||
30 7 TestProxiedServerDoltLifecycle
|
||||
30 7 TestProxiedServerGC
|
||||
30 7 TestProxiedServerUpdate
|
||||
|
||||
30 8 TestProxiedServerInitIfMissing
|
||||
30 8 TestProxiedServerPathHelpers
|
||||
30 8 TestProxiedServerReopenConcurrent
|
||||
30 8 TestProxiedServerShow2
|
||||
30 8 TestProxiedServerTypes
|
||||
30 8 TestProxiedServerUpdateClosePolicy
|
||||
|
||||
30 9 TestProxiedServerCloseConcurrent
|
||||
30 9 TestProxiedServerDep2
|
||||
30 9 TestProxiedServerListWatchCountsTheOffsetIntoMaxRows
|
||||
30 9 TestProxiedServerMolReadyGated
|
||||
30 9 TestProxiedServerSQL
|
||||
|
||||
30 10 TestProxiedServerAutoPrune
|
||||
30 10 TestProxiedServerClose2
|
||||
30 10 TestProxiedServerPing
|
||||
30 10 TestProxiedServerServeClose
|
||||
30 10 TestProxiedServerServeEdgeCounts
|
||||
|
||||
30 11 TestProxiedServerBatchDefersThenDoltCommitAdvancesHeadOnce
|
||||
30 11 TestProxiedServerReady2
|
||||
30 11 TestProxiedServerServeReadsTheJournalItJustWrote
|
||||
30 11 TestProxiedServerServeRefusesTheJournalWhenItIsDisabled
|
||||
30 11 TestProxiedServerShow
|
||||
|
||||
30 12 TestProxiedServerAssign
|
||||
30 12 TestProxiedServerConfigSetMany
|
||||
30 12 TestProxiedServerContext
|
||||
30 12 TestProxiedServerInitExplicitBeadsDirIsAuthoritative
|
||||
30 12 TestProxiedServerReadyClaimConcurrent
|
||||
30 12 TestProxiedServerServeBatchClose
|
||||
|
||||
30 13 TestProxiedServerChildren
|
||||
30 13 TestProxiedServerConfig
|
||||
30 13 TestProxiedServerDoltRemoteRemove
|
||||
30 13 TestProxiedServerExternalMutationPolicy
|
||||
30 13 TestProxiedServerPriority
|
||||
30 13 TestProxiedServerServeUpdate
|
||||
|
||||
30 14 TestProxiedServerInfo
|
||||
30 14 TestProxiedServerLint
|
||||
30 14 TestProxiedServerNote
|
||||
30 14 TestProxiedServerQuery
|
||||
30 14 TestProxiedServerUpdate3
|
||||
30 14 TestProxiedServerWispCreate
|
||||
|
||||
30 15 TestProxiedServerCompactDolt
|
||||
30 15 TestProxiedServerMolDistill
|
||||
30 15 TestProxiedServerMolProgress
|
||||
30 15 TestProxiedServerServeClaimNext
|
||||
30 15 TestProxiedServerServeRefusesAStreamOnADisabledJournal
|
||||
30 15 TestProxiedServerUnclaimIfAssigneeParity
|
||||
|
||||
30 16 TestProxiedServerCascadeParity
|
||||
30 16 TestProxiedServerExternalCreate
|
||||
30 16 TestProxiedServerGraph
|
||||
30 16 TestProxiedServerLabel
|
||||
30 16 TestProxiedServerSwarm
|
||||
30 16 TestProxiedServerTodo
|
||||
30 16 TestProxiedServerUpdateConcurrentMetadata
|
||||
|
||||
30 17 TestProxiedServerClose
|
||||
30 17 TestProxiedServerConfigConcurrent
|
||||
30 17 TestProxiedServerDeleteConcurrent
|
||||
30 17 TestProxiedServerSearch
|
||||
30 17 TestProxiedServerServeReadParity
|
||||
30 17 TestProxiedServerStatus
|
||||
|
||||
30 18 TestProxiedServerHeartbeat
|
||||
30 18 TestProxiedServerMolBond
|
||||
30 18 TestProxiedServerSQLDatabaseFlag
|
||||
30 18 TestProxiedServerServeCount
|
||||
30 18 TestProxiedServerUpdate2
|
||||
30 18 TestServerModeTypesAgreeWithCreate
|
||||
|
||||
30 19 TestProxiedServerCreate2
|
||||
30 19 TestProxiedServerEdit
|
||||
30 19 TestProxiedServerPourRejectsEnumViolatingVar
|
||||
30 19 TestProxiedServerReassignFence
|
||||
30 19 TestProxiedServerUpdateWisp
|
||||
30 19 TestServerModeServe
|
||||
|
||||
30 20 TestProxiedServerExternalPolicyDirectParity
|
||||
30 20 TestProxiedServerHistory
|
||||
30 20 TestProxiedServerImport
|
||||
30 20 TestProxiedServerMemory
|
||||
30 20 TestProxiedServerMolCurrent
|
||||
30 20 TestProxiedServerServeStreamsTheJournalItIsWriting
|
||||
|
||||
30 21 TestProxiedServerDeleteWisp
|
||||
30 21 TestProxiedServerDepConcurrent
|
||||
30 21 TestProxiedServerListSortRetiresTheWalk
|
||||
30 21 TestProxiedServerPrime
|
||||
30 21 TestProxiedServerRelate
|
||||
30 21 TestProxiedServerSetState
|
||||
30 21 TestProxiedServerUpdateConcurrentAppendNotes
|
||||
|
||||
30 22 TestProxiedServerCompactHistory
|
||||
30 22 TestProxiedServerGateLifecycle
|
||||
30 22 TestProxiedServerKV
|
||||
30 22 TestProxiedServerShow3
|
||||
30 22 TestProxiedServerStale
|
||||
30 22 TestProxiedServerStatuses
|
||||
|
||||
30 23 TestProxiedServerBackupRefusedOnExternalTopology
|
||||
30 23 TestProxiedServerCloseSiblingBlockersFanIn
|
||||
30 23 TestProxiedServerDeleteBlockedJSONSingleDoc
|
||||
30 23 TestProxiedServerQuick
|
||||
30 23 TestProxiedServerStorageClass
|
||||
30 23 TestProxiedServerUpdateHooks
|
||||
30 23 TestProxiedServerWispGC
|
||||
|
||||
30 24 TestProxiedServerCreateInfraTypeRoutesToWisps
|
||||
30 24 TestProxiedServerEpic
|
||||
30 24 TestProxiedServerLink
|
||||
30 24 TestProxiedServerMoleculeFrontDoorDirectServerProxyParity
|
||||
30 24 TestProxiedServerRecomputeBlocked
|
||||
30 24 TestServerModeServeSkipsPostRunMaintenance
|
||||
|
||||
30 25 TestProxiedServerDeferUndefer
|
||||
30 25 TestProxiedServerOrphans
|
||||
30 25 TestProxiedServerServeClaim
|
||||
30 25 TestProxiedServerShowWatch
|
||||
30 25 TestProxiedServerUpdateIfGuards
|
||||
30 25 TestProxiedServerWispCreateRejectsEnumViolatingVar
|
||||
|
||||
30 26 TestProxiedServerFindDuplicates
|
||||
30 26 TestProxiedServerGateCheck
|
||||
30 26 TestProxiedServerGateFrontDoorDirectServerProxyParity
|
||||
30 26 TestProxiedServerServeAddComment
|
||||
30 26 TestProxiedServerServeSettingsWrites
|
||||
30 26 TestProxiedServerState
|
||||
|
||||
30 27 TestProxiedServerBlocked
|
||||
30 27 TestProxiedServerMolLastActivity
|
||||
30 27 TestProxiedServerMolStale
|
||||
30 27 TestProxiedServerMoleculeFrontDoorWriteParity
|
||||
30 27 TestProxiedServerServeActivatesTheEventsJournal
|
||||
30 27 TestProxiedServerServeRefusesAStaleCheckpointWithGone
|
||||
|
||||
30 28 TestProxiedServerComment
|
||||
30 28 TestProxiedServerExport
|
||||
30 28 TestProxiedServerList
|
||||
30 28 TestProxiedServerReopen
|
||||
30 28 TestProxiedServerServeCreate
|
||||
30 28 TestProxiedServerUnclaim
|
||||
|
||||
30 29 TestProxiedServerDeferAutoWake
|
||||
30 29 TestProxiedServerDep
|
||||
30 29 TestProxiedServerDepListSingleIDWarnsOnDroppedExternalEdge
|
||||
30 29 TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
|
||||
30 29 TestProxiedServerTag
|
||||
30 29 TestProxiedServerWispList
|
||||
|
||||
30 30 TestProxiedServerEventsJournalOffByDefault
|
||||
30 30 TestProxiedServerMolSquash
|
||||
30 30 TestProxiedServerPrune
|
||||
30 30 TestProxiedServerPurgeWispsPlaneRetention
|
||||
30 30 TestProxiedServerServeRelease
|
||||
30 30 TestProxiedServerUpdateConcurrentClaim
|
||||
|
|
|
|||
62
.github/workflows/bazel.yml
vendored
62
.github/workflows/bazel.yml
vendored
|
|
@ -1023,20 +1023,24 @@ jobs:
|
|||
JOB_STATUS: ${{ job.status }}
|
||||
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)", as
|
||||
# --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the job's
|
||||
# shard script in 15 Bazel shards, so shard k runs exactly job k's tests.
|
||||
# Same build as --config=doltserver (race bd_test, non-race bd_for_tests),
|
||||
# shared with bazel-doltserver through the remote cache. PR Risk builds
|
||||
# its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd twin
|
||||
# uses build-artifacts' non-race bd-cmd-test, so this lane is the stricter
|
||||
# of the two. A job of its own, not gated by any integration switch: it
|
||||
# runs in pr.yml's call and its gate requires it (BAZEL_PROXIED).
|
||||
# Like pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)",
|
||||
# as --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the shard
|
||||
# script, so shard k runs exactly that manifest entry's tests. This lane
|
||||
# runs 30 shards of its own (.github/scripts/proxied-cmd-test-shards.txt's
|
||||
# duration-balanced 30-block; see scripts/ci/proxied_test_durations.json),
|
||||
# not the legacy jobs' 15 — shard k here is not the same split as their
|
||||
# shard k. Same build as --config=doltserver (race bd_test, non-race
|
||||
# bd_for_tests), shared with bazel-doltserver through the remote cache.
|
||||
# PR Risk builds its bd-cmd-test with -race, as here; main.yml's
|
||||
# test-proxied-cmd twin uses build-artifacts' non-race bd-cmd-test, so
|
||||
# this lane is the stricter of the two. A job of its own, not gated by
|
||||
# any integration switch: it runs in pr.yml's call and its gate requires
|
||||
# it (BAZEL_PROXIED).
|
||||
bazel-proxied:
|
||||
name: proxied-server tier
|
||||
# Remote execution only, like bazel-embedded: 15 shards of ~5-10
|
||||
# minutes, each with a dolt sql-server busy on ~4 cores, do not fit a
|
||||
# GitHub-hosted runner.
|
||||
# Remote execution only, like bazel-embedded: 30 concurrent shards, each
|
||||
# running its own dolt sql-server busy on ~4 cores, do not fit a
|
||||
# GitHub-hosted runner regardless of any single shard's wall time.
|
||||
# On same-repo pull_requests the legacy jobs stand down (pr-risk.yml's
|
||||
# bazel-coverage; D2 step 2), so this lane is then the tier's only PR
|
||||
# run and pr.yml's gate requires it to have run here and passed.
|
||||
|
|
@ -1079,8 +1083,36 @@ jobs:
|
|||
exit 1
|
||||
fi
|
||||
|
||||
# Remote: the longest shard takes ~5-8 minutes (each shard's Go
|
||||
# timeout is its job's 15m, 1200s Bazel cap), plus a compile.
|
||||
# Remote: each shard's Go timeout is its job's 15m, 1200s Bazel cap,
|
||||
# plus a compile. scripts/ci/proxied_test_durations.json's 176 entries
|
||||
# are each a clean, single-job (no concurrent heavy bazel/dolt job
|
||||
# intentionally started on the box; see that file's header for its
|
||||
# ambient-contention caveat), -test.parallel=4 top-level-test
|
||||
# duration — the exact config this job uses — from a single fresh,
|
||||
# manifest-aligned 30-shard sweep's own <testsuite time> (errors="0",
|
||||
# so a failing-but-complete suite's time is still trustworthy); every
|
||||
# one of the 176 discovered tests completed inside its shard run, so
|
||||
# no `bazel run` leaf-summation workaround was needed this time. See
|
||||
# that file's header for the full methodology, including why an
|
||||
# earlier -test.parallel=1 (serial) capture was replaced: serial
|
||||
# timings don't model this lane (which always runs parallel=4), and
|
||||
# an independent re-review measured near-zero correlation (r=0.08)
|
||||
# between the serial-derived predicted shard load and actual
|
||||
# parallel=4 wall time.
|
||||
#
|
||||
# Measured per-shard wall time for the full 30-shard local sweep that
|
||||
# produced this data: shards ranged from roughly 32s to 341s, with
|
||||
# most clustering well under this job's 25m step timeout / 30m job
|
||||
# timeout. After repacking with these corrected weights, the LPT
|
||||
# estimate clusters every shard within a few seconds of ~355-360s
|
||||
# (sum of all 176 measured durations / 30 shards) — no single test or
|
||||
# small group of tests dominates a shard the way the discarded serial
|
||||
# data wrongly implied. This is a local, single-job measurement, not
|
||||
# a verified RBE forecast (this job has only ever run remote); if a
|
||||
# shard's RBE wall time diverges sharply from this local estimate,
|
||||
# that is RBE throughput differing from local execution, not
|
||||
# necessarily a regression — compare against this job's own recent
|
||||
# run history before assuming a hang.
|
||||
- name: bazel test //... --config=doltserver-proxied
|
||||
id: test
|
||||
timeout-minutes: 25
|
||||
|
|
@ -1107,7 +1139,7 @@ jobs:
|
|||
run: >-
|
||||
python3 tools/bazel/check_shard_coverage.py
|
||||
--bep "$RUNNER_TEMP/bazel-bep.json"
|
||||
--suite //cmd/bd:bd_proxied_test .github/scripts/proxied-test-shard.sh 15
|
||||
--suite //cmd/bd:bd_proxied_test .github/scripts/proxied-test-shard.sh 30
|
||||
|
||||
- name: Upload test logs
|
||||
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
||||
|
|
|
|||
|
|
@ -1402,18 +1402,22 @@ sh_test(
|
|||
tags = ["embedded"],
|
||||
)
|
||||
|
||||
# bd_test as PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs run it: shard k
|
||||
# runs .github/scripts/proxied-test-shard.sh k+1 15 (the TestProxiedServer*
|
||||
# functions, assigned by the committed manifest) against this configuration's
|
||||
# Runs 30 Bazel shards against the TestProxiedServer*/TestServerMode*
|
||||
# functions, assigned by the manifest's 30-shard block (duration-balanced;
|
||||
# see scripts/ci/proxied_test_durations.json), against this configuration's
|
||||
# (race) bd_test with BEADS_TEST_PROXIED_SERVER=1 and the non-race
|
||||
# bd_for_tests as the subprocess binary, like the jobs' /tmp/bd-cmd-test and
|
||||
# /tmp/bd-proxied. (main.yml's twin job runs the same shard script against
|
||||
# build-artifacts' non-race bd-cmd-test; race here is the stricter one.)
|
||||
# The shared proxied harness gets its dolt sql-server from
|
||||
# internal/testutil's local backend (the pinned dolt in the test wrapper's
|
||||
# runfiles) instead of the jobs' container, and fails instead of skipping
|
||||
# when it cannot start one. It repeats bd_test's `env`. shard_count must
|
||||
# stay equal to the manifest's shard total.
|
||||
# bd_for_tests as the subprocess binary. This is a *different* shard count
|
||||
# from PR Risk's legacy fork/push jobs, "Test (Proxied Dolt Cmd N/15)"
|
||||
# (pr-risk.yml, main.yml): those keep running 15 shards of their own,
|
||||
# frozen bd-init-cost-proxy block in the same manifest file — shard k here
|
||||
# is not the same split as their shard k. PR Risk builds its bd-cmd-test
|
||||
# with -race, as here; main.yml's test-proxied-cmd twin uses
|
||||
# build-artifacts' non-race bd-cmd-test, so this lane is the stricter of
|
||||
# the two. The shared proxied harness gets its dolt sql-server from
|
||||
# internal/testutil's local backend (the pinned dolt in the test
|
||||
# wrapper's runfiles) instead of the jobs' container, and fails instead
|
||||
# of skipping when it cannot start one. It repeats bd_test's `env`.
|
||||
# shard_count must stay equal to the manifest block's shard total.
|
||||
#
|
||||
# Tags:
|
||||
# dolt-server-proxied: the proxied-server tier; starts hermetic dolt
|
||||
|
|
@ -1454,7 +1458,7 @@ sh_test(
|
|||
# 192-core worker).
|
||||
"GOMAXPROCS": "4",
|
||||
},
|
||||
shard_count = 15,
|
||||
shard_count = 30,
|
||||
tags = ["dolt-server-proxied"],
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import (
|
|||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
// TestAutoPruneProxiedServer is the auto-prune guard for bd's SECOND write
|
||||
// TestProxiedServerAutoPrune is the auto-prune guard for bd's SECOND write
|
||||
// plumbing.
|
||||
//
|
||||
// It is not a duplicate of the embedded test. Proxied mode opens no store at
|
||||
|
|
@ -20,7 +20,7 @@ import (
|
|||
// topology where the journal's writer is a short-lived CLI process against a
|
||||
// long-lived SQL server, so a missing trigger here means a workspace that
|
||||
// journals fast and never prunes.
|
||||
func TestAutoPruneProxiedServer(t *testing.T) {
|
||||
func TestProxiedServerAutoPrune(t *testing.T) {
|
||||
requireProxiedServerEnv(t)
|
||||
t.Parallel()
|
||||
|
||||
|
|
|
|||
|
|
@ -19,14 +19,14 @@ import (
|
|||
// transaction, and the read plumbing that pages them back out.
|
||||
//
|
||||
// It is the composition that has failed before (see the note on
|
||||
// TestServeActivatesTheEventsJournal): each half looked correct in isolation
|
||||
// TestProxiedServerServeActivatesTheEventsJournal): each half looked correct in isolation
|
||||
// while the server committed with an empty journal and every response looked
|
||||
// normal.
|
||||
|
||||
// TestServeReadsTheJournalItJustWrote: mutate over HTTP, then read the journal
|
||||
// TestProxiedServerServeReadsTheJournalItJustWrote: mutate over HTTP, then read the journal
|
||||
// over HTTP, and require the records to be there, gapless, and paced by a head
|
||||
// that matches.
|
||||
func TestServeReadsTheJournalItJustWrote(t *testing.T) {
|
||||
func TestProxiedServerServeReadsTheJournalItJustWrote(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
@ -138,7 +138,7 @@ func TestServeReadsTheJournalItJustWrote(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestServeRefusesAStaleCheckpointWithGone is the retention boundary over HTTP.
|
||||
// TestProxiedServerServeRefusesAStaleCheckpointWithGone is the retention boundary over HTTP.
|
||||
// A consumer whose checkpoint was pruned past must be TOLD, with the window the
|
||||
// server can still serve — not handed the surviving suffix as though it were a
|
||||
// complete history, and not handed an empty success it would read as "caught
|
||||
|
|
@ -147,7 +147,7 @@ func TestServeReadsTheJournalItJustWrote(t *testing.T) {
|
|||
// Both retention floors are zeroed so the prune can actually cut: the shipped
|
||||
// defaults (7 days / 100k rows) protect everything a test could create, which
|
||||
// is the point of them and the reason this has to say so explicitly.
|
||||
func TestServeRefusesAStaleCheckpointWithGone(t *testing.T) {
|
||||
func TestProxiedServerServeRefusesAStaleCheckpointWithGone(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
@ -214,12 +214,12 @@ func TestServeRefusesAStaleCheckpointWithGone(t *testing.T) {
|
|||
sp.shutdown(t)
|
||||
}
|
||||
|
||||
// TestServeRefusesTheJournalWhenItIsDisabled is the distinction the data cannot
|
||||
// TestProxiedServerServeRefusesTheJournalWhenItIsDisabled is the distinction the data cannot
|
||||
// make. This workspace never enabled the journal, so it records nothing and
|
||||
// never will — and the read has to say so rather than answer the empty page an
|
||||
// enabled-but-untouched workspace answers, which a consumer would poll against
|
||||
// forever.
|
||||
func TestServeRefusesTheJournalWhenItIsDisabled(t *testing.T) {
|
||||
func TestProxiedServerServeRefusesTheJournalWhenItIsDisabled(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
|
|||
|
|
@ -19,13 +19,13 @@ import (
|
|||
// that nothing was recorded. Nothing short of mutating and then reading back
|
||||
// can tell the difference.
|
||||
|
||||
// TestServeActivatesTheEventsJournal covers `bd serve` against a SERVER-MODE
|
||||
// TestProxiedServerServeActivatesTheEventsJournal covers `bd serve` against a SERVER-MODE
|
||||
// workspace, where PersistentPreRunE builds a DoltStore and no unit-of-work
|
||||
// provider at all — so serve constructs its own, from the workspace's Dolt
|
||||
// connection settings. That provider was never activated: every mutation the
|
||||
// HTTP surface accepted committed with an empty journal while /healthz stayed
|
||||
// green and every response looked normal.
|
||||
func TestServeActivatesTheEventsJournal(t *testing.T) {
|
||||
func TestProxiedServerServeActivatesTheEventsJournal(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ import (
|
|||
"github.com/steveyegge/beads/internal/storage"
|
||||
)
|
||||
|
||||
// TestEventsJournalProxiedServer is the end-to-end guard for bd's SECOND write
|
||||
// TestProxiedServerEventsJournal is the end-to-end guard for bd's SECOND write
|
||||
// plumbing. In proxied-server mode no store is ever opened: commands dispatch
|
||||
// through the unit-of-work provider, whose repositories reach the same issueops
|
||||
// emit helpers over their own pinned transactions.
|
||||
|
|
@ -24,7 +24,7 @@ import (
|
|||
// outside: every command succeeds, every write lands, and the journal is simply
|
||||
// empty. Only an end-to-end read can see it, which is why this test drives the
|
||||
// real binary against a real proxied server rather than asserting on wiring.
|
||||
func TestEventsJournalProxiedServer(t *testing.T) {
|
||||
func TestProxiedServerEventsJournal(t *testing.T) {
|
||||
requireProxiedServerEnv(t)
|
||||
t.Parallel()
|
||||
|
||||
|
|
@ -114,10 +114,10 @@ func TestEventsJournalProxiedServer(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestEventsJournalProxiedServerOffByDefault pins the default: a proxied
|
||||
// TestProxiedServerEventsJournalOffByDefault pins the default: a proxied
|
||||
// workspace that never asked for the journal writes no rows, so the feature
|
||||
// costs an ordinary team nothing.
|
||||
func TestEventsJournalProxiedServerOffByDefault(t *testing.T) {
|
||||
func TestProxiedServerEventsJournalOffByDefault(t *testing.T) {
|
||||
requireProxiedServerEnv(t)
|
||||
t.Parallel()
|
||||
|
||||
|
|
|
|||
|
|
@ -124,11 +124,11 @@ func (wc *watchConn) record(t *testing.T) (int64, map[string]any) {
|
|||
return id, rec
|
||||
}
|
||||
|
||||
// TestServeStreamsTheJournalItIsWriting: hold a stream open on one connection,
|
||||
// TestProxiedServerServeStreamsTheJournalItIsWriting: hold a stream open on one connection,
|
||||
// mutate on another, and require the records to arrive without anybody asking
|
||||
// for them — then drop the stream and resume it with Last-Event-ID, which is
|
||||
// the whole reconnection contract.
|
||||
func TestServeStreamsTheJournalItIsWriting(t *testing.T) {
|
||||
func TestProxiedServerServeStreamsTheJournalItIsWriting(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
@ -225,11 +225,11 @@ func TestServeStreamsTheJournalItIsWriting(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestServeRefusesAStreamOnADisabledJournal: the connect-time refusals are
|
||||
// TestProxiedServerServeRefusesAStreamOnADisabledJournal: the connect-time refusals are
|
||||
// ordinary problem+json responses on this operation too, and the stream is
|
||||
// never opened. A consumer that got a 200 here would hold a connection open
|
||||
// against a workspace that will never emit a record.
|
||||
func TestServeRefusesAStreamOnADisabledJournal(t *testing.T) {
|
||||
func TestProxiedServerServeRefusesAStreamOnADisabledJournal(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
|
|||
|
|
@ -22,11 +22,11 @@ import (
|
|||
"github.com/steveyegge/beads/internal/testutil"
|
||||
)
|
||||
|
||||
// TestHistoryRemoteRefusalFrontDoorMatrix runs the real bd binary through all
|
||||
// TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix runs the real bd binary through all
|
||||
// supported proxied transports. Valid command arguments are intentional: a
|
||||
// typed capability refusal must win over Cobra usage validation and must not
|
||||
// start a provider or touch durable state.
|
||||
func TestHistoryRemoteRefusalFrontDoorMatrix(t *testing.T) {
|
||||
func TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix(t *testing.T) {
|
||||
bd := buildEmbeddedBD(t)
|
||||
type fixture struct {
|
||||
name string
|
||||
|
|
@ -47,7 +47,7 @@ func TestHistoryRemoteRefusalFrontDoorMatrix(t *testing.T) {
|
|||
fixture{name: "external-unix", make: func(t *testing.T) proxiedProject {
|
||||
requireProxiedServerEnv(t)
|
||||
upstream := testutil.StartIsolatedDoltContainerHandle(t)
|
||||
socket := filepath.Join(t.TempDir(), "dolt.sock")
|
||||
socket := shortSocketPath(t, "dolt.sock")
|
||||
bridge := startHistoryUnixBridge(t, socket, upstream.Port)
|
||||
t.Cleanup(func() { _ = bridge.Process.Kill() })
|
||||
return bdProxiedInit(t, bd, "hm_unix", "--proxied-server-external-socket-path", socket)
|
||||
|
|
@ -339,11 +339,11 @@ func startHistoryUnixBridge(t *testing.T, endpoint, upstreamPort string) *exec.C
|
|||
}
|
||||
}
|
||||
|
||||
// TestHistoryRemoteSupportedFrontDoorParity checks the operations that are
|
||||
// TestProxiedServerHistoryRemoteSupportedFrontDoorParity checks the operations that are
|
||||
// intentionally shared by direct and proxied providers: history --events,
|
||||
// dolt remote remove, and dolt commit (the flush point dolt.auto-commit=batch/off
|
||||
// defers to, GH#4995).
|
||||
func TestHistoryRemoteSupportedFrontDoorParity(t *testing.T) {
|
||||
func TestProxiedServerHistoryRemoteSupportedFrontDoorParity(t *testing.T) {
|
||||
if os.Getenv("BEADS_TEST_PROXIED_SERVER") != "1" && os.Getenv(managedLocalProxiedEnvVar) != "1" {
|
||||
t.Skip("set a proxied test lane to run history parity")
|
||||
}
|
||||
|
|
|
|||
|
|
@ -26,6 +26,13 @@ import (
|
|||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// Deliberately not named TestProxiedServer...: this would make it discovered
|
||||
// by the shard scripts' ^Test(ProxiedServer|ServerMode) regex, but both its
|
||||
// subtests still fail a real "exceeded 3s bound" assertion against a stopped
|
||||
// external upstream (bead filed: outage command exceeds 3s bound against
|
||||
// stopped external upstream, tcp and unix). Renaming it into a required lane
|
||||
// before that's fixed would turn Bazel + the legacy 15-shard hash-fallback
|
||||
// red. Rename it once the underlying latency issue is resolved.
|
||||
func TestProxiedOutageReconnectAcceptanceMatrix(t *testing.T) {
|
||||
requireProxiedServerEnv(t)
|
||||
bd := buildEmbeddedBD(t)
|
||||
|
|
@ -45,7 +52,7 @@ func TestProxiedOutageReconnectAcceptanceMatrix(t *testing.T) {
|
|||
require.NoError(t, err)
|
||||
var endpoint string
|
||||
if topology.socket {
|
||||
endpoint = filepath.Join(t.TempDir(), "dolt.sock")
|
||||
endpoint = shortSocketPath(t, "dolt.sock")
|
||||
} else {
|
||||
endpoint = strconv.Itoa(gatePort)
|
||||
}
|
||||
|
|
@ -184,18 +191,12 @@ func runProxiedDeadline(t *testing.T, bd, dir string, timeout time.Duration, arg
|
|||
cmd.Dir, cmd.Env = dir, bdProxiedEnv(dir)
|
||||
var out, errOut strings.Builder
|
||||
cmd.Stdout, cmd.Stderr = &out, &errOut
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
if cmd.Process != nil {
|
||||
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
|
||||
}
|
||||
case <-done:
|
||||
}
|
||||
}()
|
||||
// CommandContext calls Cancel only after Start, so reading cmd.Process
|
||||
// here is ordered after Start's write to it -- unlike the watcher
|
||||
// goroutine this replaces, which raced Start/Run's write to cmd.Process
|
||||
// from a separate goroutine (caught by -race once the deadline expired).
|
||||
cmd.Cancel = func() error { return syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) }
|
||||
err = cmd.Run()
|
||||
close(done)
|
||||
return out.String(), errOut.String(), err, ctx.Err() != nil
|
||||
}
|
||||
|
||||
|
|
@ -238,3 +239,16 @@ func startOutageBridge(t *testing.T, endpoint, upstreamPort string, socket bool)
|
|||
})
|
||||
return cmd
|
||||
}
|
||||
|
||||
// shortSocketPath returns a unix-socket path under a short private directory:
|
||||
// t.TempDir() nests the (64-byte-truncated) test name under TMPDIR and the
|
||||
// suite root, which overflows sockaddr_un's 108-byte sun_path.
|
||||
func shortSocketPath(t *testing.T, name string) string {
|
||||
t.Helper()
|
||||
dir, err := os.MkdirTemp("", "bds")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = os.RemoveAll(dir) })
|
||||
return filepath.Join(dir, name)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -284,6 +284,16 @@ func TestServerModeServeSkipsPostRunMaintenance(t *testing.T) {
|
|||
// identity, so it naming the project database while every operation answers
|
||||
// from the global one is a lie with a straight face. Without the fix the
|
||||
// handshake and the startup line both report p.database here.
|
||||
//
|
||||
// Deliberately not named TestServerMode...: this would make it discovered by
|
||||
// the shard scripts' ^Test(ProxiedServer|ServerMode) regex, but it still
|
||||
// fails: `bd serve --global` refuses to auto-apply 4 pending schema
|
||||
// migrations (v65 to v69) to a shared/proxied server database and exits 1
|
||||
// with a consent-required usage message instead of starting (bead filed:
|
||||
// shared-server global DB created at schema v65, serve --global refuses the
|
||||
// #5920 migration guard). Renaming it into a required lane before that's
|
||||
// resolved would turn Bazel + the legacy 15-shard hash-fallback + main.yml's
|
||||
// push job red. Rename it once the guard interaction is resolved.
|
||||
func TestSharedServerModeServeGlobalReportsTheServedDatabase(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
|
|
|
|||
|
|
@ -212,7 +212,7 @@ func TestEmbeddedCreateStorageClass(t *testing.T) {
|
|||
// `bd create --file` used to accept --storage-class and the per-type config
|
||||
// default and honor neither. buildMarkdownBatchRequest is the ONE projection
|
||||
// both transports use, so these subtests cover the direct half of that fix
|
||||
// and TestStorageClassProxiedServer covers the other.
|
||||
// and TestProxiedServerStorageClass covers the other.
|
||||
t.Run("markdown_batch", func(t *testing.T) {
|
||||
batch := `## Batch one
|
||||
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ import (
|
|||
// CI note: this shard is gated on BEADS_TEST_PROXIED_SERVER=1 and runs mostly
|
||||
// only on push-to-main, so it is meant to be run locally alongside any change
|
||||
// to the create input path.
|
||||
func TestStorageClassProxiedServer(t *testing.T) {
|
||||
func TestProxiedServerStorageClass(t *testing.T) {
|
||||
requireSharedProxiedServer(t)
|
||||
t.Parallel()
|
||||
|
||||
|
|
|
|||
|
|
@ -948,7 +948,9 @@ manifests. On those PRs they are the tiers' only pre-merge run, and
|
|||
- `tools/bazel/check_shard_coverage.py` runs after each tier. It requires:
|
||||
- every Bazel shard of `//cmd/bd:bd_embedded_test`,
|
||||
`//internal/storage/embeddeddolt:embeddeddolt_embedded_test`,
|
||||
`//cmd/bd:bd_proxied_test` (15) and
|
||||
`//cmd/bd:bd_proxied_test` (30; PR Risk's own legacy
|
||||
`test-proxied-cmd` fork/push jobs still run 15 shards of their own,
|
||||
frozen manifest block — a different split, not this one) and
|
||||
`//internal/storage/dolt:dolt_server_full_test` (16) to have run
|
||||
exactly the tests its shard script lists (list-only mode, minus
|
||||
`TestMain`, which `grep '^func Test'` lists but which is never a
|
||||
|
|
|
|||
|
|
@ -111,8 +111,13 @@ proxied-server cmd/bd tier ("Test (Proxied Dolt Cmd N/15)",
|
|||
`//cmd/bd:bd_proxied_test`), and `--config=doltserver-integration` the
|
||||
server-Dolt storage tier ("Test (Server Dolt Conformance)", "Test (Server
|
||||
Dolt Full Suite N/16)", `//internal/storage/dolt:dolt_server_*_test`), which
|
||||
builds with the integration tag like `--config=integration`. Each shard
|
||||
runs its CI job's shard script, so Bazel shard k runs the tests of job k+1.
|
||||
builds with the integration tag like `--config=integration`. Each shard runs
|
||||
its CI job's shard script, so for `--config=doltserver-integration` Bazel
|
||||
shard k runs the tests of job k+1 (both split the manifest's 16-shard block
|
||||
the same way). `--config=doltserver-proxied`'s `bd_proxied_test` instead
|
||||
runs the manifest's own 30-shard block — bin-packed by measured duration,
|
||||
not the legacy jobs' 15-shard, bd-init-cost-proxy block — so shard k there
|
||||
is not job k+1's tests; it is a different split of the same tests.
|
||||
|
||||
An ambient `BEADS_DOLT_SERVER_PORT` or `BEADS_DOLT_PORT` is never honored by
|
||||
the suites that call `testutil.EnsureDoltContainerForTestMain`. When a test
|
||||
|
|
|
|||
|
|
@ -5,82 +5,396 @@ Discovery must stay in step with proxied-test-shard.sh: both select the
|
|||
TestProxiedServer* and TestServerMode* entry points, which share the
|
||||
requireSharedProxiedServer gate and the shared dolt sql-server.
|
||||
|
||||
Cost proxy for each such top-level function is its bd-init count
|
||||
(newSharedProxiedProject occurrences) — inits dominate proxied wall-time. Tests
|
||||
are packed into TOTAL shards longest-processing-time-first so the heaviest shard
|
||||
is minimized. Prints the manifest to stdout.
|
||||
Two cost models, selected by --weights (default: inits, unchanged from
|
||||
before this flag existed — the committed 15-shard block for the legacy
|
||||
pr-risk.yml/main.yml jobs was generated this way and is NOT regenerated by
|
||||
this script's --weights=duration path; it stays frozen so those jobs' shard
|
||||
assignment keeps its exact current behavior):
|
||||
|
||||
Usage: gen_proxied_shard_manifest.py [total_shards]
|
||||
--weights=inits (default): bd-init count (newSharedProxiedProject
|
||||
occurrences) per top-level function — a cheap static proxy for
|
||||
wall-time that undercounts a test whose cost is dominated by what it
|
||||
does per init rather than how many inits it runs (see the two
|
||||
known-expensive entries in the committed 15-shard manifest's header).
|
||||
|
||||
--weights=duration: measured wall-time from
|
||||
scripts/ci/proxied_test_durations.json (that file's header explains its
|
||||
provenance and limits). A function missing from that file — a test
|
||||
added since it was last captured — falls back to its inits cost times
|
||||
the file's "seconds_per_init_fallback" ratio, so a brand-new test still
|
||||
gets a plausible seconds-shaped weight instead of a unit-mismatched one.
|
||||
|
||||
Either way, a from-scratch pack puts TOTAL shards' worth of tests together
|
||||
longest-processing-time-first (heaviest first onto the currently lightest
|
||||
shard) so the heaviest shard's total is minimized.
|
||||
|
||||
--write is INCREMENTAL by default: it keeps every existing assignment in the
|
||||
requested total_shards block as-is, drops names that no longer exist (a
|
||||
test renamed, split or deleted since the block was last written), and LPT-
|
||||
places only the newly-discovered names onto whichever shards are currently
|
||||
lightest (by the requested --weights' cost model) — it never moves a test
|
||||
that is already assigned. This means two unrelated PRs that each add one new
|
||||
proxied test do not generate a textual conflict with each other (each only
|
||||
appends its own new test's line) and do not silently invalidate each other's
|
||||
packing. Pass --repack to instead throw away the existing block and do a
|
||||
full from-scratch LPT pack, exactly like earlier versions of this script
|
||||
always did — intentionally rebalancing every assignment, so expect large,
|
||||
unrelated-looking diffs when you do this on purpose.
|
||||
|
||||
--check verifies exact-once NAME coverage only: every currently-discovered
|
||||
TestProxiedServer*/TestServerMode* function is listed in the requested
|
||||
total_shards block exactly once, with no stale (no-longer-existing) or
|
||||
duplicate entries. It does NOT verify that the block matches a fresh
|
||||
re-pack's shard *assignments* — an incremental --write intentionally
|
||||
diverges from a from-scratch repack over time, and reweighting
|
||||
proxied_test_durations.json no longer by itself fails --check (only adding,
|
||||
renaming, or removing a test does). If you need to confirm the packing is
|
||||
still well-balanced after a lot of incremental drift, run with --repack
|
||||
and read the stderr shard-load report, or pass --check anyway: it still
|
||||
reports the block's current loads to stderr even though it only gates on
|
||||
name coverage.
|
||||
|
||||
The manifest file holds more than one block (one per total_shards value in
|
||||
use): the legacy, frozen 15-shard block pr-risk.yml/main.yml's jobs read, and
|
||||
the Bazel-only lane's own block (30 shards as of F2). By default this prints
|
||||
only the requested total's block to stdout, which is NOT safe to redirect
|
||||
straight into the manifest file (`gen... 30 > file` deletes every other
|
||||
block). Use --write to update the file in place instead: it replaces only
|
||||
the block for the requested total_shards and leaves every other block
|
||||
untouched. Use --check to verify the committed block for a total_shards
|
||||
has exact-once name coverage, without writing anything — this is the form a
|
||||
CI policy check should call.
|
||||
|
||||
Usage: gen_proxied_shard_manifest.py [total_shards] [--weights=inits|duration]
|
||||
[--write [--repack] | --check] [--manifest PATH]
|
||||
"""
|
||||
import argparse
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
TOTAL = int(sys.argv[1]) if len(sys.argv) > 1 else 15
|
||||
# Relative to the repo root, like discover_inits_cost's cmd/bd/*_test.go glob:
|
||||
# this script must be run from (or with --manifest pointed relative to) the
|
||||
# repo root, matching every other caller (the shell shard scripts, CI).
|
||||
default_manifest_path = '.github/scripts/proxied-cmd-test-shards.txt'
|
||||
|
||||
func_re = re.compile(r'^func (Test(?:ProxiedServer|ServerMode)[A-Za-z0-9_]+)\(')
|
||||
|
||||
costs = {}
|
||||
for path in glob.glob('cmd/bd/*_test.go'):
|
||||
with open(path) as fh:
|
||||
lines = fh.readlines()
|
||||
cur = None
|
||||
for ln in lines:
|
||||
m = func_re.match(ln)
|
||||
if m:
|
||||
cur = m.group(1)
|
||||
costs.setdefault(cur, 0)
|
||||
elif ln.startswith('func '):
|
||||
cur = None
|
||||
if cur:
|
||||
costs[cur] += ln.count('newSharedProxiedProject')
|
||||
|
||||
# Every discovered test costs at least 1 (setup/teardown) even with no inits.
|
||||
for k in costs:
|
||||
costs[k] = max(costs[k], 1)
|
||||
def discover_inits_cost():
|
||||
"""Return {test_name: bd-init count}, floored at 1 (setup/teardown)."""
|
||||
costs = {}
|
||||
for path in glob.glob('cmd/bd/*_test.go'):
|
||||
with open(path) as fh:
|
||||
lines = fh.readlines()
|
||||
cur = None
|
||||
for ln in lines:
|
||||
m = func_re.match(ln)
|
||||
if m:
|
||||
cur = m.group(1)
|
||||
costs.setdefault(cur, 0)
|
||||
elif ln.startswith('func '):
|
||||
cur = None
|
||||
if cur:
|
||||
costs[cur] += ln.count('newSharedProxiedProject')
|
||||
for k in costs:
|
||||
costs[k] = max(costs[k], 1)
|
||||
return costs
|
||||
|
||||
# LPT bin-packing: heaviest first onto the currently-lightest shard.
|
||||
order = sorted(costs, key=lambda k: (-costs[k], k))
|
||||
shards = [[] for _ in range(TOTAL)]
|
||||
loads = [0] * TOTAL
|
||||
for name in order:
|
||||
i = loads.index(min(loads))
|
||||
shards[i].append(name)
|
||||
loads[i] += costs[name]
|
||||
|
||||
out = []
|
||||
out.append('# Proxied-server cmd test shard manifest.')
|
||||
out.append('#')
|
||||
out.append('# Format: <total_shards> <shard_number> <top_level_test_name>')
|
||||
out.append('#')
|
||||
out.append(f'# {TOTAL}-shard split, bin-packed longest-processing-time-first by')
|
||||
out.append('# estimated cost (bd-init count; per-init migration chains dominate).')
|
||||
out.append('# Regenerate with scripts/ci/gen_proxied_shard_manifest.py after adding,')
|
||||
out.append('# splitting, or reweighting TestProxiedServer*/TestServerMode*')
|
||||
out.append('# functions. Newly-added tests not listed here hash-distribute via')
|
||||
out.append('# proxied-test-shard.sh.')
|
||||
out.append('#')
|
||||
out.append('# This file is generated: notes added here are erased by the next')
|
||||
out.append('# regeneration. Add them to the generator instead. Known blind spots')
|
||||
out.append('# of the cost proxy (it counts inits, not the work done per init):')
|
||||
out.append('#')
|
||||
out.append('# TestProxiedServerListSortRetiresTheWalk is the most expensive')
|
||||
out.append('# entry and one of the cheapest-looking: off a single init it seeds')
|
||||
out.append('# 1400 rows and walks the listing three ways. It is the measurement')
|
||||
out.append('# behind the claim that sort=priority retires the client-side walk,')
|
||||
out.append('# so it has to run on PRs.')
|
||||
out.append('#')
|
||||
out.append('# TestProxiedServerServeReadParity is the CLI/HTTP read parity')
|
||||
out.append('# oracle, the behavioural half of the anti-drift claim. It being in')
|
||||
out.append('# this lane, and this lane being required by the PR Risk CI gate,')
|
||||
out.append('# is what makes that claim enforced on PRs rather than asserted in')
|
||||
out.append('# a doc comment.')
|
||||
out.append('')
|
||||
for i in range(TOTAL):
|
||||
for name in sorted(shards[i]):
|
||||
out.append(f'{TOTAL} {i + 1} {name}')
|
||||
def duration_cost(inits):
|
||||
"""Return {test_name: seconds}, from the committed duration capture,
|
||||
falling back to inits * seconds_per_init_fallback for an undated test."""
|
||||
durations_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'proxied_test_durations.json')
|
||||
with open(durations_path) as fh:
|
||||
data = json.load(fh)
|
||||
measured = data['tests']
|
||||
fallback_ratio = data['seconds_per_init_fallback']
|
||||
stale = sorted(name for name in measured if name not in inits)
|
||||
if stale:
|
||||
sys.stderr.write(
|
||||
f'warning: proxied_test_durations.json has {len(stale)} capture(s) for '
|
||||
'tests that no longer exist in cmd/bd/*_test.go (renamed, split or '
|
||||
f'deleted since the last capture): {", ".join(stale)}\n')
|
||||
costs = {}
|
||||
for name, cost in inits.items():
|
||||
costs[name] = measured[name] if name in measured else cost * fallback_ratio
|
||||
costs[name] = max(costs[name], 0.001)
|
||||
return costs
|
||||
|
||||
|
||||
def pack(costs, total):
|
||||
"""LPT bin-pack costs into `total` shards. Returns (shards, loads)."""
|
||||
order = sorted(costs, key=lambda k: (-costs[k], k))
|
||||
shards = [[] for _ in range(total)]
|
||||
loads = [0.0] * total
|
||||
for name in order:
|
||||
i = loads.index(min(loads))
|
||||
shards[i].append(name)
|
||||
loads[i] += costs[name]
|
||||
return shards, loads
|
||||
|
||||
|
||||
def read_assignments(path, total):
|
||||
"""Return {name: shard_number} for path's block matching total_shards ==
|
||||
total, or None if path or that block does not exist yet."""
|
||||
if not os.path.exists(path):
|
||||
return None
|
||||
for b in split_blocks(open(path).read()):
|
||||
if b['total'] == total:
|
||||
assignments = {}
|
||||
for ln in b['body']:
|
||||
m = re.match(r'^(\d+) (\d+) (\S+)$', ln)
|
||||
if m and int(m.group(1)) == total:
|
||||
assignments[m.group(3)] = int(m.group(2))
|
||||
return assignments
|
||||
return None
|
||||
|
||||
|
||||
def incremental_pack(costs, total, existing):
|
||||
"""Keep every name in `existing` that is still in `costs` on its current
|
||||
shard; drop names no longer in `costs` (renamed/split/deleted); LPT-place
|
||||
every name in `costs` not already in `existing` onto whichever shard is
|
||||
currently lightest. Never moves an already-assigned test. Returns
|
||||
(shards, loads), like pack()."""
|
||||
shards = [[] for _ in range(total)]
|
||||
loads = [0.0] * total
|
||||
for name, shard_num in existing.items():
|
||||
if name not in costs or not (1 <= shard_num <= total):
|
||||
continue # stale, or out of range for a changed total: re-place below
|
||||
shards[shard_num - 1].append(name)
|
||||
loads[shard_num - 1] += costs[name]
|
||||
placed = {n for s in shards for n in s}
|
||||
new_names = sorted((n for n in costs if n not in placed), key=lambda k: (-costs[k], k))
|
||||
for name in new_names:
|
||||
i = loads.index(min(loads))
|
||||
shards[i].append(name)
|
||||
loads[i] += costs[name]
|
||||
return shards, loads
|
||||
|
||||
|
||||
def render(total, shards, weights):
|
||||
out = []
|
||||
out.append('# Proxied-server cmd test shard manifest.')
|
||||
out.append('#')
|
||||
out.append('# Format: <total_shards> <shard_number> <top_level_test_name>')
|
||||
out.append('#')
|
||||
if weights == 'inits':
|
||||
out.append(f'# {total}-shard split, bin-packed longest-processing-time-first by')
|
||||
out.append('# estimated cost (bd-init count; per-init migration chains dominate).')
|
||||
out.append(f'# Regenerating it (scripts/ci/gen_proxied_shard_manifest.py {total} --write)')
|
||||
out.append('# is deliberately NOT part of normal maintenance; see the frozen note')
|
||||
out.append('# below. Newly-added tests not listed here hash-distribute via')
|
||||
out.append('# proxied-test-shard.sh: cksum(name) % total.')
|
||||
out.append('#')
|
||||
out.append('# This file is generated: notes added here are erased by the next')
|
||||
out.append('# regeneration. Add them to the generator instead. Known blind spots')
|
||||
out.append('# of the cost proxy (it counts inits, not the work done per init):')
|
||||
out.append('#')
|
||||
out.append('# TestProxiedServerListSortRetiresTheWalk is the most expensive')
|
||||
out.append('# entry and one of the cheapest-looking: off a single init it seeds')
|
||||
out.append('# 1400 rows and walks the listing three ways. It is the measurement')
|
||||
out.append('# behind the claim that sort=priority retires the client-side walk,')
|
||||
out.append('# so it has to run on PRs.')
|
||||
out.append('#')
|
||||
out.append('# TestProxiedServerServeReadParity is the CLI/HTTP read parity')
|
||||
out.append('# oracle, the behavioural half of the anti-drift claim. It being in')
|
||||
out.append('# this lane, and this lane being required by the PR Risk CI gate,')
|
||||
out.append('# is what makes that claim enforced on PRs rather than asserted in')
|
||||
out.append('# a doc comment.')
|
||||
out.append('#')
|
||||
out.append(f'# This {total}-shard block is frozen for pr-risk.yml/main.yml\'s legacy')
|
||||
out.append('# fork/push jobs (see engdocs/TESTING.md): do not regenerate it even')
|
||||
out.append('# after adding tests (they hash-distribute until a maintainer chooses')
|
||||
out.append('# to reweight on purpose). --check does not cover this block; only the')
|
||||
out.append('# Bazel-only --weights=duration block is checked in CI.')
|
||||
else:
|
||||
out.append(f'# {total}-shard split for the Bazel-only proxied-server tier')
|
||||
out.append('# (bazel-proxied in .github/workflows/bazel.yml), bin-packed')
|
||||
out.append('# longest-processing-time-first by measured wall-time from')
|
||||
out.append('# scripts/ci/proxied_test_durations.json (see that file for')
|
||||
out.append('# provenance and its "relative weight, not absolute SLA" caveat).')
|
||||
out.append(f'# Regenerate with scripts/ci/gen_proxied_shard_manifest.py {total}')
|
||||
out.append('# --weights=duration --write after adding or removing')
|
||||
out.append('# TestProxiedServer*/TestServerMode* functions. --write is incremental:')
|
||||
out.append('# it keeps every already-assigned test on its current shard and only')
|
||||
out.append('# places newly-discovered names, onto whichever shard is currently')
|
||||
out.append('# lightest, so two unrelated PRs that each add one test do not textually')
|
||||
out.append('# conflict or silently invalidate each other\'s packing. Pass --repack to')
|
||||
out.append('# force a full from-scratch rebalance instead (a deliberate, separate')
|
||||
out.append('# change — expect a large diff). --check only verifies every discovered')
|
||||
out.append('# name is listed here exactly once (no stale or duplicate entries); it')
|
||||
out.append('# does NOT verify the packing is still well-balanced, so a reweighted')
|
||||
out.append('# proxied_test_durations.json does not by itself fail --check — rerun')
|
||||
out.append('# with --repack periodically (or whenever shard loads look uneven in CI)')
|
||||
out.append('# to rebalance on purpose. Newly-added tests not listed here')
|
||||
out.append('# hash-distribute via proxied-test-shard.sh: cksum(name) % total,')
|
||||
out.append('# unrelated to proxied_test_durations.json (that file only feeds this')
|
||||
out.append('# generator\'s own cost estimate for a test it has not measured yet).')
|
||||
out.append('#')
|
||||
out.append('# This file is generated: notes added here are erased by the next')
|
||||
out.append('# regeneration. Add them to the generator or the durations file')
|
||||
out.append('# instead.')
|
||||
out.append('')
|
||||
for i in range(total):
|
||||
for name in sorted(shards[i]):
|
||||
out.append(f'{total} {i + 1} {name}')
|
||||
out.append('')
|
||||
return out
|
||||
|
||||
sys.stderr.write('shard loads (est. inits): ' +
|
||||
', '.join(f'{i + 1}:{loads[i]}' for i in range(TOTAL)) + '\n')
|
||||
sys.stderr.write(f'heaviest shard: {max(loads)} inits\n')
|
||||
print('\n'.join(out).rstrip() + '\n', end='')
|
||||
|
||||
def split_blocks(text):
|
||||
"""Split a manifest file's text into its per-total_shards blocks.
|
||||
|
||||
A block is a run of leading '#' comment (and blank spacer) lines,
|
||||
immediately followed by a run of lines that do not start with '#' (shard
|
||||
assignment lines and the blank lines render() puts between shards) that
|
||||
ends at the next '#' line or end of file. Returns a list of dicts with
|
||||
'header' and 'body' line lists and the block's 'total' (parsed from its
|
||||
first assignment line; None for a block with no assignment lines yet).
|
||||
"""
|
||||
lines = text.split('\n')
|
||||
blocks = []
|
||||
i, n = 0, len(lines)
|
||||
while i < n:
|
||||
header = []
|
||||
while i < n and lines[i].startswith('#'):
|
||||
header.append(lines[i])
|
||||
i += 1
|
||||
while i < n and lines[i] == '' and not (i + 1 < n and lines[i + 1].startswith('#')):
|
||||
header.append(lines[i])
|
||||
i += 1
|
||||
body = []
|
||||
while i < n and not lines[i].startswith('#'):
|
||||
body.append(lines[i])
|
||||
i += 1
|
||||
if not header and not body:
|
||||
continue
|
||||
total = None
|
||||
for bl in body:
|
||||
m = re.match(r'^(\d+) ', bl)
|
||||
if m:
|
||||
total = int(m.group(1))
|
||||
break
|
||||
blocks.append({'header': header, 'body': body, 'total': total})
|
||||
return blocks
|
||||
|
||||
|
||||
def write_block(path, total, out):
|
||||
"""Replace path's block for total_shards == total with out (a render()
|
||||
list), preserving every other block and its position. Appends out as a
|
||||
new trailing block if path has none for this total yet."""
|
||||
text = open(path).read() if os.path.exists(path) else ''
|
||||
blocks = split_blocks(text) if text else []
|
||||
result = []
|
||||
replaced = False
|
||||
for b in blocks:
|
||||
if b['total'] == total:
|
||||
result.extend(out)
|
||||
replaced = True
|
||||
else:
|
||||
result.extend(b['header'])
|
||||
result.extend(b['body'])
|
||||
if not replaced:
|
||||
if result and result[-1] != '':
|
||||
result.append('')
|
||||
result.extend(out)
|
||||
with open(path, 'w') as fh:
|
||||
fh.write('\n'.join(result).rstrip('\n') + '\n')
|
||||
|
||||
|
||||
def check_coverage(path, total, universe):
|
||||
"""Return None if path's committed block for total_shards == total lists
|
||||
every name in `universe` (the currently-discovered test set) exactly
|
||||
once, with no stale or duplicate names; otherwise an actionable error
|
||||
string. Does NOT check the block's shard *assignments* — see this
|
||||
script's module docstring for why (incremental --write)."""
|
||||
if not os.path.exists(path):
|
||||
return f'{path} does not exist'
|
||||
for b in split_blocks(open(path).read()):
|
||||
if b['total'] == total:
|
||||
names = []
|
||||
for ln in b['body']:
|
||||
m = re.match(r'^(\d+) (\d+) (\S+)$', ln)
|
||||
if m and int(m.group(1)) == total:
|
||||
names.append(m.group(3))
|
||||
universe_set = set(universe)
|
||||
counts = {}
|
||||
for n in names:
|
||||
counts[n] = counts.get(n, 0) + 1
|
||||
dupes = sorted(n for n, c in counts.items() if c > 1)
|
||||
stale = sorted(n for n in counts if n not in universe_set)
|
||||
missing = sorted(n for n in universe_set if n not in counts)
|
||||
if not dupes and not stale and not missing:
|
||||
return None
|
||||
weights_guess = 'duration' if any('duration' in h for h in b['header']) else 'inits'
|
||||
msg = [f"{path}'s {total}-shard block does not have exact-once "
|
||||
'coverage of the currently-discovered TestProxiedServer*/'
|
||||
'TestServerMode* test set.']
|
||||
if missing:
|
||||
msg.append(f'missing ({len(missing)}): {", ".join(missing)}')
|
||||
if stale:
|
||||
msg.append(f'stale/unknown ({len(stale)}): {", ".join(stale)}')
|
||||
if dupes:
|
||||
msg.append(f'duplicated ({len(dupes)}): {", ".join(dupes)}')
|
||||
msg.append(f'Run: python3 {sys.argv[0]} {total} --weights={weights_guess} --write')
|
||||
return '\n'.join(msg)
|
||||
return f'{path} has no block for total_shards={total}'
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.split('\n\n')[0])
|
||||
ap.add_argument('total_shards', nargs='?', type=int, default=15)
|
||||
ap.add_argument('--weights', choices=['inits', 'duration'], default='inits')
|
||||
ap.add_argument('--manifest', default=default_manifest_path,
|
||||
help=f'manifest file to read/write (default: {default_manifest_path})')
|
||||
mode = ap.add_mutually_exclusive_group()
|
||||
mode.add_argument('--write', action='store_true',
|
||||
help='rewrite only this total_shards block in --manifest in place '
|
||||
'(incremental by default; see --repack)')
|
||||
mode.add_argument('--check', action='store_true',
|
||||
help="exit non-zero unless --manifest's block for this total_shards has "
|
||||
'exact-once coverage of the discovered test set (does not write '
|
||||
'anything, and does not check shard assignments — see --repack to '
|
||||
'preview those)')
|
||||
ap.add_argument('--repack', action='store_true',
|
||||
help='ignore any existing block for this total_shards and do a full '
|
||||
'from-scratch LPT pack, instead of the default incremental placement '
|
||||
'that keeps every already-assigned test on its current shard')
|
||||
args = ap.parse_args()
|
||||
|
||||
inits = discover_inits_cost()
|
||||
costs = inits if args.weights == 'inits' else duration_cost(inits)
|
||||
|
||||
existing = None if args.repack else read_assignments(args.manifest, args.total_shards)
|
||||
if existing:
|
||||
shards, loads = incremental_pack(costs, args.total_shards, existing)
|
||||
else:
|
||||
shards, loads = pack(costs, args.total_shards)
|
||||
out = render(args.total_shards, shards, args.weights)
|
||||
|
||||
unit = 'inits' if args.weights == 'inits' else 's'
|
||||
fmt = (lambda v: str(v)) if args.weights == 'inits' else (lambda v: f'{v:.1f}')
|
||||
mode_label = 'repack' if (args.repack or not existing) else 'incremental'
|
||||
sys.stderr.write(f'shard loads ({mode_label}, est. {unit}, weights={args.weights}): ' +
|
||||
', '.join(f'{i + 1}:{fmt(loads[i])}' for i in range(args.total_shards)) + '\n')
|
||||
sys.stderr.write(f'heaviest shard: {fmt(max(loads))} {unit}\n')
|
||||
|
||||
if args.check:
|
||||
err = check_coverage(args.manifest, args.total_shards, costs.keys())
|
||||
if err:
|
||||
sys.stderr.write('error: ' + err + '\n')
|
||||
sys.exit(1)
|
||||
return
|
||||
if args.write:
|
||||
write_block(args.manifest, args.total_shards, out)
|
||||
sys.stderr.write(f'wrote {args.manifest}\'s {args.total_shards}-shard block '
|
||||
f'({mode_label})\n')
|
||||
return
|
||||
print('\n'.join(out).rstrip() + '\n', end='')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
|
|
|||
233
scripts/ci/proxied_test_durations.json
Normal file
233
scripts/ci/proxied_test_durations.json
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
{
|
||||
"_comment": [
|
||||
"Measured locally with `bazel --nohome_rc test --config=doltserver-proxied",
|
||||
"//cmd/bd:bd_proxied_test` -- the exact config bazel.yml's proxied-server tier",
|
||||
"uses, including `-test.parallel=4` (local sandboxed execution only; no remote",
|
||||
"executor or cache flags were configured for this measurement run; Bazel's own",
|
||||
"--remote_download_regex etc. in that config do not enable remote execution",
|
||||
"without --config=remote-exec, which was not passed).",
|
||||
"",
|
||||
"This replaces an earlier capture that used `-test.parallel=1` (serial). That",
|
||||
"methodology avoided double-counting a parallel subtest group's own duration on",
|
||||
"top of its children's, which was a genuine bug in an even earlier pass -- but",
|
||||
"serial timings do not model this lane, which always runs at parallel=4, and an",
|
||||
"independent re-review measured near-zero correlation (r=0.08) between the",
|
||||
"serial-derived predicted shard load and actual parallel=4 wall time. These",
|
||||
"values are each a clean, single-job (no other concurrent heavy bazel/dolt job",
|
||||
"intentionally started on the box; see below), parallel=4 top-level test's own",
|
||||
"<testsuite time=...> attribute from a single fresh, manifest-aligned 30-shard",
|
||||
"sweep where every shard ran its exact assigned test set once (no duplicates,",
|
||||
"no gaps -- see scripts/ci/gen_proxied_shard_manifest.py's exact-once coverage",
|
||||
"check). A top-level Go test's <testsuite time> already reflects, not adds to,",
|
||||
"its own parallel children's wall-clock time, so no leaf-summation step was",
|
||||
"needed this time: every one of the 176 discovered tests completed inside a",
|
||||
"single shard run with errors=\"0\" (none needed Bazel's per-target default",
|
||||
"timeout worked around via a separate `bazel run`, unlike the prior capture).",
|
||||
"",
|
||||
"`seconds_per_init_fallback` is the weighted average seconds-per-init over",
|
||||
"every measured test in this file (sum of all durations / sum of all inits),",
|
||||
"used only for a brand-new test with no entry yet (see",
|
||||
"scripts/ci/gen_proxied_shard_manifest.py's discover_inits_cost()). It dropped",
|
||||
"from the serial capture's 39.7275 to this capture's lower value because",
|
||||
"parallel=4 genuinely shortens most tests' wall time relative to inits count;",
|
||||
"this is expected, not a data error.",
|
||||
"",
|
||||
"This file still does not claim shard totals land within a few seconds of each",
|
||||
"other -- see .github/workflows/bazel.yml for the current, honest expected",
|
||||
"spread for the 30-shard proxied lane, now also derived from this same sweep.",
|
||||
"",
|
||||
"The box this was measured on also runs other engineers'/agents' unrelated",
|
||||
"Bazel jobs; this capture was not run with exclusive use of the machine, only",
|
||||
"with no other *intentional, task-related* heavy bazel/dolt job started",
|
||||
"concurrently by this task. Absolute seconds may be inflated by ambient",
|
||||
"contention; relative ordering between tests, which is what LPT packing",
|
||||
"actually depends on, should still be far more representative than the",
|
||||
"discarded serial data (confirmed: 30/30 shards passed, 0 failures, 0 races,",
|
||||
"with the two blocker fixes -- shortSocketPath and cmd.Cancel -- applied).",
|
||||
"",
|
||||
"Regenerate with a local `bazel --nohome_rc test --config=doltserver-proxied",
|
||||
"//cmd/bd:bd_proxied_test --build_event_json_file=<path>` sweep, then pull",
|
||||
"each shard's bazel-testlogs test.xml <testsuite time> per top-level test --",
|
||||
"not from downloaded CI artifacts -- so the parallel=4, CI-representative",
|
||||
"methodology above is applied consistently."
|
||||
],
|
||||
"seconds_per_init_fallback": 17.1946,
|
||||
"tests": {
|
||||
"TestProxiedServerAssign": 38.67,
|
||||
"TestProxiedServerAutoPrune": 67.5,
|
||||
"TestProxiedServerBackupRefusedOnExternalTopology": 5.96,
|
||||
"TestProxiedServerBatchDefersThenDoltCommitAdvancesHeadOnce": 9.18,
|
||||
"TestProxiedServerBatchParity": 60.04,
|
||||
"TestProxiedServerBlocked": 138.63,
|
||||
"TestProxiedServerCascadeParity": 78.23,
|
||||
"TestProxiedServerChildren": 11.28,
|
||||
"TestProxiedServerCleanDatabases": 7.14,
|
||||
"TestProxiedServerCleanDatabasesPurgeDropped": 7.76,
|
||||
"TestProxiedServerClose": 174.6,
|
||||
"TestProxiedServerClose2": 218.83,
|
||||
"TestProxiedServerCloseConcurrent": 38.25,
|
||||
"TestProxiedServerCloseSiblingBlockersFanIn": 99.71,
|
||||
"TestProxiedServerComment": 64.04,
|
||||
"TestProxiedServerCompactDolt": 11.55,
|
||||
"TestProxiedServerCompactHistory": 12.38,
|
||||
"TestProxiedServerConfig": 188.91,
|
||||
"TestProxiedServerConfigConcurrent": 48.59,
|
||||
"TestProxiedServerConfigSetMany": 196.57,
|
||||
"TestProxiedServerContext": 72.27,
|
||||
"TestProxiedServerCount": 60.65,
|
||||
"TestProxiedServerCountIncludeInfra": 34.69,
|
||||
"TestProxiedServerCreate": 46.42,
|
||||
"TestProxiedServerCreate2": 165.07,
|
||||
"TestProxiedServerCreateInfraTypeRoutesToWisps": 142.17,
|
||||
"TestProxiedServerDeferAutoWake": 12.95,
|
||||
"TestProxiedServerDeferUndefer": 51.14,
|
||||
"TestProxiedServerDelete": 44.82,
|
||||
"TestProxiedServerDeleteBlockedJSONSingleDoc": 8.09,
|
||||
"TestProxiedServerDeleteConcurrent": 10.57,
|
||||
"TestProxiedServerDeleteForceOrphans": 9.16,
|
||||
"TestProxiedServerDeleteWisp": 7.06,
|
||||
"TestProxiedServerDep": 121.62,
|
||||
"TestProxiedServerDep2": 227.28,
|
||||
"TestProxiedServerDepConcurrent": 30.55,
|
||||
"TestProxiedServerDepListSingleIDWarnsOnDroppedExternalEdge": 10.13,
|
||||
"TestProxiedServerDoltLifecycle": 6.18,
|
||||
"TestProxiedServerDoltRemoteRemove": 72.67,
|
||||
"TestProxiedServerDuplicates": 44.77,
|
||||
"TestProxiedServerEdit": 10.16,
|
||||
"TestProxiedServerEpic": 109.49,
|
||||
"TestProxiedServerEventsJournal": 6.48,
|
||||
"TestProxiedServerEventsJournalOffByDefault": 65.83,
|
||||
"TestProxiedServerExport": 26.44,
|
||||
"TestProxiedServerExternalCreate": 5.73,
|
||||
"TestProxiedServerExternalMutationPolicy": 25.0,
|
||||
"TestProxiedServerExternalPolicyDirectParity": 18.12,
|
||||
"TestProxiedServerFindDuplicates": 51.66,
|
||||
"TestProxiedServerGC": 12.73,
|
||||
"TestProxiedServerGateCheck": 140.99,
|
||||
"TestProxiedServerGateCreateInheritsRepoMetadata": 47.1,
|
||||
"TestProxiedServerGateFrontDoorDirectServerProxyParity": 17.36,
|
||||
"TestProxiedServerGateLifecycle": 49.79,
|
||||
"TestProxiedServerGateList": 41.19,
|
||||
"TestProxiedServerGraph": 15.43,
|
||||
"TestProxiedServerHeartbeat": 57.23,
|
||||
"TestProxiedServerHistory": 21.81,
|
||||
"TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix": 59.61,
|
||||
"TestProxiedServerHistoryRemoteSupportedFrontDoorParity": 5.97,
|
||||
"TestProxiedServerHuman": 263.77,
|
||||
"TestProxiedServerImport": 160.85,
|
||||
"TestProxiedServerInfo": 28.32,
|
||||
"TestProxiedServerInitExplicitBeadsDirIsAuthoritative": 11.99,
|
||||
"TestProxiedServerInitIfMissing": 6.69,
|
||||
"TestProxiedServerKV": 94.24,
|
||||
"TestProxiedServerLabel": 179.25,
|
||||
"TestProxiedServerLink": 51.49,
|
||||
"TestProxiedServerLint": 15.51,
|
||||
"TestProxiedServerList": 112.68,
|
||||
"TestProxiedServerListComments": 31.13,
|
||||
"TestProxiedServerListSortRetiresTheWalk": 46.29,
|
||||
"TestProxiedServerListWatchCountsTheOffsetIntoMaxRows": 10.54,
|
||||
"TestProxiedServerMemory": 58.87,
|
||||
"TestProxiedServerMolBond": 79.34,
|
||||
"TestProxiedServerMolBurn": 253.65,
|
||||
"TestProxiedServerMolCurrent": 85.1,
|
||||
"TestProxiedServerMolDistill": 78.03,
|
||||
"TestProxiedServerMolLastActivity": 111.33,
|
||||
"TestProxiedServerMolProgress": 182.51,
|
||||
"TestProxiedServerMolReadyGated": 66.34,
|
||||
"TestProxiedServerMolShow": 36.15,
|
||||
"TestProxiedServerMolSquash": 118.23,
|
||||
"TestProxiedServerMolStale": 53.95,
|
||||
"TestProxiedServerMoleculeFrontDoorDirectServerProxyParity": 22.98,
|
||||
"TestProxiedServerMoleculeFrontDoorWriteParity": 19.52,
|
||||
"TestProxiedServerMultiStatementSQL": 6.76,
|
||||
"TestProxiedServerNote": 11.21,
|
||||
"TestProxiedServerOrphans": 12.11,
|
||||
"TestProxiedServerPathHelpers": 0.0,
|
||||
"TestProxiedServerPing": 11.73,
|
||||
"TestProxiedServerPour": 253.65,
|
||||
"TestProxiedServerPourRejectsEnumViolatingVar": 21.66,
|
||||
"TestProxiedServerPrime": 94.24,
|
||||
"TestProxiedServerPriority": 41.35,
|
||||
"TestProxiedServerPromote": 255.67,
|
||||
"TestProxiedServerPrune": 119.06,
|
||||
"TestProxiedServerPurgeWispsPlaneRetention": 24.55,
|
||||
"TestProxiedServerQuery": 42.94,
|
||||
"TestProxiedServerQuick": 22.7,
|
||||
"TestProxiedServerReady": 238.66,
|
||||
"TestProxiedServerReady2": 71.96,
|
||||
"TestProxiedServerReadyClaimConcurrent": 20.86,
|
||||
"TestProxiedServerReadyConcurrent": 11.34,
|
||||
"TestProxiedServerReadyWisp": 255.67,
|
||||
"TestProxiedServerReassignFence": 57.62,
|
||||
"TestProxiedServerReclaim": 11.62,
|
||||
"TestProxiedServerRecomputeBlocked": 19.71,
|
||||
"TestProxiedServerRelate": 160.07,
|
||||
"TestProxiedServerReopen": 125.1,
|
||||
"TestProxiedServerReopenConcurrent": 66.34,
|
||||
"TestProxiedServerSQL": 13.91,
|
||||
"TestProxiedServerSQLDatabaseFlag": 12.28,
|
||||
"TestProxiedServerSQLStatementRouting": 18.36,
|
||||
"TestProxiedServerSearch": 78.35,
|
||||
"TestProxiedServerServeActivatesTheEventsJournal": 9.87,
|
||||
"TestProxiedServerServeAddComment": 25.29,
|
||||
"TestProxiedServerServeAddDependencies": 15.59,
|
||||
"TestProxiedServerServeBatchClose": 16.3,
|
||||
"TestProxiedServerServeClaim": 16.32,
|
||||
"TestProxiedServerServeClaimNext": 17.04,
|
||||
"TestProxiedServerServeClose": 38.42,
|
||||
"TestProxiedServerServeCount": 21.49,
|
||||
"TestProxiedServerServeCreate": 11.84,
|
||||
"TestProxiedServerServeDelete": 13.83,
|
||||
"TestProxiedServerServeEdgeCounts": 20.14,
|
||||
"TestProxiedServerServeLifecycle": 21.66,
|
||||
"TestProxiedServerServeModeGate": 31.59,
|
||||
"TestProxiedServerServeReadParity": 15.44,
|
||||
"TestProxiedServerServeReadRevision": 10.4,
|
||||
"TestProxiedServerServeReadsTheJournalItJustWrote": 38.39,
|
||||
"TestProxiedServerServeRefusesAStaleCheckpointWithGone": 22.47,
|
||||
"TestProxiedServerServeRefusesAStreamOnADisabledJournal": 42.35,
|
||||
"TestProxiedServerServeRefusesTheJournalWhenItIsDisabled": 19.4,
|
||||
"TestProxiedServerServeRelatedIssues": 28.93,
|
||||
"TestProxiedServerServeRelease": 11.29,
|
||||
"TestProxiedServerServeRemoveDependency": 12.52,
|
||||
"TestProxiedServerServeSettingsWrites": 10.68,
|
||||
"TestProxiedServerServeStreamsTheJournalItIsWriting": 11.89,
|
||||
"TestProxiedServerServeUpdate": 17.56,
|
||||
"TestProxiedServerServeUpdateClaim": 13.57,
|
||||
"TestProxiedServerSetState": 15.28,
|
||||
"TestProxiedServerShow": 216.16,
|
||||
"TestProxiedServerShow2": 232.1,
|
||||
"TestProxiedServerShow3": 157.69,
|
||||
"TestProxiedServerShowWatch": 109.69,
|
||||
"TestProxiedServerStale": 15.61,
|
||||
"TestProxiedServerState": 110.26,
|
||||
"TestProxiedServerStatus": 28.61,
|
||||
"TestProxiedServerStatuses": 27.0,
|
||||
"TestProxiedServerStorageClass": 150.78,
|
||||
"TestProxiedServerSwarm": 29.89,
|
||||
"TestProxiedServerTag": 33.46,
|
||||
"TestProxiedServerTodo": 43.18,
|
||||
"TestProxiedServerTypes": 12.51,
|
||||
"TestProxiedServerTypesAgreeWithCreate": 14.34,
|
||||
"TestProxiedServerUnclaim": 16.57,
|
||||
"TestProxiedServerUnclaimIfAssigneeParity": 25.33,
|
||||
"TestProxiedServerUpdate": 238.19,
|
||||
"TestProxiedServerUpdate2": 170.45,
|
||||
"TestProxiedServerUpdate3": 184.1,
|
||||
"TestProxiedServerUpdateClosePolicy": 37.4,
|
||||
"TestProxiedServerUpdateConcurrentAppendNotes": 6.6,
|
||||
"TestProxiedServerUpdateConcurrentClaim": 17.68,
|
||||
"TestProxiedServerUpdateConcurrentMetadata": 8.68,
|
||||
"TestProxiedServerUpdateHooks": 20.14,
|
||||
"TestProxiedServerUpdateIfGuards": 141.97,
|
||||
"TestProxiedServerUpdateWisp": 82.23,
|
||||
"TestProxiedServerWispCreate": 74.62,
|
||||
"TestProxiedServerWispCreateRejectsEnumViolatingVar": 25.49,
|
||||
"TestProxiedServerWispGC": 52.72,
|
||||
"TestProxiedServerWispList": 118.23,
|
||||
"TestServerModeDeferAutoWake": 32.77,
|
||||
"TestServerModeServe": 19.28,
|
||||
"TestServerModeServeSkipsPostRunMaintenance": 10.52,
|
||||
"TestServerModeTypesAgreeWithCreate": 16.0
|
||||
}
|
||||
}
|
||||
|
|
@ -3321,6 +3321,34 @@ func bazelRuleBlock(build, name string) string {
|
|||
return build[start : i+end+3]
|
||||
}
|
||||
|
||||
// bazelProxiedShardCount returns cmd/bd:bd_proxied_test's own shard_count
|
||||
// from cmd/bd/BUILD.bazel: the single source of truth for the Bazel-only
|
||||
// bazel-proxied lane's shard split, which no longer has to equal PR
|
||||
// Risk's/main.yml's legacy test-proxied-cmd jobs' matrix size (F2). Under
|
||||
// `bazel test`, scripts_test's runfiles hold no other package's BUILD file,
|
||||
// so this falls back to the literal the structural checks below pin under
|
||||
// plain `go test` (TestBazelDoltServerTiersMirrorPRRisk and
|
||||
// TestBazelRetiredLanesCheckListedTestsRan both fail if cmd/bd/BUILD.bazel's
|
||||
// shard_count ever drifts from this fallback).
|
||||
func bazelProxiedShardCount(t *testing.T) int {
|
||||
t.Helper()
|
||||
const bazelTestFallback = 30
|
||||
if os.Getenv("TEST_SRCDIR") != "" {
|
||||
return bazelTestFallback
|
||||
}
|
||||
root := sourceRepoRoot(t)
|
||||
rule := bazelRuleBlock(readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "bd_proxied_test")
|
||||
m := regexp.MustCompile(`(?m)^ shard_count = (\d+),$`).FindStringSubmatch(rule)
|
||||
if m == nil {
|
||||
t.Fatalf("cmd/bd:bd_proxied_test has no `shard_count = N,` in cmd/bd/BUILD.bazel:\n%s", rule)
|
||||
}
|
||||
n, err := strconv.Atoi(m[1])
|
||||
if err != nil {
|
||||
t.Fatalf("cmd/bd:bd_proxied_test shard_count: %v", err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// bazel-integration mirrors main.yml's integration jobs with the unmodified
|
||||
// --config=integration (whose .bazelrc filter and flags
|
||||
// TestBazelrcIntegrationLane pins): no extra filter, selector or remote
|
||||
|
|
@ -3521,15 +3549,38 @@ func TestBazelDoltServerTiersMirrorPRRisk(t *testing.T) {
|
|||
|
||||
type shardTier struct {
|
||||
workflow, job, step, script, binVar, pkg, target, env string
|
||||
// noBuildShardPin is true when this tier's target's BUILD.bazel
|
||||
// shard_count is allowed to differ from this workflow/job's own
|
||||
// matrix size, because a separate Bazel-only lane shards that same
|
||||
// target on its own manifest block (F2). Reading the expected value
|
||||
// back out of the same BUILD.bazel file we are about to check would
|
||||
// be tautological, so these tiers skip the `shard_count = N,`
|
||||
// sub-assertion below entirely; the real cross-file pin for them
|
||||
// lives in pr_risk_bazel_coverage_test.go, which ties BUILD.bazel's
|
||||
// shard_count to bazel.yml's check_shard_coverage.py argument.
|
||||
noBuildShardPin bool
|
||||
}
|
||||
tiers := []shardTier{
|
||||
{"pr-risk.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
|
||||
{"main.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
|
||||
{"pr-risk.yml", "test-server-storage-full", "Test", ".github/scripts/server-storage-test-shard.sh", "BEADS_TEST_SERVER_TEST_BINARY", "internal/storage/dolt", "dolt_server_full_test", "BEADS_TEST_ENV_RUN_DOLT"},
|
||||
// pr-risk.yml/main.yml's own "Test (Proxied Dolt Cmd N/15)" matrix
|
||||
// stays 15 (the legacy, frozen bd-init-cost-proxy manifest block),
|
||||
// but cmd/bd:bd_proxied_test's shard_count need not match: the
|
||||
// Bazel-only bazel-proxied job (bazel.yml) runs that target with its
|
||||
// own duration-balanced manifest block, not these jobs'. Shard k in
|
||||
// one split is not shard k in the other; see cmd/bd/BUILD.bazel's
|
||||
// bd_proxied_test comment and bazel.yml's bazel-proxied comment. Both
|
||||
// matrices are still required (below, after this loop) to equal each
|
||||
// other and the legacy manifest block's total.
|
||||
{"pr-risk.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER", true},
|
||||
{"main.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER", true},
|
||||
{"pr-risk.yml", "test-server-storage-full", "Test", ".github/scripts/server-storage-test-shard.sh", "BEADS_TEST_SERVER_TEST_BINARY", "internal/storage/dolt", "dolt_server_full_test", "BEADS_TEST_ENV_RUN_DOLT", false},
|
||||
}
|
||||
proxiedMatrixLen := map[string]int{}
|
||||
for _, c := range tiers {
|
||||
j := readCIWorkflow(t, c.workflow).job(t, c.job)
|
||||
shards := len(j.Strategy.Matrix.Shard)
|
||||
if c.target == "bd_proxied_test" {
|
||||
proxiedMatrixLen[c.workflow] = shards
|
||||
}
|
||||
step := j.step(t, c.step)
|
||||
if want := "bash " + c.script + " ${{ matrix.shard }} " + strconv.Itoa(shards); strings.TrimSpace(step.Run) != want {
|
||||
t.Errorf("%s %s runs %q, want %q", c.workflow, c.job, step.Run, want)
|
||||
|
|
@ -3540,17 +3591,21 @@ func TestBazelDoltServerTiersMirrorPRRisk(t *testing.T) {
|
|||
if os.Getenv("TEST_SRCDIR") != "" {
|
||||
continue // scripts_test's runfiles hold no other package's BUILD
|
||||
}
|
||||
buildShards := shards
|
||||
root := sourceRepoRoot(t)
|
||||
rule := bazelRuleBlock(readPolicyFile(t, root, c.pkg+"/BUILD.bazel"), c.target)
|
||||
for _, want := range []string{
|
||||
want := []string{
|
||||
`srcs = ["//tools/bazel:go_test_manifest_shard.sh"],`,
|
||||
`"$(rootpath //:` + c.script + `)",`,
|
||||
`"` + c.binVar + `",`,
|
||||
"shard_count = " + strconv.Itoa(shards) + ",",
|
||||
`timeout = "eternal",`,
|
||||
} {
|
||||
if !strings.Contains(rule, want) {
|
||||
t.Errorf("%s:%s does not contain %q (%s %s):\n%s", c.pkg, c.target, want, c.workflow, c.job, rule)
|
||||
}
|
||||
if !c.noBuildShardPin {
|
||||
want = append(want, "shard_count = "+strconv.Itoa(buildShards)+",")
|
||||
}
|
||||
for _, w := range want {
|
||||
if !strings.Contains(rule, w) {
|
||||
t.Errorf("%s:%s does not contain %q (%s %s):\n%s", c.pkg, c.target, w, c.workflow, c.job, rule)
|
||||
}
|
||||
}
|
||||
mm := shardManifestDefault.FindStringSubmatch(readPolicyFile(t, root, c.script))
|
||||
|
|
@ -3564,6 +3619,13 @@ func TestBazelDoltServerTiersMirrorPRRisk(t *testing.T) {
|
|||
}
|
||||
}
|
||||
}
|
||||
// S1: pr-risk.yml's and main.yml's legacy test-proxied-cmd matrices must
|
||||
// stay equal to each other (they share one frozen manifest block); a
|
||||
// divergence must fail here even though cmd/bd:bd_proxied_test's own
|
||||
// shard_count is no longer pinned to either of them (above).
|
||||
if a, b := proxiedMatrixLen["pr-risk.yml"], proxiedMatrixLen["main.yml"]; a != b {
|
||||
t.Errorf("pr-risk.yml test-proxied-cmd has %d shards, main.yml has %d; these share one legacy manifest block and must match", a, b)
|
||||
}
|
||||
|
||||
conf := risk.job(t, "test-server-storage").step(t, "Test").Run
|
||||
quoted := regexp.MustCompile(`(-test\.[a-z]+) '([^']*)'|(-test\.[a-z]+=\S+|-test\.v)`)
|
||||
|
|
|
|||
|
|
@ -985,28 +985,61 @@ func copyMap(m map[string]string) map[string]string {
|
|||
// unsharded targets did not only skip.
|
||||
func TestBazelRetiredLanesCheckListedTestsRan(t *testing.T) {
|
||||
risk := readCIWorkflow(t, prRiskWorkflowName)
|
||||
type suite struct{ job, step, label, script string }
|
||||
type suite struct {
|
||||
job, step, label, script string
|
||||
// shardCount overrides the expected check_shard_coverage.py shard
|
||||
// count for this suite when the Bazel lane's own manifest block runs
|
||||
// a different number of shards than this PR Risk job's matrix. Zero
|
||||
// means "same as this suite's PR Risk job matrix size" (the common
|
||||
// case for a lane that is a drop-in retirement of the legacy job).
|
||||
// -1 means "read cmd/bd:bd_proxied_test's own shard_count from
|
||||
// cmd/bd/BUILD.bazel" (bazelProxiedShardCount); only label ==
|
||||
// "//cmd/bd:bd_proxied_test" may use -1 (enforced below), so
|
||||
// copying this onto another suite needs a reviewed change there.
|
||||
shardCount int
|
||||
}
|
||||
for _, c := range []struct {
|
||||
lane, config string
|
||||
suites []suite
|
||||
whole []string
|
||||
}{
|
||||
{bazelEmbedJobName, "embedded", []suite{
|
||||
{"test-embedded-cmd", "Test", "//cmd/bd:bd_embedded_test", ".github/scripts/embedded-test-shard.sh"},
|
||||
{"test-embedded-storage", "Test", "//internal/storage/embeddeddolt:embeddeddolt_embedded_test", ".github/scripts/embedded-storage-test-shard.sh"},
|
||||
{"test-embedded-cmd", "Test", "//cmd/bd:bd_embedded_test", ".github/scripts/embedded-test-shard.sh", 0},
|
||||
{"test-embedded-storage", "Test", "//internal/storage/embeddeddolt:embeddeddolt_embedded_test", ".github/scripts/embedded-storage-test-shard.sh", 0},
|
||||
}, []string{"//internal/storage/embeddeddolt:embeddeddolt_conformance_core_test", "//internal/storage/embeddeddolt:embeddeddolt_conformance_audit_test"}},
|
||||
{bazelProxiedJobName, "doltserver-proxied", []suite{
|
||||
{"test-proxied-cmd", "Test proxied-server cmd shard", "//cmd/bd:bd_proxied_test", ".github/scripts/proxied-test-shard.sh"},
|
||||
// bazel-proxied runs its own duration-balanced manifest block
|
||||
// (scripts/ci/proxied_test_durations.json), not PR Risk's frozen
|
||||
// 15-shard bd-init-cost-proxy block: it is not a drop-in
|
||||
// retirement of test-proxied-cmd's shard count, just its tests.
|
||||
// bazelProxiedShardCount reads the real count from
|
||||
// cmd/bd/BUILD.bazel, making this suite's `want` (below) a
|
||||
// genuine cross-file pin against bazel.yml's own
|
||||
// check_shard_coverage.py argument, not an independently
|
||||
// hard-coded literal that could drift from BUILD.bazel unnoticed
|
||||
// (S1). This -1 sentinel is deliberately restricted to
|
||||
// bd_proxied_test below: copying it onto another suite requires
|
||||
// touching that guard, not just this literal.
|
||||
{"test-proxied-cmd", "Test proxied-server cmd shard", "//cmd/bd:bd_proxied_test", ".github/scripts/proxied-test-shard.sh", -1},
|
||||
}, nil},
|
||||
{bazelServerJobName, "doltserver-integration", []suite{
|
||||
{"test-server-storage-full", "Test", "//internal/storage/dolt:dolt_server_full_test", ".github/scripts/server-storage-test-shard.sh"},
|
||||
{"test-server-storage-full", "Test", "//internal/storage/dolt:dolt_server_full_test", ".github/scripts/server-storage-test-shard.sh", 0},
|
||||
}, []string{"//internal/storage/dolt:dolt_server_conformance_test"}},
|
||||
} {
|
||||
job := readCIWorkflow(t, bazelWorkflowName).job(t, c.lane)
|
||||
step := job.step(t, "Every listed test ran in its shard")
|
||||
want := []string{"python3 tools/bazel/check_shard_coverage.py", `--bep "$RUNNER_TEMP/bazel-bep.json"`}
|
||||
for _, s := range c.suites {
|
||||
shards := len(risk.job(t, s.job).Strategy.Matrix.Shard)
|
||||
shards := s.shardCount
|
||||
switch {
|
||||
case shards < 0:
|
||||
if s.label != "//cmd/bd:bd_proxied_test" {
|
||||
t.Fatalf("%s: shardCount<0 (read live from BUILD.bazel) is only allowed for //cmd/bd:bd_proxied_test, not %s", s.job, s.label)
|
||||
}
|
||||
shards = bazelProxiedShardCount(t)
|
||||
case shards == 0:
|
||||
shards = len(risk.job(t, s.job).Strategy.Matrix.Shard)
|
||||
}
|
||||
if !strings.Contains(risk.job(t, s.job).step(t, s.step).Run, s.script) {
|
||||
t.Errorf("pr-risk.yml %s no longer runs %s; update this suite", s.job, s.script)
|
||||
}
|
||||
|
|
@ -1246,45 +1279,103 @@ func TestShardScriptsListOnlyRealTests(t *testing.T) {
|
|||
for _, d := range regexp.MustCompile(`(?m)^func (Test\w*)\(\w+ \*testing\.T\) \{`).FindAllStringSubmatch(src.String(), -1) {
|
||||
declared[d[1]] = true
|
||||
}
|
||||
shards := len(risk.job(t, c.job).Strategy.Matrix.Shard)
|
||||
// The scripts' hash fallback forks per test (seconds per shard for
|
||||
// the server suite): list the shards concurrently.
|
||||
outs, errs := make([][]byte, shards+1), make([]error, shards+1)
|
||||
var wg sync.WaitGroup
|
||||
for k := 1; k <= shards; k++ {
|
||||
wg.Add(1)
|
||||
go func(k int) {
|
||||
defer wg.Done()
|
||||
cmd := exec.Command("bash", c.script, strconv.Itoa(k), strconv.Itoa(shards))
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "BEADS_TEST_SHARD_LIST_ONLY=1")
|
||||
outs[k], errs[k] = cmd.Output()
|
||||
}(k)
|
||||
|
||||
// B1: validate every total this script's committed manifest holds a
|
||||
// block for, plus this job's own PR Risk matrix size and (for
|
||||
// test-proxied-cmd) the Bazel lane's own shard_count — not just
|
||||
// whichever total happens to equal this PR Risk job's matrix. Before
|
||||
// F2 those always coincided; now the Bazel-only bazel-proxied lane
|
||||
// reads a manifest block (30) that no PR-Risk-matrix-only check ever
|
||||
// exercises, so a fork PR (which never runs bazel-proxied) could
|
||||
// corrupt that block and still merge green. Looping over every
|
||||
// distinct total in the manifest closes that gap for this script and
|
||||
// any other script that later grows a second block the same way.
|
||||
mm := shardManifestDefault.FindStringSubmatch(readPolicyFile(t, root, c.script))
|
||||
if mm == nil {
|
||||
t.Fatalf("%s has no ${BEADS_TEST_SHARD_MANIFEST:-...} default manifest", c.script)
|
||||
}
|
||||
wg.Wait()
|
||||
listed := 0
|
||||
for k := 1; k <= shards; k++ {
|
||||
if errs[k] != nil {
|
||||
t.Fatalf("%s %d %d: %v", c.script, k, shards, errs[k])
|
||||
totalsSet := map[int]bool{len(risk.job(t, c.job).Strategy.Matrix.Shard): true}
|
||||
for _, line := range strings.Split(readPolicyFile(t, root, mm[1]), "\n") {
|
||||
line, _, _ = strings.Cut(line, "#")
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) == 0 {
|
||||
continue
|
||||
}
|
||||
for _, line := range strings.Split(string(outs[k]), "\n") {
|
||||
name, ok := strings.CutPrefix(line, " ")
|
||||
if !ok || !strings.HasPrefix(name, "Test") || strings.ContainsAny(name, " :") {
|
||||
continue
|
||||
}
|
||||
listed++
|
||||
isTest := declared[name]
|
||||
switch {
|
||||
case notTests[name] && isTest:
|
||||
t.Errorf("%s shard %d lists %s, which check_shard_coverage.py drops, but it is a real test", c.script, k, name)
|
||||
case !notTests[name] && !isTest:
|
||||
t.Errorf("%s shard %d lists %s, which is not a `func %s(t *testing.T)` test in %s: check_shard_coverage.py would report it missing on every run (add it to NOT_TESTS only if go test never runs it)",
|
||||
c.script, k, name, name, c.pkg)
|
||||
}
|
||||
if n, err := strconv.Atoi(fields[0]); err == nil {
|
||||
totalsSet[n] = true
|
||||
}
|
||||
}
|
||||
if listed < 50 {
|
||||
t.Errorf("%s listed only %d tests over %d shards; did the list-only output format change?", c.script, listed, shards)
|
||||
if c.script == ".github/scripts/proxied-test-shard.sh" {
|
||||
totalsSet[bazelProxiedShardCount(t)] = true
|
||||
}
|
||||
totals := make([]int, 0, len(totalsSet))
|
||||
for n := range totalsSet {
|
||||
totals = append(totals, n)
|
||||
}
|
||||
sort.Ints(totals)
|
||||
|
||||
for _, shards := range totals {
|
||||
// The scripts' hash fallback forks per test (seconds per shard
|
||||
// for the server suite): list the shards concurrently.
|
||||
outs, errs := make([][]byte, shards+1), make([]error, shards+1)
|
||||
var wg sync.WaitGroup
|
||||
for k := 1; k <= shards; k++ {
|
||||
wg.Add(1)
|
||||
go func(k int) {
|
||||
defer wg.Done()
|
||||
cmd := exec.Command("bash", c.script, strconv.Itoa(k), strconv.Itoa(shards))
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "BEADS_TEST_SHARD_LIST_ONLY=1")
|
||||
outs[k], errs[k] = cmd.Output()
|
||||
}(k)
|
||||
}
|
||||
wg.Wait()
|
||||
listed, manifestSum := 0, 0
|
||||
for k := 1; k <= shards; k++ {
|
||||
if errs[k] != nil {
|
||||
// The script itself exits 1 on a duplicate manifest entry
|
||||
// or a manifest entry that names no discovered test
|
||||
// (rename/typo/stale-after-delete), for the requested
|
||||
// total only: this is what catches a corrupted block
|
||||
// that a PR-Risk-matrix-only check at a different total
|
||||
// would never see.
|
||||
t.Fatalf("%s %d %d: %v\n%s", c.script, k, shards, errs[k], outs[k])
|
||||
}
|
||||
for _, line := range strings.Split(string(outs[k]), "\n") {
|
||||
if mc := regexp.MustCompile(`^ manifest: (\d+), fallback: \d+$`).FindStringSubmatch(line); mc != nil {
|
||||
n, _ := strconv.Atoi(mc[1])
|
||||
manifestSum += n
|
||||
continue
|
||||
}
|
||||
name, ok := strings.CutPrefix(line, " ")
|
||||
if !ok || !strings.HasPrefix(name, "Test") || strings.ContainsAny(name, " :") {
|
||||
continue
|
||||
}
|
||||
listed++
|
||||
isTest := declared[name]
|
||||
switch {
|
||||
case notTests[name] && isTest:
|
||||
t.Errorf("%s shard %d/%d lists %s, which check_shard_coverage.py drops, but it is a real test", c.script, k, shards, name)
|
||||
case !notTests[name] && !isTest:
|
||||
t.Errorf("%s shard %d/%d lists %s, which is not a `func %s(t *testing.T)` test in %s: check_shard_coverage.py would report it missing on every run (add it to NOT_TESTS only if go test never runs it)",
|
||||
c.script, k, shards, name, name, c.pkg)
|
||||
}
|
||||
}
|
||||
}
|
||||
if listed < 50 {
|
||||
t.Errorf("%s at %d shards listed only %d tests; did the list-only output format change?", c.script, shards, listed)
|
||||
}
|
||||
// S1: a total that is supposed to have a committed manifest
|
||||
// block (every total this loop considers does: it is either a
|
||||
// live job's matrix size or a total this script's own manifest
|
||||
// already names) must not have silently gone 100% hash fallback,
|
||||
// which is what "the whole block was deleted" looks like from
|
||||
// here: check_shard_coverage.py would still pass (it rebuilds
|
||||
// its expectation from this same script), so nothing else
|
||||
// catches it.
|
||||
if manifestSum == 0 {
|
||||
t.Errorf("%s at %d shards: manifest entries for this total sum to 0 across all shards (100%% hash fallback); its committed block in %s may have been deleted", c.script, shards, mm[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
for name := range notTests {
|
||||
|
|
@ -1294,6 +1385,42 @@ func TestShardScriptsListOnlyRealTests(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// S3: the Bazel-only 30-shard block is not frozen like the legacy 15-shard
|
||||
// block (TestShardScriptsListOnlyRealTests's B1 fix catches outright
|
||||
// corruption, but not a committed block that has drifted from the currently
|
||||
// discovered TestProxiedServer*/TestServerMode* test set, e.g. a test added,
|
||||
// renamed, or removed without anyone running --write). gen_proxied_shard_
|
||||
// manifest.py --check verifies only that the committed block names every
|
||||
// discovered test exactly once -- not that its shard *assignments* match a
|
||||
// fresh LPT pack -- and fails with the exact command to fix it when a name
|
||||
// is missing, stale, or duplicated. It deliberately does NOT fail merely
|
||||
// because proxied_test_durations.json's weights changed and the existing
|
||||
// packing is now suboptimal: two PRs each adding one proxied test would
|
||||
// otherwise force a full repack and conflict on unrelated shard lines (see
|
||||
// --repack below for the explicit opt-in to that). Run --check here so a
|
||||
// block with missing/stale/duplicate names fails go test ./scripts/...
|
||||
// (and so scripts-go-checks, which runs on fork PRs) instead of only
|
||||
// surfacing as a test silently never running in any shard. The legacy
|
||||
// 15-shard block is deliberately excluded: its header documents that it is
|
||||
// frozen and must not be regenerated (see
|
||||
// .github/scripts/proxied-cmd-test-shards.txt and engdocs/TESTING.md), so a
|
||||
// --check against it would always fail by design.
|
||||
func TestProxiedShardManifestGeneratorNotStale(t *testing.T) {
|
||||
if os.Getenv("TEST_SRCDIR") != "" {
|
||||
t.Skip("scripts_test's runfiles hold neither the generator's sources nor cmd/bd")
|
||||
}
|
||||
python, err := exec.LookPath("python3")
|
||||
if err != nil {
|
||||
t.Skip("python3 not available")
|
||||
}
|
||||
root := sourceRepoRoot(t)
|
||||
cmd := exec.Command(python, "scripts/ci/gen_proxied_shard_manifest.py", "30", "--weights=duration", "--check")
|
||||
cmd.Dir = root
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Errorf("gen_proxied_shard_manifest.py 30 --weights=duration --check: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
// Review G3: since D2 the retired tiers' Bazel lanes (embedded, proxied,
|
||||
// server-storage) are those tiers' only pre-merge run on same-repo PRs, so
|
||||
// nothing that reaches them may narrow them (select fewer tests, or turn
|
||||
|
|
|
|||
|
|
@ -37,6 +37,7 @@ scripts TestReleaseFormula* skip # .beads/ is in .bazelignore, so the formula c
|
|||
scripts TestBazelGatedLanesNeverRetryFlakyTests skip # reads every BUILD and .bzl file in the checkout; runs under go test
|
||||
scripts TestCheckShardCoverageScript skip # runs tools/bazel/check_shard_coverage.py, not in the runfiles; runs under go test
|
||||
scripts TestShardScriptsListOnlyRealTests skip # runs the real .github/scripts shard scripts over the source tree; runs under go test
|
||||
scripts TestProxiedShardManifestGeneratorNotStale skip # runs scripts/ci/gen_proxied_shard_manifest.py --check, not in the runfiles; runs under go test
|
||||
scripts TestShardScriptNarrowingCheck skip # reads the real .github/scripts shard scripts; runs under go test
|
||||
scripts TestBazelOnlySkipsAreAllowlisted skip # parses every _test.go file in the checkout; runs under go test
|
||||
scripts TestRunAllowlistedGoTestsScript skip # runs tools/bazel/run_allowlisted_go_tests.py, not in the runfiles; runs under go test
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue