ci(bazel): server-Dolt storage and proxied tiers on remote execution (#6940)
* test(integration): let SubprocessRunner reuse a prebuilt test binary
BEADS_TEST_SUBPROCESS_BINARY hands SubprocessRunner an already-built test
binary of the package instead of running `go test -c`, which no Bazel
action can do offline (it needs the module source tree and module cache).
A Bazel target passes the running go_test itself: the helper tests
(TestHelperSchemaInit, TestHelperMultiStore) live in the same package.
Unset, behavior is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(dolt): keep a lane's REQUIRE switch out of the leak-probe child
TestSuiteFixtureLeakFailsOwningSuite re-runs the suite with
BEADS_TEST_SKIP=dolt on purpose. When the parent runs under
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 (a fail-closed server lane), the child
inherited it and TestMain exited FATAL before the leak sweep the test
checks. Clear it in the child like the other suite-scoped variables.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(bazel): proxied-server and server-Dolt storage tiers (D6d, D6e)
PR Risk's "Test (Proxied Dolt Cmd N/15)" (also main.yml's) and "Test
(Server Dolt Conformance)" / "Test (Server Dolt Full Suite N/16)" as
Bazel variants against local dolt sql-servers (internal/testutil local
backend, pinned dolt from the test wrapper), fail-closed like the
dolt-server lane:
- //cmd/bd:bd_proxied_test: 15 shards; shard k runs the job's
proxied-test-shard.sh k+1 15 through go_test_manifest_shard.sh over the
race bd_test, non-race bd_for_tests as the subprocess bd,
BEADS_TEST_PROXIED_SERVER=1, GOMAXPROCS=4. Tag dolt-server-proxied,
--config=doltserver-proxied (the doltserver lane's build).
- //internal/storage/dolt:dolt_server_{conformance,full}_test over
dolt_race_off (go_test_race_off of dolt_test: the jobs' binary is not
race); the full suite runs server-storage-test-shard.sh k+1 16 with
BEADS_TEST_ENV_RUN_DOLT=1 and hands SubprocessRunner its own binary.
Tag dolt-server-integration, --config=doltserver-integration, whose
build flags are exactly --config=integration's, so the tier shares that
lane's compile (race-off deps land where bd_for_tests already is)
instead of compiling the tagged graph a third time.
- dolt_test gains //internal/storage/schema:migrations as data (the
PR4107/migration replay tests read ../schema/migrations).
- test_env.sh: BEADS_TEST_GIT_IDENTITY=1 writes the jobs' global git
identity ("Configure Git"); the git-remote tests commit in fresh repos.
- Both tags join the taxonomy and bazelPRCoreExcludedTags, so prcore/ci
and integration exclude them.
Per-shard names and statuses equal a replay of the jobs' commands
(container backend) for all 32 jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the proxied and server storage tiers remotely (advisory)
bazel-doltserver gains a second, remote-only step, `bazel test //...
--config=doltserver-proxied` (15 shards of ~5-10 min with a busy dolt
sql-server each do not fit a GitHub-hosted runner; forks and rbe=off keep
the local lane unchanged). bazel-integration gains `bazel test //...
--config=doltserver-integration`, which shares its build flags, so the
server storage tier compiles nothing the integration step has not. Both
steps write their own BEP, run check_testcases.py on it, run even when the
job's first test step failed, and upload logs on failure.
Policy (TestBazelDoltServerTiersMirrorPRRisk): each variant runs its jobs'
shard script with their shard total and has script + manifest in data;
the conformance variant passes the job's flags; the jobs' race/non-race
binaries and switches (BEADS_TEST_PROXIED_SERVER, BEADS_TEST_ENV_RUN_DOLT)
match; doltserver-integration's tags and race equal --config=integration's;
the steps are pinned to their jobs. checkDoltServerRules now holds all
three dolt-server lane tags to the local-backend, fail-closed,
remote-capable rules.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ci(bazel): run the proxied and server storage tiers in jobs of their own
Review of D6d/e: as steps of bazel-doltserver and bazel-integration the
tiers inherited those jobs' gating (a caller switching the integration
lane off would drop PR Risk's PR-time server tier; a gate requiring the
dolt-server lane would silently require the proxied tier too) and
lengthened their critical paths.
bazel-proxied (--config=doltserver-proxied, 15 shards) and
bazel-server-storage (--config=doltserver-integration: conformance + 16
shards) are remote-only jobs that need only rbe, each with its own BEP,
check_testcases.py step and log upload, 25-minute test step inside a
30-minute job. bazel-doltserver and bazel-integration are back to their
previous content, including the dolt-server step's 80-minute timeout for
local runs. The server tier shares bazel-integration's compiles through
the remote cache (same build flags).
The proxied tier mirrors PR Risk's race bd-cmd-test; main.yml's twin
job runs build-artifacts' non-race one, so Bazel is the stricter of the
two. Comments now say so, and the policy test pins main.yml's build so a
change there is noticed. Policy: the new jobs are pinned (command,
config, remote-only if, BEP + testcase steps, timeouts, log upload), and
no tier step may run in bazel-integration or bazel-doltserver.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
9116068c54
commit
e7138e93af
12 changed files with 707 additions and 36 deletions
55
.bazelrc
55
.bazelrc
|
|
@ -37,7 +37,7 @@ test:prcore --test_env=BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION=1
|
|||
# Every tag that belongs to another lane (scripts/bazel_policy_test.go
|
||||
# bazelPRCoreExcludedTags). integration-only targets run zero tests outside
|
||||
# --config=integration.
|
||||
test:prcore --test_tag_filters=-requires-docker,-dolt-server,-embedded,-manual,-integration-only
|
||||
test:prcore --test_tag_filters=-requires-docker,-dolt-server,-dolt-server-proxied,-dolt-server-integration,-embedded,-manual,-integration-only
|
||||
|
||||
test:ci --config=prcore
|
||||
test:ci --keep_going --test_summary=terse
|
||||
|
|
@ -123,6 +123,57 @@ test:doltserver --test_arg=-test.parallel=4
|
|||
test:doltserver --local_test_jobs=4
|
||||
test:doltserver --remote_download_regex=.*/test\.(log|xml)$
|
||||
|
||||
# Proxied-server tier: PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs, as
|
||||
# //cmd/bd:bd_proxied_test (dolt-server-proxied tag). Kept out of
|
||||
# --config=doltserver because bazel.yml runs that lane locally too (forks,
|
||||
# rbe=off), where 15 shards of ~10 minutes, each with a dolt sql-server busy
|
||||
# on ~4 cores, do not fit; bazel.yml runs this one only with remote
|
||||
# execution, in a job of its own (bazel-proxied). The build is
|
||||
# --config=doltserver's: race, like PR Risk's /tmp/bd-cmd-test (main.yml's
|
||||
# twin job runs build-artifacts' non-race bd-cmd-test, so this is the
|
||||
# stricter of the two); the subprocess bd is the non-race bd_for_tests, like
|
||||
# the jobs'. Each shard runs its job's
|
||||
# shard script, whose -test.timeout=15m is past "long": the target is
|
||||
# eternal, which rbe-west caps at 1200s.
|
||||
test:doltserver-proxied --@rules_go//go/config:race
|
||||
test:doltserver-proxied --test_tag_filters=dolt-server-proxied
|
||||
test:doltserver-proxied --build_tests_only
|
||||
test:doltserver-proxied --keep_going
|
||||
test:doltserver-proxied --test_summary=terse
|
||||
test:doltserver-proxied --test_timeout=-1,-1,-1,1200
|
||||
test:doltserver-proxied --test_env=GO_TEST_WRAP_TESTV=1
|
||||
# The jobs pass no -parallel: GOMAXPROCS (the target's 4, a runner's vCPUs).
|
||||
test:doltserver-proxied --test_arg=-test.parallel=4
|
||||
test:doltserver-proxied --local_test_jobs=4
|
||||
test:doltserver-proxied --remote_download_regex=.*/test\.(log|xml)$
|
||||
|
||||
# Server-Dolt storage tier: PR Risk's "Test (Server Dolt Conformance)" and
|
||||
# "Test (Server Dolt Full Suite N/16)" jobs, as
|
||||
# //internal/storage/dolt:dolt_server_{conformance,full}_test
|
||||
# (dolt-server-integration tag): the integration-tagged internal/storage/dolt
|
||||
# tests, non-race like the jobs' binary, against a local dolt sql-server.
|
||||
# The build flags are exactly --config=integration's (the tags, and race for
|
||||
# everything the variants do not turn off), so the two lanes share every
|
||||
# compile: the variants' race-off binary lands in the configuration
|
||||
# --config=integration already builds the non-race //cmd/bd:bd_for_tests in.
|
||||
# A per-target gotags transition would instead compile the whole graph a
|
||||
# third time. No BEADS_TEST_SKIP, no -short; the full suite's
|
||||
# BEADS_TEST_ENV_RUN_DOLT=1 is target env. Remote only in bazel.yml, in a
|
||||
# job of its own (bazel-server-storage) that shares bazel-integration's
|
||||
# compiles through the remote cache.
|
||||
build:doltserver-integration --@rules_go//go/config:tags=gms_pure_go,integration
|
||||
test:doltserver-integration --@rules_go//go/config:race
|
||||
test:doltserver-integration --test_tag_filters=dolt-server-integration
|
||||
test:doltserver-integration --build_tests_only
|
||||
test:doltserver-integration --keep_going
|
||||
test:doltserver-integration --test_summary=terse
|
||||
test:doltserver-integration --test_timeout=-1,-1,-1,1200
|
||||
test:doltserver-integration --test_env=GO_TEST_WRAP_TESTV=1
|
||||
# The jobs pass no -parallel: GOMAXPROCS (the targets' 4, a runner's vCPUs).
|
||||
test:doltserver-integration --test_arg=-test.parallel=4
|
||||
test:doltserver-integration --local_test_jobs=4
|
||||
test:doltserver-integration --remote_download_regex=.*/test\.(log|xml)$
|
||||
|
||||
# Integration lane: main.yml's "Main Linux integration packages (N/6)" and
|
||||
# "Main Linux integration cmd/bd (N/8)" jobs (and nightly.yml's Full Test
|
||||
# Suite, which runs the same `go test` over ./... in one job): every package
|
||||
|
|
@ -141,7 +192,7 @@ test:integration --test_env=BEADS_TEST_SKIP=dolt
|
|||
# keeps every container- or server-backed test skipped, which is what the
|
||||
# untagged base targets already do. scripts/bazel_policy_test.go derives the
|
||||
# required exclusions from the PR-core list, so a new lane tag lands here too.
|
||||
test:integration --test_tag_filters=-requires-docker,-dolt-server,-embedded,-manual
|
||||
test:integration --test_tag_filters=-requires-docker,-dolt-server,-dolt-server-proxied,-dolt-server-integration,-embedded,-manual
|
||||
test:integration --build_tests_only
|
||||
test:integration --keep_going
|
||||
test:integration --test_summary=terse
|
||||
|
|
|
|||
132
.github/workflows/bazel.yml
vendored
132
.github/workflows/bazel.yml
vendored
|
|
@ -1,8 +1,9 @@
|
|||
# Bazel side-by-side lane (advisory). Runs `bazel test //... --config=ci`, the
|
||||
# PR Core-parity selection (race, -short, dolt skip, -requires-docker), on the
|
||||
# same commit as pr.yml's go test lanes, plus one job per mirrored CI job
|
||||
# (pure-Go/js-wasm, docker lane, pr-risk.yml's embedded-Dolt tier, main.yml's
|
||||
# integration jobs). It is NOT
|
||||
# (pure-Go/js-wasm, the dolt-server lane, pr-risk.yml's embedded-Dolt,
|
||||
# proxied-server and server-Dolt storage tiers, main.yml's integration jobs).
|
||||
# It is NOT
|
||||
# in ci-gate's CI_GATE_REQUIRED; promotion needs the equivalence record
|
||||
# described in tools/bazel/equivalence.py and a human decision.
|
||||
#
|
||||
|
|
@ -17,8 +18,8 @@
|
|||
# local: fork PRs, rbe=off dispatches, and same-repo runs without the
|
||||
# secret (Dependabot PRs: same-repo, but no Actions secrets): local
|
||||
# execution on ubuntu-latest, slower but the same results, for every job
|
||||
# but the remote-only ones (bazel-embedded, bazel-integration), which
|
||||
# are skipped.
|
||||
# but the remote-only ones (bazel-embedded, bazel-integration,
|
||||
# bazel-proxied, bazel-server-storage), which are skipped.
|
||||
# skip: same-repo runs while RBE_WEST_WORKERS is unset (the farm switch).
|
||||
# workflow_dispatch with rbe=off simulates a fork run. The decision is also a
|
||||
# workflow_call output, so a caller's gate can tell an expected skip from a
|
||||
|
|
@ -677,3 +678,126 @@ jobs:
|
|||
bazel-testlogs/**/test.xml
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
# pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)", as
|
||||
# --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the job's
|
||||
# shard script in 15 Bazel shards, so shard k runs exactly job k's tests.
|
||||
# Same build as --config=doltserver (race bd_test, non-race bd_for_tests),
|
||||
# shared with bazel-doltserver through the remote cache. PR Risk builds
|
||||
# its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd twin
|
||||
# uses build-artifacts' non-race bd-cmd-test, so this lane is the stricter
|
||||
# of the two. A job of its own, not gated by any integration switch, so a
|
||||
# caller's gate can decide on it separately.
|
||||
bazel-proxied:
|
||||
name: Bazel / proxied-server tier (advisory)
|
||||
# Remote execution only, like bazel-embedded: 15 shards of ~5-10
|
||||
# minutes, each with a dolt sql-server busy on ~4 cores, do not fit a
|
||||
# GitHub-hosted runner.
|
||||
needs: rbe
|
||||
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
||||
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Bazel
|
||||
id: bazel
|
||||
uses: ./.github/actions/setup-bazel
|
||||
env:
|
||||
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
||||
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
||||
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
||||
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
||||
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
||||
|
||||
# Remote: the longest shard takes ~5-8 minutes (each shard's Go
|
||||
# timeout is its job's 15m, 1200s Bazel cap), plus a compile.
|
||||
- name: bazel test //... --config=doltserver-proxied
|
||||
id: test
|
||||
timeout-minutes: 25
|
||||
run: |
|
||||
set -o pipefail
|
||||
start=$(date +%s)
|
||||
rc=0
|
||||
bazel test //... --config=doltserver-proxied \
|
||||
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
||||
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
||||
echo "bazel test --config=doltserver-proxied: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
||||
exit "$rc"
|
||||
|
||||
# As in bazel-embedded.
|
||||
- name: Every target and shard ran tests
|
||||
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
||||
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
||||
|
||||
- name: Upload test logs
|
||||
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: bazel-proxied-testlogs
|
||||
path: |
|
||||
bazel-testlogs/**/test.log
|
||||
bazel-testlogs/**/test.xml
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
# pr-risk.yml's server-Dolt storage tier, "Test (Server Dolt Conformance)"
|
||||
# and "Test (Server Dolt Full Suite N/16)", as --config=doltserver-integration:
|
||||
# the conformance and 16-shard full-suite variants of
|
||||
# //internal/storage/dolt over a non-race dolt_test build, each shard
|
||||
# running the job's own shard script. --config=doltserver-integration has
|
||||
# exactly --config=integration's build flags, so it compiles nothing
|
||||
# bazel-integration does not (shared through the remote cache). A job of
|
||||
# its own, not gated by any integration switch: this is a PR-time tier,
|
||||
# unlike main.yml's integration jobs.
|
||||
bazel-server-storage:
|
||||
name: Bazel / server-Dolt storage tier (advisory)
|
||||
# Remote execution only, like bazel-integration: locally the tagged race
|
||||
# build of //... alone is most of the timeout.
|
||||
needs: rbe
|
||||
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
||||
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Bazel
|
||||
id: bazel
|
||||
uses: ./.github/actions/setup-bazel
|
||||
env:
|
||||
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
||||
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
||||
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
||||
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
||||
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
||||
|
||||
# Remote: the longest shard takes ~2-3 minutes (the storage suite's Go
|
||||
# timeout is 15m per shard, 1200s Bazel cap), plus a compile.
|
||||
- name: bazel test //... --config=doltserver-integration
|
||||
id: test
|
||||
timeout-minutes: 25
|
||||
run: |
|
||||
set -o pipefail
|
||||
start=$(date +%s)
|
||||
rc=0
|
||||
bazel test //... --config=doltserver-integration \
|
||||
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
||||
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
||||
echo "bazel test --config=doltserver-integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
||||
exit "$rc"
|
||||
|
||||
# As in bazel-embedded.
|
||||
- name: Every target and shard ran tests
|
||||
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
||||
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
||||
|
||||
- name: Upload test logs
|
||||
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: bazel-server-storage-testlogs
|
||||
path: |
|
||||
bazel-testlogs/**/test.log
|
||||
bazel-testlogs/**/test.xml
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
|
|
|||
13
BUILD.bazel
13
BUILD.bazel
|
|
@ -69,9 +69,11 @@ filegroup(
|
|||
visibility = ["//scripts:__pkg__"],
|
||||
)
|
||||
|
||||
# PR Risk's embedded-Dolt shard scripts and their committed manifests. The
|
||||
# embedded variants (//cmd/bd:bd_embedded_test,
|
||||
# //internal/storage/embeddeddolt:embeddeddolt_embedded_test) run them through
|
||||
# PR Risk's manifest-sharded tiers' shard scripts and their committed
|
||||
# manifests (embedded-Dolt cmd and storage, proxied-server cmd, server-Dolt
|
||||
# storage). The variants (//cmd/bd:bd_embedded_test, //cmd/bd:bd_proxied_test,
|
||||
# //internal/storage/embeddeddolt:embeddeddolt_embedded_test,
|
||||
# //internal/storage/dolt:dolt_server_full_test) run them through
|
||||
# tools/bazel/go_test_manifest_shard.sh, so each Bazel shard runs exactly the
|
||||
# tests of the CI job with the same shard number.
|
||||
exports_files(
|
||||
|
|
@ -80,9 +82,14 @@ exports_files(
|
|||
".github/scripts/embedded-storage-test-shard.sh",
|
||||
".github/scripts/embedded-storage-test-shards.txt",
|
||||
".github/scripts/embedded-test-shard.sh",
|
||||
".github/scripts/proxied-cmd-test-shards.txt",
|
||||
".github/scripts/proxied-test-shard.sh",
|
||||
".github/scripts/server-storage-test-shard.sh",
|
||||
".github/scripts/server-storage-test-shards.txt",
|
||||
],
|
||||
visibility = [
|
||||
"//cmd/bd:__pkg__",
|
||||
"//internal/storage/dolt:__pkg__",
|
||||
"//internal/storage/embeddeddolt:__pkg__",
|
||||
],
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1326,6 +1326,62 @@ sh_test(
|
|||
tags = ["embedded"],
|
||||
)
|
||||
|
||||
# bd_test as PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs run it: shard k
|
||||
# runs .github/scripts/proxied-test-shard.sh k+1 15 (the TestProxiedServer*
|
||||
# functions, assigned by the committed manifest) against this configuration's
|
||||
# (race) bd_test with BEADS_TEST_PROXIED_SERVER=1 and the non-race
|
||||
# bd_for_tests as the subprocess binary, like the jobs' /tmp/bd-cmd-test and
|
||||
# /tmp/bd-proxied. (main.yml's twin job runs the same shard script against
|
||||
# build-artifacts' non-race bd-cmd-test; race here is the stricter one.)
|
||||
# The shared proxied harness gets its dolt sql-server from
|
||||
# internal/testutil's local backend (the pinned dolt in the test wrapper's
|
||||
# runfiles) instead of the jobs' container, and fails instead of skipping
|
||||
# when it cannot start one. It repeats bd_test's `env`. shard_count must
|
||||
# stay equal to the manifest's shard total.
|
||||
#
|
||||
# Tags:
|
||||
# dolt-server-proxied: the proxied-server tier; starts hermetic dolt
|
||||
# sql-servers; excluded from --config=prcore/ci, selected by
|
||||
# --config=doltserver-proxied.
|
||||
sh_test(
|
||||
name = "bd_proxied_test",
|
||||
size = "large",
|
||||
# The script's -test.timeout=15m fires (with a goroutine dump) before
|
||||
# Bazel's 1200s action limit (eternal under the lane's --test_timeout).
|
||||
timeout = "eternal",
|
||||
srcs = ["//tools/bazel:go_test_manifest_shard.sh"],
|
||||
args = [
|
||||
"$(rootpath //:.github/scripts/proxied-test-shard.sh)",
|
||||
"BEADS_TEST_CMD_BINARY",
|
||||
"$(rootpath :bd_test)",
|
||||
],
|
||||
data = [
|
||||
":bd_for_tests",
|
||||
":bd_test",
|
||||
":go_srcs",
|
||||
"//:.github/scripts/proxied-cmd-test-shards.txt",
|
||||
"//:.github/scripts/proxied-test-shard.sh",
|
||||
"@go_sdk//:bin/gofmt",
|
||||
],
|
||||
env = {
|
||||
"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)",
|
||||
"BEADS_TEST_DOLT_SERVER": "local",
|
||||
# The jobs' "Configure Git and Dolt identity" global git identity
|
||||
# (tools/bazel/test_env.sh).
|
||||
"BEADS_TEST_GIT_IDENTITY": "1",
|
||||
"BEADS_TEST_GOFMT": "$(rlocationpath @go_sdk//:bin/gofmt)",
|
||||
"BEADS_TEST_PROXIED_SERVER": "1",
|
||||
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
|
||||
# The 4-vCPU runner shape: -test.parallel defaults to GOMAXPROCS, and
|
||||
# the job runs 4 tests at once, not one per worker core (39
|
||||
# concurrent `bd init --proxied-server` against one server hung a
|
||||
# 192-core worker).
|
||||
"GOMAXPROCS": "4",
|
||||
},
|
||||
shard_count = 15,
|
||||
tags = ["dolt-server-proxied"],
|
||||
)
|
||||
|
||||
# CLI flag registrations pinned by //internal/httpapi spec parity tests.
|
||||
exports_files(
|
||||
[
|
||||
|
|
|
|||
|
|
@ -104,6 +104,15 @@ suites of pr.yml's "Test (storage domain + uow)" and "Contract corpus" jobs
|
|||
on the `local` backend (the `dolt-server` targets); they need no docker and
|
||||
execute remotely with `--config=remote-exec`. `--config=docker` runs the same
|
||||
suites on the `container` backend (host docker) as the A/B control.
|
||||
PR Risk's heavier server tiers have configs of their own, run by bazel.yml
|
||||
only with remote execution, each in a job of its own (`bazel-proxied`,
|
||||
`bazel-server-storage`): `--config=doltserver-proxied` is the
|
||||
proxied-server cmd/bd tier ("Test (Proxied Dolt Cmd N/15)",
|
||||
`//cmd/bd:bd_proxied_test`), and `--config=doltserver-integration` the
|
||||
server-Dolt storage tier ("Test (Server Dolt Conformance)", "Test (Server
|
||||
Dolt Full Suite N/16)", `//internal/storage/dolt:dolt_server_*_test`), which
|
||||
builds with the integration tag like `--config=integration`. Each shard
|
||||
runs its CI job's shard script, so Bazel shard k runs the tests of job k+1.
|
||||
|
||||
Tests that need a temporary repository or store should use `t.TempDir()` and
|
||||
`t.Cleanup()`. Temporary repositories must set a repository-local hooks path;
|
||||
|
|
|
|||
|
|
@ -1,4 +1,6 @@
|
|||
load("@rules_go//go:def.bzl", "go_library", "go_test")
|
||||
load("@rules_shell//shell:sh_test.bzl", "sh_test")
|
||||
load("//tools/bazel:race.bzl", "go_test_race_off")
|
||||
|
||||
go_library(
|
||||
name = "dolt",
|
||||
|
|
@ -116,15 +118,13 @@ go_library(
|
|||
],
|
||||
)
|
||||
|
||||
# Under --config=integration, TestMultiProcessSchemaInit* and
|
||||
# TestMultiStoreConcurrent_Subprocess build this package with a real
|
||||
# `go test -c` (integration.SubprocessRunner). That needs the module's source
|
||||
# tree and every dependency's module (dolt, go-mysql-server, ...) in a module
|
||||
# cache, which no action has offline, so unlike //internal/testutil/integration
|
||||
# (whose build fails on purpose) this target gets no BEADS_TEST_GO_BINARY/SDK
|
||||
# data. Fine while the lane keeps BEADS_TEST_SKIP=dolt (main.yml's integration
|
||||
# jobs), which skips those tests; a dolt-backed integration variant must build
|
||||
# the helper binary with Bazel instead.
|
||||
# TestMultiProcessSchemaInit* and TestMultiStoreConcurrent_Subprocess build
|
||||
# this package with a real `go test -c` (integration.SubprocessRunner), which
|
||||
# needs the module's source tree and a module cache no action has. They run
|
||||
# only against a server: skipped under --config=integration
|
||||
# (BEADS_TEST_SKIP=dolt, like main.yml's integration jobs), and run by
|
||||
# dolt_server_full_test, which hands SubprocessRunner its own test binary
|
||||
# (BEADS_TEST_SUBPROCESS_BINARY) instead.
|
||||
go_test(
|
||||
name = "dolt_test",
|
||||
srcs = [
|
||||
|
|
@ -354,10 +354,12 @@ go_test(
|
|||
"workspace_config_contract_test.go",
|
||||
],
|
||||
# Source-scan guards (journal/version scope completeness, helper sentinel
|
||||
# census) parse this package and ../issueops at test time.
|
||||
# census) parse this package and ../issueops at test time; the PR4107 and
|
||||
# migration replay tests (server-backed) read ../schema/migrations.
|
||||
data = [
|
||||
":go_srcs",
|
||||
"//internal/storage/issueops:go_srcs",
|
||||
"//internal/storage/schema:migrations",
|
||||
],
|
||||
embed = [":dolt"],
|
||||
deps = [
|
||||
|
|
@ -454,6 +456,102 @@ go_test(
|
|||
}),
|
||||
)
|
||||
|
||||
# dolt_test without the race detector: PR Risk's server-Dolt jobs run
|
||||
# build-embedded's /tmp/dolt-conformance-test, built by `go test -c` with the
|
||||
# integration and gms_pure_go tags but without -race. tools/bazel/race.bzl lands
|
||||
# its deps in the configuration the non-race //cmd/bd:bd_for_tests is built
|
||||
# in; under --config=doltserver-integration (the integration lane's build
|
||||
# flags) that is the configuration --config=integration builds bd_for_tests
|
||||
# in, so the server tier compiles no third copy of the graph.
|
||||
#
|
||||
# Tags:
|
||||
# manual: only the dolt_server_* variants' input, never a build target of
|
||||
# its own.
|
||||
go_test_race_off(
|
||||
name = "dolt_race_off",
|
||||
tags = ["manual"],
|
||||
test = ":dolt_test",
|
||||
)
|
||||
|
||||
# PR Risk's "Test (Server Dolt Conformance)" job: conformance.RunAll against
|
||||
# server-mode Dolt, with the job's binary flags. TestMain's server is a local
|
||||
# `dolt sql-server` started from the pinned dolt in the test wrapper's
|
||||
# runfiles instead of the job's container, and a server that cannot start
|
||||
# fails the target (BEADS_TEST_REQUIRE_DOLT_CONTAINER=1) instead of skipping
|
||||
# the suite. It repeats dolt_test's `env` (it has none).
|
||||
#
|
||||
# Tags:
|
||||
# dolt-server-integration: starts hermetic dolt sql-servers and needs the
|
||||
# integration build tag; excluded from --config=prcore/ci and
|
||||
# --config=integration, selected by --config=doltserver-integration.
|
||||
sh_test(
|
||||
name = "dolt_server_conformance_test",
|
||||
size = "large",
|
||||
# The job's -test.timeout=15m fires (with a goroutine dump) well before
|
||||
# Bazel's 1200s action limit (eternal under the lane's --test_timeout).
|
||||
timeout = "eternal",
|
||||
srcs = ["//tools/bazel:go_test_variant.sh"],
|
||||
args = [
|
||||
"$(rootpath :dolt_race_off)",
|
||||
"-test.v",
|
||||
"-test.count=1",
|
||||
"-test.timeout=15m",
|
||||
"-test.run=^TestConformance$$",
|
||||
],
|
||||
data = [":dolt_race_off"],
|
||||
env = {
|
||||
"BEADS_TEST_DOLT_SERVER": "local",
|
||||
# The job's "Configure Git" global identity (tools/bazel/test_env.sh).
|
||||
"BEADS_TEST_GIT_IDENTITY": "1",
|
||||
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
|
||||
# The 4-vCPU runner shape: -test.parallel defaults to GOMAXPROCS.
|
||||
"GOMAXPROCS": "4",
|
||||
},
|
||||
tags = ["dolt-server-integration"],
|
||||
)
|
||||
|
||||
# PR Risk's "Test (Server Dolt Full Suite N/16)" jobs: shard k runs
|
||||
# .github/scripts/server-storage-test-shard.sh k+1 16 (every top-level test
|
||||
# but TestConformance, assigned by the committed manifest) against
|
||||
# dolt_race_off, with the jobs' BEADS_TEST_ENV_RUN_DOLT=1 and a local dolt
|
||||
# sql-server as in dolt_server_conformance_test. SubprocessRunner's helper
|
||||
# processes re-exec this same binary (BEADS_TEST_SUBPROCESS_BINARY) where the
|
||||
# job compiles the package again with `go test -tags integration -c`; go.mod
|
||||
# lets integration.ModuleRoot find the module root it no longer builds in.
|
||||
# shard_count must stay equal to the manifest's shard total.
|
||||
#
|
||||
# Tags:
|
||||
# dolt-server-integration: as dolt_server_conformance_test.
|
||||
sh_test(
|
||||
name = "dolt_server_full_test",
|
||||
size = "large",
|
||||
# The script's -test.timeout=15m fires before Bazel's 1200s limit.
|
||||
timeout = "eternal",
|
||||
srcs = ["//tools/bazel:go_test_manifest_shard.sh"],
|
||||
args = [
|
||||
"$(rootpath //:.github/scripts/server-storage-test-shard.sh)",
|
||||
"BEADS_TEST_SERVER_TEST_BINARY",
|
||||
"$(rootpath :dolt_race_off)",
|
||||
],
|
||||
data = [
|
||||
":dolt_race_off",
|
||||
":go_srcs",
|
||||
"//:.github/scripts/server-storage-test-shard.sh",
|
||||
"//:.github/scripts/server-storage-test-shards.txt",
|
||||
"//:go.mod",
|
||||
],
|
||||
env = {
|
||||
"BEADS_TEST_DOLT_SERVER": "local",
|
||||
"BEADS_TEST_ENV_RUN_DOLT": "1",
|
||||
"BEADS_TEST_GIT_IDENTITY": "1",
|
||||
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
|
||||
"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath :dolt_race_off)",
|
||||
"GOMAXPROCS": "4",
|
||||
},
|
||||
shard_count = 16,
|
||||
tags = ["dolt-server-integration"],
|
||||
)
|
||||
|
||||
# --- begin go_srcs (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
|
||||
|
||||
filegroup(
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import (
|
|||
"time"
|
||||
|
||||
"github.com/steveyegge/beads/internal/doltserver"
|
||||
"github.com/steveyegge/beads/internal/testutil"
|
||||
"github.com/steveyegge/beads/internal/testutil/bazeltest"
|
||||
)
|
||||
|
||||
|
|
@ -67,6 +68,10 @@ func TestSuiteFixtureLeakFailsOwningSuite(t *testing.T) {
|
|||
defer writeEnd.Close()
|
||||
t.Setenv("BEADS_SUITE_LEAK_PROBE", "1")
|
||||
t.Setenv("BEADS_TEST_SKIP", "dolt")
|
||||
// The child skips Dolt on purpose, so a lane's fail-closed switch (set
|
||||
// when this suite runs against a required server) must not turn that
|
||||
// skip into a FATAL before the child reaches its leak sweep.
|
||||
t.Setenv(testutil.EnvRequireDoltContainer, "")
|
||||
t.Setenv(doltserver.AllowLeakEnv, "")
|
||||
// This child is a complete suite, not one of the schema-init helpers
|
||||
// which deliberately bypass suite ownership and shutdown.
|
||||
|
|
|
|||
|
|
@ -133,10 +133,32 @@ func goBinary() (string, error) {
|
|||
return bazeltest.RunfileEnv(GoBinaryEnv)
|
||||
}
|
||||
|
||||
// PrebuiltTestBinaryEnv names a variable that hands SubprocessRunner an
|
||||
// already-built test binary of the package it would compile, instead of
|
||||
// running `go test -c`. Under Bazel no action has the module's source tree
|
||||
// and module cache that `go test -c` needs, but the test binary that is
|
||||
// running is that package's test binary, integration-tagged, so the target
|
||||
// passes it (resolved like any other runfile):
|
||||
//
|
||||
// env = {"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath <the go_test>)"},
|
||||
const PrebuiltTestBinaryEnv = "BEADS_TEST_SUBPROCESS_BINARY"
|
||||
|
||||
// Build compiles the test binary (once). Returns the path to the binary.
|
||||
// With BEADS_TEST_SUBPROCESS_BINARY set it compiles nothing and returns that
|
||||
// binary.
|
||||
func (r *SubprocessRunner) Build(t *testing.T) string {
|
||||
t.Helper()
|
||||
r.once.Do(func() {
|
||||
if os.Getenv(PrebuiltTestBinaryEnv) != "" {
|
||||
bin, err := bazeltest.RunfileEnv(PrebuiltTestBinaryEnv)
|
||||
if err != nil {
|
||||
r.buildErr = fmt.Errorf("prebuilt test binary: %w", err)
|
||||
return
|
||||
}
|
||||
t.Logf("SubprocessRunner: using prebuilt test binary %s (%s) for %s", bin, PrebuiltTestBinaryEnv, r.pkg)
|
||||
r.testBin = bin
|
||||
return
|
||||
}
|
||||
r.testBin = filepath.Join(t.TempDir(), "integration.test")
|
||||
t.Logf("SubprocessRunner: building test binary: go test -tags %s -c -o %s %s", r.tags, r.testBin, r.pkg)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
|
|
|
|||
|
|
@ -679,8 +679,14 @@ var allowedBazelTestTags = map[string]string{
|
|||
// pinned dolt in its runfiles, so it runs on any worker, and a server that
|
||||
// cannot start fails it rather than skipping, so its cached result holds.
|
||||
"dolt-server": "starts hermetic dolt sql-servers (or completes the lane's job without -short); excluded from --config=prcore/ci, run by --config=doltserver",
|
||||
"embedded": "embedded-Dolt tier variant; excluded from --config=prcore/ci, run by --config=embedded",
|
||||
"manual": "never part of //...: a repro/bench harness, or a build input only another target needs; excluded from --config=prcore/ci",
|
||||
// The same rules as dolt-server (hermetic servers, remote, fail-closed:
|
||||
// checkDoltServerRules), for the tiers bazel.yml runs only with remote
|
||||
// execution and, for the server storage tier, under the integration
|
||||
// lane's build flags.
|
||||
"dolt-server-proxied": "proxied-server cmd/bd tier: starts hermetic dolt sql-servers; excluded from --config=prcore/ci, run by --config=doltserver-proxied",
|
||||
"dolt-server-integration": "server-Dolt storage tier: starts hermetic dolt sql-servers and needs the integration build tag; excluded from --config=prcore/ci, run by --config=doltserver-integration",
|
||||
"embedded": "embedded-Dolt tier variant; excluded from --config=prcore/ci, run by --config=embedded",
|
||||
"manual": "never part of //...: a repro/bench harness, or a build input only another target needs; excluded from --config=prcore/ci",
|
||||
// For a go_test whose every test file is `//go:build integration`: in any
|
||||
// other configuration rules_go drops those files and the target runs
|
||||
// zero tests, which check_testcases.py rejects and equivalence.py can
|
||||
|
|
@ -694,7 +700,7 @@ var allowedBazelTestTags = map[string]string{
|
|||
// and vice versa (TestBazelPRCoreExcludedTagsMatchTaxonomy), so a new lane
|
||||
// tag lands here, and through bazelIntegrationExcludedTags in the
|
||||
// integration lane's filter too.
|
||||
var bazelPRCoreExcludedTags = []string{"requires-docker", "dolt-server", "embedded", "manual", "integration-only"}
|
||||
var bazelPRCoreExcludedTags = []string{"requires-docker", "dolt-server", "dolt-server-proxied", "dolt-server-integration", "embedded", "manual", "integration-only"}
|
||||
|
||||
// bazelIntegrationRunsTags are the PR-core-excluded tags --config=integration
|
||||
// runs: the integration lane is main.yml's integration jobs, whose
|
||||
|
|
|
|||
|
|
@ -1471,6 +1471,8 @@ const (
|
|||
bazelEmbedJobName = "bazel-embedded"
|
||||
bazelRBEJobName = "rbe"
|
||||
bazelIntegJobName = "bazel-integration"
|
||||
bazelProxiedJobName = "bazel-proxied"
|
||||
bazelServerJobName = "bazel-server-storage"
|
||||
setupBazelActionDir = ".github/actions/setup-bazel"
|
||||
uploadArtifactSHA = "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
|
||||
downloadArtifactSHA = "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c"
|
||||
|
|
@ -1486,11 +1488,11 @@ const (
|
|||
|
||||
// bazel.yml's jobs: the rbe job that decides the execution mode, the
|
||||
// --config=ci lane, and one job per CI job a Bazel config mirrors.
|
||||
var bazelJobNames = []string{bazelDoltJobName, bazelEmbedJobName, bazelIntegJobName, bazelPureJobName, bazelJobName, bazelRBEJobName}
|
||||
var bazelJobNames = []string{bazelDoltJobName, bazelEmbedJobName, bazelIntegJobName, bazelProxiedJobName, bazelPureJobName, bazelServerJobName, bazelJobName, bazelRBEJobName}
|
||||
|
||||
// The lanes that only run remotely (skipped unless the rbe job chose remote);
|
||||
// every other lane also runs locally.
|
||||
var bazelRemoteOnlyJobs = map[string]bool{bazelEmbedJobName: true, bazelIntegJobName: true}
|
||||
var bazelRemoteOnlyJobs = map[string]bool{bazelEmbedJobName: true, bazelIntegJobName: true, bazelProxiedJobName: true, bazelServerJobName: true}
|
||||
|
||||
// The rbe job's decision step reads exactly one secret, and only to test it
|
||||
// for emptiness: its env value is a boolean, not the secret.
|
||||
|
|
@ -1868,8 +1870,16 @@ var (
|
|||
doltServerExtraEnv = map[string][]string{
|
||||
"fix_dolt_test": {`"BEADS_FIX_REQUIRE_DOLT": "1"`},
|
||||
"protocol_dolt_test": {`"BEADS_PROTOCOL_REQUIRE_DOLT": "1"`},
|
||||
// Without it every proxied test self-skips (the jobs' step env).
|
||||
"bd_proxied_test": {`"BEADS_TEST_PROXIED_SERVER": "1"`},
|
||||
// federation_test.go fails instead of skipping without a server.
|
||||
"dolt_server_full_test": {`"BEADS_TEST_ENV_RUN_DOLT": "1"`},
|
||||
}
|
||||
bazelRuleNameRe = regexp.MustCompile(`(?m)^\s*name\s*=\s*"([^"]+)"`)
|
||||
// doltServerLaneTags are the tags of the lanes whose targets start
|
||||
// hermetic dolt sql-servers; checkDoltServerRules holds each of them to
|
||||
// the same rules.
|
||||
doltServerLaneTags = []string{"dolt-server", "dolt-server-proxied", "dolt-server-integration"}
|
||||
bazelRuleNameRe = regexp.MustCompile(`(?m)^\s*name\s*=\s*"([^"]+)"`)
|
||||
)
|
||||
|
||||
// bazelTopRules splits a BUILD file into its top-level calls: each starts at
|
||||
|
|
@ -1893,14 +1903,26 @@ func bazelTopRules(build string) []string {
|
|||
return rules
|
||||
}
|
||||
|
||||
// checkDoltServerRules checks each rule tagged dolt-server in one package's
|
||||
// BUILD file on its own (a file-wide search would accept another target's
|
||||
// env), and that the package has at least one.
|
||||
// doltServerRuleTag returns the dolt-server lane tag a rule carries, or "".
|
||||
func doltServerRuleTag(rule string) string {
|
||||
for _, tag := range doltServerLaneTags {
|
||||
if strings.Contains(rule, `"`+tag+`"`) {
|
||||
return tag
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// checkDoltServerRules checks each rule tagged for a dolt-server lane
|
||||
// (doltServerLaneTags) in one package's BUILD file on its own (a file-wide
|
||||
// search would accept another target's env), and that the package has at
|
||||
// least one.
|
||||
func checkDoltServerRules(pkg, build string) []error {
|
||||
var errs []error
|
||||
found := 0
|
||||
for _, rule := range bazelTopRules(stripStarlarkComments(build)) {
|
||||
if !strings.Contains(rule, `"dolt-server"`) {
|
||||
tag := doltServerRuleTag(rule)
|
||||
if tag == "" {
|
||||
continue
|
||||
}
|
||||
found++
|
||||
|
|
@ -1908,14 +1930,14 @@ func checkDoltServerRules(pkg, build string) []error {
|
|||
if m := bazelRuleNameRe.FindStringSubmatch(rule); m != nil {
|
||||
name = m[1]
|
||||
}
|
||||
where := "//" + pkg + ":" + name
|
||||
where := "//" + pkg + ":" + name + " (" + tag + ")"
|
||||
for _, env := range append(append([]string{}, doltServerRuleEnv...), doltServerExtraEnv[name]...) {
|
||||
if !strings.Contains(rule, env) {
|
||||
errs = append(errs, errors.New(where+" (dolt-server) lacks rule env "+env))
|
||||
errs = append(errs, errors.New(where+" lacks rule env "+env))
|
||||
}
|
||||
}
|
||||
if strings.Contains(rule, `"no-remote-exec"`) {
|
||||
errs = append(errs, errors.New(where+" (dolt-server) is tagged no-remote-exec; the lane runs on remote workers"))
|
||||
errs = append(errs, errors.New(where+" is tagged no-remote-exec; the lane runs on remote workers"))
|
||||
}
|
||||
}
|
||||
if found == 0 {
|
||||
|
|
@ -1960,6 +1982,13 @@ sh_test(
|
|||
"no target": docker,
|
||||
"fix switch": strings.Replace(good, `"uow_dolt_test"`, `"fix_dolt_test"`, 1),
|
||||
"protocol switch": strings.Replace(good, `"uow_dolt_test"`, `"protocol_dolt_test"`, 1),
|
||||
"proxied switch": strings.Replace(strings.Replace(good, `"uow_dolt_test"`, `"bd_proxied_test"`, 1),
|
||||
`tags = ["dolt-server"]`, `tags = ["dolt-server-proxied"]`, 1),
|
||||
"run-dolt switch": strings.Replace(strings.Replace(good, `"uow_dolt_test"`, `"dolt_server_full_test"`, 1),
|
||||
`tags = ["dolt-server"]`, `tags = ["dolt-server-integration"]`, 1),
|
||||
"proxied container": strings.Replace(strings.Replace(good, `"local"`, `"container"`, 1),
|
||||
`tags = ["dolt-server"]`, `tags = ["dolt-server-proxied"]`, 1),
|
||||
"integration no-remote-exec": strings.Replace(good, `tags = ["dolt-server"]`, `tags = ["dolt-server-integration", "no-remote-exec"]`, 1),
|
||||
} {
|
||||
if bad == good {
|
||||
t.Fatalf("%s: mutation did not apply", name)
|
||||
|
|
@ -2054,6 +2083,260 @@ func TestBazelIntegrationJob(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// bazelrcLines returns .bazelrc's trimmed lines as a set.
|
||||
func bazelrcLines(t *testing.T) map[string]bool {
|
||||
t.Helper()
|
||||
lines := map[string]bool{}
|
||||
for _, line := range strings.Split(readPolicyFile(t, bazelPolicyRoot(t), ".bazelrc"), "\n") {
|
||||
lines[strings.TrimSpace(line)] = true
|
||||
}
|
||||
return lines
|
||||
}
|
||||
|
||||
// assertBazelTierStep checks the test step of a remote-only tier job in
|
||||
// bazel.yml: it runs exactly `bazel test //... --config=<config>` with a BEP,
|
||||
// keeps bazel's exit status, has a timeout below the job's, and is followed
|
||||
// by check_testcases.py on that BEP.
|
||||
func assertBazelTierStep(t *testing.T, job ciWorkflowJob, jobName, config string) {
|
||||
t.Helper()
|
||||
const id, bep, wantIf = "test", "bazel-bep.json", ""
|
||||
step := job.step(t, "bazel test //... --config="+config)
|
||||
if step.ID != id || step.If != wantIf || step.TimeoutMinutes == 0 || step.TimeoutMinutes >= job.TimeoutMinutes {
|
||||
t.Errorf("%s step --config=%s: id=%q if=%q timeout-minutes=%d; want id %q, if %q, a timeout below the job's %d",
|
||||
jobName, config, step.ID, step.If, step.TimeoutMinutes, id, wantIf, job.TimeoutMinutes)
|
||||
}
|
||||
cmd := regexp.MustCompile(`\s*\\\n\s*`).ReplaceAllString(step.Run, " ")
|
||||
wantCmd := `bazel test //... --config=` + config + ` --build_event_json_file="$RUNNER_TEMP/` + bep + `"`
|
||||
if !strings.Contains(cmd, wantCmd) || !strings.Contains(step.Run, "set -o pipefail") || strings.Count(step.Run, "bazel test //") != 1 ||
|
||||
strings.Contains(step.Run, "--config=remote-exec") || strings.Contains(step.Run, "--test_tag_filters") {
|
||||
t.Errorf("%s step --config=%s does not run exactly %q:\n%s", jobName, config, wantCmd, step.Run)
|
||||
}
|
||||
assertTestStepKeepsExitStatus(t, step)
|
||||
found := false
|
||||
for _, s := range job.Steps {
|
||||
if strings.TrimSpace(s.Run) == `python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/`+bep+`"` {
|
||||
found = true
|
||||
if s.If != "${{ always() && steps."+id+".outcome != 'skipped' }}" || (s.ContinueOnError != nil && s.ContinueOnError != false) {
|
||||
t.Errorf("%s: check_testcases.py for %s: if=%q continue-on-error=%v", jobName, bep, s.If, s.ContinueOnError)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("%s: no check_testcases.py step on %s (a shard or selector that runs no tests exits 0)", jobName, bep)
|
||||
}
|
||||
}
|
||||
|
||||
// The proxied-server tier (pr-risk.yml "Test (Proxied Dolt Cmd N/15)"; main.yml's
|
||||
// twin runs the same shard script on a non-race binary) and the server-Dolt storage tier (pr-risk.yml "Test (Server Dolt
|
||||
// Conformance)", "Test (Server Dolt Full Suite N/16)") as Bazel variants:
|
||||
// each manifest-sharded variant runs its jobs' shard script with their shard
|
||||
// total, the conformance variant the job's exact flags, with the jobs' race
|
||||
// setting, subprocess binaries and switches; the lanes' configs keep the
|
||||
// jobs' selection, and the storage lane shares --config=integration's build.
|
||||
// bazel.yml runs both tiers remotely only.
|
||||
func TestBazelDoltServerTiersMirrorPRRisk(t *testing.T) {
|
||||
rc := bazelrcLines(t)
|
||||
for _, want := range []string{
|
||||
"test:doltserver-proxied --@rules_go//go/config:race",
|
||||
"test:doltserver-proxied --test_tag_filters=dolt-server-proxied",
|
||||
"test:doltserver-proxied --test_timeout=-1,-1,-1,1200",
|
||||
"test:doltserver-integration --test_tag_filters=dolt-server-integration",
|
||||
"test:doltserver-integration --test_timeout=-1,-1,-1,1200",
|
||||
} {
|
||||
if !rc[want] {
|
||||
t.Errorf(".bazelrc lacks %q", want)
|
||||
}
|
||||
}
|
||||
var tierTags, integTags map[string]bool
|
||||
for line := range rc {
|
||||
for _, config := range []string{"doltserver-proxied", "doltserver-integration"} {
|
||||
if strings.HasPrefix(line, "test:"+config+" ") && (strings.Contains(line, "-test.short") || strings.Contains(line, "BEADS_TEST_SKIP") ||
|
||||
strings.Contains(line, "-test.run") || strings.Contains(line, "-test.skip") || strings.Contains(line, "--test_filter")) {
|
||||
t.Errorf(".bazelrc %q: the jobs run their scripts' selection, without -short or BEADS_TEST_SKIP", line)
|
||||
}
|
||||
}
|
||||
if v, ok := strings.CutPrefix(line, "build:doltserver-integration --@rules_go//go/config:tags="); ok {
|
||||
tierTags = tagSet(v)
|
||||
}
|
||||
if v, ok := strings.CutPrefix(line, "build:integration --@rules_go//go/config:tags="); ok {
|
||||
integTags = tagSet(v)
|
||||
}
|
||||
}
|
||||
// The same build flags as --config=integration, or the tier compiles the
|
||||
// tagged graph once more.
|
||||
if tierTags == nil || !sameTagSet(tierTags, integTags) {
|
||||
t.Errorf(".bazelrc build:doltserver-integration tags = %v, want build:integration's %v", tierTags, integTags)
|
||||
}
|
||||
if rc["test:integration --@rules_go//go/config:race"] != rc["test:doltserver-integration --@rules_go//go/config:race"] {
|
||||
t.Error(".bazelrc: test:doltserver-integration and test:integration differ in race; they must share their build")
|
||||
}
|
||||
|
||||
risk := readCIWorkflow(t, "pr-risk.yml")
|
||||
build := risk.job(t, "build-embedded")
|
||||
for step, want := range map[string]string{
|
||||
// The proxied jobs' test binary is this race build (--config=
|
||||
// doltserver-proxied's bd_test), their subprocess bd the non-race one
|
||||
// (bd_for_tests).
|
||||
"Build embedded cmd test binary": "go test -tags gms_pure_go -race -c -o /tmp/bd-cmd-test ./cmd/bd/",
|
||||
"Build proxied bd subprocess binary": "go build -tags gms_pure_go -o /tmp/bd-proxied ./cmd/bd/",
|
||||
} {
|
||||
if got := strings.TrimSpace(build.step(t, step).Run); got != want {
|
||||
t.Errorf("pr-risk.yml %q = %q, want %q", step, got, want)
|
||||
}
|
||||
}
|
||||
// main.yml's twin proxied jobs run build-artifacts' bd-cmd-test, which
|
||||
// is not race: bd_proxied_test (race, like PR Risk's) is the stricter of
|
||||
// the two. Pinned so a change there is a decision, not drift.
|
||||
mainYML := readCIWorkflow(t, "main.yml")
|
||||
if run := mainYML.job(t, "build-artifacts").step(t, "Build reusable Linux artifacts").Run; !strings.Contains(run, `go test -tags "$BEADS_BUILD_TAGS" -c -o artifacts/bd-cmd-test ./cmd/bd`+"\n") {
|
||||
t.Errorf("main.yml build-artifacts no longer builds the non-race bd-cmd-test this tier is documented against (.bazelrc, cmd/bd:bd_proxied_test):\n%s", run)
|
||||
}
|
||||
if got := mainYML.job(t, "test-proxied-cmd").step(t, "Test proxied-server cmd shard").Env["BEADS_TEST_CMD_BINARY"]; got != "${{ github.workspace }}/ci-build-artifacts/bd-cmd-test" {
|
||||
t.Errorf("main.yml test-proxied-cmd BEADS_TEST_CMD_BINARY = %q, want build-artifacts' bd-cmd-test", got)
|
||||
}
|
||||
// The server jobs' binary: integration-tagged (the lane's build tags) and
|
||||
// not race (dolt_race_off).
|
||||
m := regexp.MustCompile(`^go test -tags=(\S+) -c -o /tmp/dolt-conformance-test \./internal/storage/dolt/$`).
|
||||
FindStringSubmatch(strings.TrimSpace(build.step(t, "Build server Dolt conformance test binary").Run))
|
||||
if m == nil || !sameTagSet(tagSet(m[1]), tierTags) {
|
||||
t.Errorf("pr-risk.yml's server Dolt test binary is no longer the non-race `go test -tags=<build:doltserver-integration tags> -c`: %v", m)
|
||||
}
|
||||
|
||||
type shardTier struct {
|
||||
workflow, job, step, script, binVar, pkg, target, env string
|
||||
}
|
||||
tiers := []shardTier{
|
||||
{"pr-risk.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
|
||||
{"main.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
|
||||
{"pr-risk.yml", "test-server-storage-full", "Test", ".github/scripts/server-storage-test-shard.sh", "BEADS_TEST_SERVER_TEST_BINARY", "internal/storage/dolt", "dolt_server_full_test", "BEADS_TEST_ENV_RUN_DOLT"},
|
||||
}
|
||||
for _, c := range tiers {
|
||||
j := readCIWorkflow(t, c.workflow).job(t, c.job)
|
||||
shards := len(j.Strategy.Matrix.Shard)
|
||||
step := j.step(t, c.step)
|
||||
if want := "bash " + c.script + " ${{ matrix.shard }} " + strconv.Itoa(shards); strings.TrimSpace(step.Run) != want {
|
||||
t.Errorf("%s %s runs %q, want %q", c.workflow, c.job, step.Run, want)
|
||||
}
|
||||
if step.Env[c.env] != "1" {
|
||||
t.Errorf("%s %s no longer sets %s=1; update %s:%s", c.workflow, c.job, c.env, c.pkg, c.target)
|
||||
}
|
||||
if os.Getenv("TEST_SRCDIR") != "" {
|
||||
continue // scripts_test's runfiles hold no other package's BUILD
|
||||
}
|
||||
root := sourceRepoRoot(t)
|
||||
rule := bazelRuleBlock(readPolicyFile(t, root, c.pkg+"/BUILD.bazel"), c.target)
|
||||
for _, want := range []string{
|
||||
`srcs = ["//tools/bazel:go_test_manifest_shard.sh"],`,
|
||||
`"$(rootpath //:` + c.script + `)",`,
|
||||
`"` + c.binVar + `",`,
|
||||
"shard_count = " + strconv.Itoa(shards) + ",",
|
||||
`timeout = "eternal",`,
|
||||
} {
|
||||
if !strings.Contains(rule, want) {
|
||||
t.Errorf("%s:%s does not contain %q (%s %s):\n%s", c.pkg, c.target, want, c.workflow, c.job, rule)
|
||||
}
|
||||
}
|
||||
mm := shardManifestDefault.FindStringSubmatch(readPolicyFile(t, root, c.script))
|
||||
if mm == nil {
|
||||
t.Fatalf("%s has no ${BEADS_TEST_SHARD_MANIFEST:-...} default manifest", c.script)
|
||||
}
|
||||
data := bazelAttrBlock(rule, "data")
|
||||
for _, file := range []string{c.script, mm[1]} {
|
||||
if !strings.Contains(data, `"//:`+file+`",`) {
|
||||
t.Errorf("%s:%s data lacks //:%s (without the manifest every shard falls back to hashing):\n%s", c.pkg, c.target, file, data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
conf := risk.job(t, "test-server-storage").step(t, "Test").Run
|
||||
quoted := regexp.MustCompile(`(-test\.[a-z]+) '([^']*)'|(-test\.[a-z]+=\S+|-test\.v)`)
|
||||
fields := strings.Fields(conf)
|
||||
if len(fields) == 0 || fields[0] != "/tmp/dolt-conformance-test" {
|
||||
t.Fatalf("pr-risk.yml test-server-storage no longer runs /tmp/dolt-conformance-test: %q", conf)
|
||||
}
|
||||
var wantArgs []string
|
||||
for _, m := range quoted.FindAllStringSubmatch(conf, -1) {
|
||||
if m[1] != "" {
|
||||
wantArgs = append(wantArgs, `"`+m[1]+"="+strings.ReplaceAll(m[2], "$", "$$")+`",`)
|
||||
} else {
|
||||
wantArgs = append(wantArgs, `"`+m[3]+`",`)
|
||||
}
|
||||
}
|
||||
if len(wantArgs) != 4 {
|
||||
t.Fatalf("parsed %v from pr-risk.yml test-server-storage %q", wantArgs, conf)
|
||||
}
|
||||
|
||||
if os.Getenv("TEST_SRCDIR") == "" {
|
||||
root := sourceRepoRoot(t)
|
||||
doltBuild := readPolicyFile(t, root, "internal/storage/dolt/BUILD.bazel")
|
||||
rule := bazelRuleBlock(doltBuild, "dolt_server_conformance_test")
|
||||
for _, w := range append(wantArgs, `"$(rootpath :dolt_race_off)",`, `srcs = ["//tools/bazel:go_test_variant.sh"],`) {
|
||||
if !strings.Contains(bazelAttrBlock(rule, "args")+rule, w) {
|
||||
t.Errorf("dolt:dolt_server_conformance_test does not contain %q (pr-risk.yml test-server-storage):\n%s", w, rule)
|
||||
}
|
||||
}
|
||||
if strings.Contains(rule, "BEADS_TEST_ENV_RUN_DOLT") {
|
||||
t.Error("dolt:dolt_server_conformance_test sets BEADS_TEST_ENV_RUN_DOLT; test-server-storage does not")
|
||||
}
|
||||
if r := bazelRuleBlock(doltBuild, "dolt_race_off"); !strings.Contains(r, "go_test_race_off(") || !strings.Contains(r, `test = ":dolt_test",`) {
|
||||
t.Errorf("dolt:dolt_race_off must be go_test_race_off of :dolt_test (the jobs' binary is not race):\n%s", r)
|
||||
}
|
||||
full := bazelRuleBlock(doltBuild, "dolt_server_full_test")
|
||||
for _, w := range []string{`"$(rootpath :dolt_race_off)",`, `"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath :dolt_race_off)"`, `"//:go.mod",`} {
|
||||
if !strings.Contains(full, w) {
|
||||
t.Errorf("dolt:dolt_server_full_test lacks %q (SubprocessRunner reuses the test binary; ModuleRoot needs go.mod):\n%s", w, full)
|
||||
}
|
||||
}
|
||||
proxied := bazelRuleBlock(readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "bd_proxied_test")
|
||||
for _, w := range []string{`"$(rootpath :bd_test)",`, `"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)"`} {
|
||||
if !strings.Contains(proxied, w) {
|
||||
t.Errorf("cmd/bd:bd_proxied_test lacks %q (race bd_test, non-race subprocess bd, like the jobs):\n%s", w, proxied)
|
||||
}
|
||||
}
|
||||
for pkg, build := range map[string]string{"cmd/bd": readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "internal/storage/dolt": doltBuild} {
|
||||
for _, err := range checkDoltServerRules(pkg, build) {
|
||||
t.Error(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Each tier is a remote-only job of its own (the job if is pinned by
|
||||
// TestBazelWorkflowIsAdvisory via bazelRemoteOnlyJobs): not a step of
|
||||
// bazel-doltserver (which also runs locally, and whose job a gate may
|
||||
// require) or of bazel-integration (which a caller may switch off,
|
||||
// although the server tier is a PR-time tier).
|
||||
workflow := readCIWorkflow(t, bazelWorkflowName)
|
||||
for _, c := range []struct{ job, config, logs string }{
|
||||
{bazelProxiedJobName, "doltserver-proxied", "bazel-proxied-testlogs"},
|
||||
{bazelServerJobName, "doltserver-integration", "bazel-server-storage-testlogs"},
|
||||
} {
|
||||
job := workflow.job(t, c.job)
|
||||
if job.TimeoutMinutes == 0 || job.TimeoutMinutes > 30 {
|
||||
t.Errorf("%s timeout-minutes = %d; it runs remotely only (longest shard ~2-8 min), keep it at most 30", c.job, job.TimeoutMinutes)
|
||||
}
|
||||
assertBazelTierStep(t, job, c.job, c.config)
|
||||
if n := len(job.Steps); n != 5 {
|
||||
t.Errorf("%s has %d steps, want checkout, setup-bazel, the tier, check_testcases.py, log upload", c.job, n)
|
||||
}
|
||||
logs := job.step(t, "Upload test logs")
|
||||
if logs.If != "${{ failure() && steps.test.outcome != 'skipped' }}" || logs.With["name"] != c.logs || !strings.HasPrefix(logs.Uses, "actions/upload-artifact@") {
|
||||
t.Errorf("%s test-log upload: if=%q name=%q uses=%q", c.job, logs.If, logs.With["name"], logs.Uses)
|
||||
}
|
||||
}
|
||||
for name, j := range workflow.Jobs {
|
||||
for _, step := range j.Steps {
|
||||
proxied := strings.Contains(step.Run, "--config=doltserver-proxied")
|
||||
server := strings.Contains(step.Run, "--config=doltserver-integration")
|
||||
if (proxied && name != bazelProxiedJobName) || (server && name != bazelServerJobName) {
|
||||
t.Errorf("%s step %q runs a dolt-server tier outside its own job", name, step.Name)
|
||||
}
|
||||
if (name == bazelIntegJobName || name == bazelDoltJobName) && (proxied || server || strings.Contains(step.Run, "doltserver-")) {
|
||||
t.Errorf("%s step %q runs the proxied or server storage tier; each has its own job", name, step.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// bazel-embedded replaces pr-risk.yml's embedded-Dolt tier: --config=embedded
|
||||
// runs the embedded variants race like the jobs' binaries, with the jobs'
|
||||
// parallelism; each shard variant runs its job's shard script with the job's
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
#!/usr/bin/env bash
|
||||
# Runs one Bazel shard of an existing go_test binary through a PR Risk shard
|
||||
# script (.github/scripts/embedded-test-shard.sh and
|
||||
# embedded-storage-test-shard.sh), so the Bazel target runs exactly the tests
|
||||
# script (.github/scripts/embedded-test-shard.sh,
|
||||
# embedded-storage-test-shard.sh, proxied-test-shard.sh,
|
||||
# server-storage-test-shard.sh), so the Bazel target runs exactly the tests
|
||||
# the CI job with the same shard number does: the same discovery, the same
|
||||
# committed manifest and hash fallback, the same binary flags. Like
|
||||
# go_test_variant.sh it reuses the go_test's binary instead of compiling the
|
||||
|
|
@ -14,7 +15,8 @@
|
|||
# and where the script's source globs (cmd/bd/*_embedded_test.go, ...)
|
||||
# resolve.
|
||||
# <binary env var>: the variable the script reads the binary path from
|
||||
# (BEADS_TEST_CMD_BINARY or BEADS_TEST_EMBEDDED_TEST_BINARY); set to an
|
||||
# (BEADS_TEST_CMD_BINARY, BEADS_TEST_EMBEDDED_TEST_BINARY or
|
||||
# BEADS_TEST_SERVER_TEST_BINARY); set to an
|
||||
# absolute path, since tests re-exec it from their package directory.
|
||||
# The remaining arguments (and any --test_arg) follow the script's own
|
||||
# binary flags.
|
||||
|
|
|
|||
|
|
@ -55,6 +55,14 @@ trap 'reap_root_processes 2>/dev/null || true; chmod -R u+w "$root" 2>/dev/null
|
|||
|
||||
mkdir -p "$root/home" "$root/xdg-config" "$root/dolt-root" "$root/tmp"
|
||||
: >"$root/gitconfig"
|
||||
# The global git identity of the CI jobs that run `git config --global
|
||||
# user.name/user.email` before their tests (PR Risk's server-Dolt and
|
||||
# proxied-server jobs): a target mirroring such a job sets
|
||||
# BEADS_TEST_GIT_IDENTITY=1 in its env. Everything else keeps pr-core's empty
|
||||
# global gitconfig.
|
||||
if [[ "${BEADS_TEST_GIT_IDENTITY:-}" == 1 ]]; then
|
||||
printf '[user]\n\tname = CI Bot\n\temail = ci@beads.test\n' >"$root/gitconfig"
|
||||
fi
|
||||
# Dolt identity, as beads_test_env_enter sets with `dolt config --global`:
|
||||
# tests that shell out to dolt commit need an author. Written directly so the
|
||||
# wrapper does not depend on a dolt binary. The two *.disabled keys stop every
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue