ci(bazel): server-Dolt storage and proxied tiers on remote execution (#6940)

* test(integration): let SubprocessRunner reuse a prebuilt test binary

BEADS_TEST_SUBPROCESS_BINARY hands SubprocessRunner an already-built test
binary of the package instead of running `go test -c`, which no Bazel
action can do offline (it needs the module source tree and module cache).
A Bazel target passes the running go_test itself: the helper tests
(TestHelperSchemaInit, TestHelperMultiStore) live in the same package.
Unset, behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(dolt): keep a lane's REQUIRE switch out of the leak-probe child

TestSuiteFixtureLeakFailsOwningSuite re-runs the suite with
BEADS_TEST_SKIP=dolt on purpose. When the parent runs under
BEADS_TEST_REQUIRE_DOLT_CONTAINER=1 (a fail-closed server lane), the child
inherited it and TestMain exited FATAL before the leak sweep the test
checks. Clear it in the child like the other suite-scoped variables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(bazel): proxied-server and server-Dolt storage tiers (D6d, D6e)

PR Risk's "Test (Proxied Dolt Cmd N/15)" (also main.yml's) and "Test
(Server Dolt Conformance)" / "Test (Server Dolt Full Suite N/16)" as
Bazel variants against local dolt sql-servers (internal/testutil local
backend, pinned dolt from the test wrapper), fail-closed like the
dolt-server lane:

- //cmd/bd:bd_proxied_test: 15 shards; shard k runs the job's
  proxied-test-shard.sh k+1 15 through go_test_manifest_shard.sh over the
  race bd_test, non-race bd_for_tests as the subprocess bd,
  BEADS_TEST_PROXIED_SERVER=1, GOMAXPROCS=4. Tag dolt-server-proxied,
  --config=doltserver-proxied (the doltserver lane's build).
- //internal/storage/dolt:dolt_server_{conformance,full}_test over
  dolt_race_off (go_test_race_off of dolt_test: the jobs' binary is not
  race); the full suite runs server-storage-test-shard.sh k+1 16 with
  BEADS_TEST_ENV_RUN_DOLT=1 and hands SubprocessRunner its own binary.
  Tag dolt-server-integration, --config=doltserver-integration, whose
  build flags are exactly --config=integration's, so the tier shares that
  lane's compile (race-off deps land where bd_for_tests already is)
  instead of compiling the tagged graph a third time.
- dolt_test gains //internal/storage/schema:migrations as data (the
  PR4107/migration replay tests read ../schema/migrations).
- test_env.sh: BEADS_TEST_GIT_IDENTITY=1 writes the jobs' global git
  identity ("Configure Git"); the git-remote tests commit in fresh repos.
- Both tags join the taxonomy and bazelPRCoreExcludedTags, so prcore/ci
  and integration exclude them.

Per-shard names and statuses equal a replay of the jobs' commands
(container backend) for all 32 jobs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the proxied and server storage tiers remotely (advisory)

bazel-doltserver gains a second, remote-only step, `bazel test //...
--config=doltserver-proxied` (15 shards of ~5-10 min with a busy dolt
sql-server each do not fit a GitHub-hosted runner; forks and rbe=off keep
the local lane unchanged). bazel-integration gains `bazel test //...
--config=doltserver-integration`, which shares its build flags, so the
server storage tier compiles nothing the integration step has not. Both
steps write their own BEP, run check_testcases.py on it, run even when the
job's first test step failed, and upload logs on failure.

Policy (TestBazelDoltServerTiersMirrorPRRisk): each variant runs its jobs'
shard script with their shard total and has script + manifest in data;
the conformance variant passes the job's flags; the jobs' race/non-race
binaries and switches (BEADS_TEST_PROXIED_SERVER, BEADS_TEST_ENV_RUN_DOLT)
match; doltserver-integration's tags and race equal --config=integration's;
the steps are pinned to their jobs. checkDoltServerRules now holds all
three dolt-server lane tags to the local-backend, fail-closed,
remote-capable rules.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the proxied and server storage tiers in jobs of their own

Review of D6d/e: as steps of bazel-doltserver and bazel-integration the
tiers inherited those jobs' gating (a caller switching the integration
lane off would drop PR Risk's PR-time server tier; a gate requiring the
dolt-server lane would silently require the proxied tier too) and
lengthened their critical paths.

bazel-proxied (--config=doltserver-proxied, 15 shards) and
bazel-server-storage (--config=doltserver-integration: conformance + 16
shards) are remote-only jobs that need only rbe, each with its own BEP,
check_testcases.py step and log upload, 25-minute test step inside a
30-minute job. bazel-doltserver and bazel-integration are back to their
previous content, including the dolt-server step's 80-minute timeout for
local runs. The server tier shares bazel-integration's compiles through
the remote cache (same build flags).

The proxied tier mirrors PR Risk's race bd-cmd-test; main.yml's twin
job runs build-artifacts' non-race one, so Bazel is the stricter of the
two. Comments now say so, and the policy test pins main.yml's build so a
change there is noticed. Policy: the new jobs are pinned (command,
config, remote-only if, BEP + testcase steps, timeouts, log upload), and
no tier step may run in bazel-integration or bazel-doltserver.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Julian Knutsen 2026-09-28 23:01:40 -07:00 • committed by GitHub
commit e7138e93af
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 707 additions and 36 deletions

View file

@ -37,7 +37,7 @@ test:prcore --test_env=BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION=1
# Every tag that belongs to another lane (scripts/bazel_policy_test.go
# bazelPRCoreExcludedTags). integration-only targets run zero tests outside
# --config=integration.
test:prcore --test_tag_filters=-requires-docker,-dolt-server,-embedded,-manual,-integration-only
test:prcore --test_tag_filters=-requires-docker,-dolt-server,-dolt-server-proxied,-dolt-server-integration,-embedded,-manual,-integration-only
test:ci --config=prcore
test:ci --keep_going --test_summary=terse
@ -123,6 +123,57 @@ test:doltserver --test_arg=-test.parallel=4
test:doltserver --local_test_jobs=4
test:doltserver --remote_download_regex=.*/test\.(log|xml)$
# Proxied-server tier: PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs, as
# //cmd/bd:bd_proxied_test (dolt-server-proxied tag). Kept out of
# --config=doltserver because bazel.yml runs that lane locally too (forks,
# rbe=off), where 15 shards of ~10 minutes, each with a dolt sql-server busy
# on ~4 cores, do not fit; bazel.yml runs this one only with remote
# execution, in a job of its own (bazel-proxied). The build is
# --config=doltserver's: race, like PR Risk's /tmp/bd-cmd-test (main.yml's
# twin job runs build-artifacts' non-race bd-cmd-test, so this is the
# stricter of the two); the subprocess bd is the non-race bd_for_tests, like
# the jobs'. Each shard runs its job's
# shard script, whose -test.timeout=15m is past "long": the target is
# eternal, which rbe-west caps at 1200s.
test:doltserver-proxied --@rules_go//go/config:race
test:doltserver-proxied --test_tag_filters=dolt-server-proxied
test:doltserver-proxied --build_tests_only
test:doltserver-proxied --keep_going
test:doltserver-proxied --test_summary=terse
test:doltserver-proxied --test_timeout=-1,-1,-1,1200
test:doltserver-proxied --test_env=GO_TEST_WRAP_TESTV=1
# The jobs pass no -parallel: GOMAXPROCS (the target's 4, a runner's vCPUs).
test:doltserver-proxied --test_arg=-test.parallel=4
test:doltserver-proxied --local_test_jobs=4
test:doltserver-proxied --remote_download_regex=.*/test\.(log|xml)$
# Server-Dolt storage tier: PR Risk's "Test (Server Dolt Conformance)" and
# "Test (Server Dolt Full Suite N/16)" jobs, as
# //internal/storage/dolt:dolt_server_{conformance,full}_test
# (dolt-server-integration tag): the integration-tagged internal/storage/dolt
# tests, non-race like the jobs' binary, against a local dolt sql-server.
# The build flags are exactly --config=integration's (the tags, and race for
# everything the variants do not turn off), so the two lanes share every
# compile: the variants' race-off binary lands in the configuration
# --config=integration already builds the non-race //cmd/bd:bd_for_tests in.
# A per-target gotags transition would instead compile the whole graph a
# third time. No BEADS_TEST_SKIP, no -short; the full suite's
# BEADS_TEST_ENV_RUN_DOLT=1 is target env. Remote only in bazel.yml, in a
# job of its own (bazel-server-storage) that shares bazel-integration's
# compiles through the remote cache.
build:doltserver-integration --@rules_go//go/config:tags=gms_pure_go,integration
test:doltserver-integration --@rules_go//go/config:race
test:doltserver-integration --test_tag_filters=dolt-server-integration
test:doltserver-integration --build_tests_only
test:doltserver-integration --keep_going
test:doltserver-integration --test_summary=terse
test:doltserver-integration --test_timeout=-1,-1,-1,1200
test:doltserver-integration --test_env=GO_TEST_WRAP_TESTV=1
# The jobs pass no -parallel: GOMAXPROCS (the targets' 4, a runner's vCPUs).
test:doltserver-integration --test_arg=-test.parallel=4
test:doltserver-integration --local_test_jobs=4
test:doltserver-integration --remote_download_regex=.*/test\.(log|xml)$
# Integration lane: main.yml's "Main Linux integration packages (N/6)" and
# "Main Linux integration cmd/bd (N/8)" jobs (and nightly.yml's Full Test
# Suite, which runs the same `go test` over ./... in one job): every package
@ -141,7 +192,7 @@ test:integration --test_env=BEADS_TEST_SKIP=dolt
# keeps every container- or server-backed test skipped, which is what the
# untagged base targets already do. scripts/bazel_policy_test.go derives the
# required exclusions from the PR-core list, so a new lane tag lands here too.
test:integration --test_tag_filters=-requires-docker,-dolt-server,-embedded,-manual
test:integration --test_tag_filters=-requires-docker,-dolt-server,-dolt-server-proxied,-dolt-server-integration,-embedded,-manual
test:integration --build_tests_only
test:integration --keep_going
test:integration --test_summary=terse

View file

@ -1,8 +1,9 @@
# Bazel side-by-side lane (advisory). Runs `bazel test //... --config=ci`, the
# PR Core-parity selection (race, -short, dolt skip, -requires-docker), on the
# same commit as pr.yml's go test lanes, plus one job per mirrored CI job
# (pure-Go/js-wasm, docker lane, pr-risk.yml's embedded-Dolt tier, main.yml's
# integration jobs). It is NOT
# (pure-Go/js-wasm, the dolt-server lane, pr-risk.yml's embedded-Dolt,
# proxied-server and server-Dolt storage tiers, main.yml's integration jobs).
# It is NOT
# in ci-gate's CI_GATE_REQUIRED; promotion needs the equivalence record
# described in tools/bazel/equivalence.py and a human decision.
#
@ -17,8 +18,8 @@
# local: fork PRs, rbe=off dispatches, and same-repo runs without the
# secret (Dependabot PRs: same-repo, but no Actions secrets): local
# execution on ubuntu-latest, slower but the same results, for every job
# but the remote-only ones (bazel-embedded, bazel-integration), which
# are skipped.
# but the remote-only ones (bazel-embedded, bazel-integration,
# bazel-proxied, bazel-server-storage), which are skipped.
# skip: same-repo runs while RBE_WEST_WORKERS is unset (the farm switch).
# workflow_dispatch with rbe=off simulates a fork run. The decision is also a
# workflow_call output, so a caller's gate can tell an expected skip from a
@ -677,3 +678,126 @@ jobs:
bazel-testlogs/**/test.xml
if-no-files-found: ignore
retention-days: 7
# pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)", as
# --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the job's
# shard script in 15 Bazel shards, so shard k runs exactly job k's tests.
# Same build as --config=doltserver (race bd_test, non-race bd_for_tests),
# shared with bazel-doltserver through the remote cache. PR Risk builds
# its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd twin
# uses build-artifacts' non-race bd-cmd-test, so this lane is the stricter
# of the two. A job of its own, not gated by any integration switch, so a
# caller's gate can decide on it separately.
bazel-proxied:
name: Bazel / proxied-server tier (advisory)
# Remote execution only, like bazel-embedded: 15 shards of ~5-10
# minutes, each with a dolt sql-server busy on ~4 cores, do not fit a
# GitHub-hosted runner.
needs: rbe
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
if: ${{ needs.rbe.outputs.enabled == 'true' }}
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
- name: Set up Bazel
id: bazel
uses: ./.github/actions/setup-bazel
env:
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
# Remote: the longest shard takes ~5-8 minutes (each shard's Go
# timeout is its job's 15m, 1200s Bazel cap), plus a compile.
- name: bazel test //... --config=doltserver-proxied
id: test
timeout-minutes: 25
run: |
set -o pipefail
start=$(date +%s)
rc=0
bazel test //... --config=doltserver-proxied \
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
echo "bazel test --config=doltserver-proxied: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
exit "$rc"
# As in bazel-embedded.
- name: Every target and shard ran tests
if: ${{ always() && steps.test.outcome != 'skipped' }}
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
- name: Upload test logs
if: ${{ failure() && steps.test.outcome != 'skipped' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: bazel-proxied-testlogs
path: |
bazel-testlogs/**/test.log
bazel-testlogs/**/test.xml
if-no-files-found: ignore
retention-days: 7
# pr-risk.yml's server-Dolt storage tier, "Test (Server Dolt Conformance)"
# and "Test (Server Dolt Full Suite N/16)", as --config=doltserver-integration:
# the conformance and 16-shard full-suite variants of
# //internal/storage/dolt over a non-race dolt_test build, each shard
# running the job's own shard script. --config=doltserver-integration has
# exactly --config=integration's build flags, so it compiles nothing
# bazel-integration does not (shared through the remote cache). A job of
# its own, not gated by any integration switch: this is a PR-time tier,
# unlike main.yml's integration jobs.
bazel-server-storage:
name: Bazel / server-Dolt storage tier (advisory)
# Remote execution only, like bazel-integration: locally the tagged race
# build of //... alone is most of the timeout.
needs: rbe
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
if: ${{ needs.rbe.outputs.enabled == 'true' }}
timeout-minutes: 30
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
- name: Set up Bazel
id: bazel
uses: ./.github/actions/setup-bazel
env:
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
# Remote: the longest shard takes ~2-3 minutes (the storage suite's Go
# timeout is 15m per shard, 1200s Bazel cap), plus a compile.
- name: bazel test //... --config=doltserver-integration
id: test
timeout-minutes: 25
run: |
set -o pipefail
start=$(date +%s)
rc=0
bazel test //... --config=doltserver-integration \
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
echo "bazel test --config=doltserver-integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
exit "$rc"
# As in bazel-embedded.
- name: Every target and shard ran tests
if: ${{ always() && steps.test.outcome != 'skipped' }}
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
- name: Upload test logs
if: ${{ failure() && steps.test.outcome != 'skipped' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: bazel-server-storage-testlogs
path: |
bazel-testlogs/**/test.log
bazel-testlogs/**/test.xml
if-no-files-found: ignore
retention-days: 7

View file

@ -69,9 +69,11 @@ filegroup(
visibility = ["//scripts:__pkg__"],
)
# PR Risk's embedded-Dolt shard scripts and their committed manifests. The
# embedded variants (//cmd/bd:bd_embedded_test,
# //internal/storage/embeddeddolt:embeddeddolt_embedded_test) run them through
# PR Risk's manifest-sharded tiers' shard scripts and their committed
# manifests (embedded-Dolt cmd and storage, proxied-server cmd, server-Dolt
# storage). The variants (//cmd/bd:bd_embedded_test, //cmd/bd:bd_proxied_test,
# //internal/storage/embeddeddolt:embeddeddolt_embedded_test,
# //internal/storage/dolt:dolt_server_full_test) run them through
# tools/bazel/go_test_manifest_shard.sh, so each Bazel shard runs exactly the
# tests of the CI job with the same shard number.
exports_files(
@ -80,9 +82,14 @@ exports_files(
".github/scripts/embedded-storage-test-shard.sh",
".github/scripts/embedded-storage-test-shards.txt",
".github/scripts/embedded-test-shard.sh",
".github/scripts/proxied-cmd-test-shards.txt",
".github/scripts/proxied-test-shard.sh",
".github/scripts/server-storage-test-shard.sh",
".github/scripts/server-storage-test-shards.txt",
],
visibility = [
"//cmd/bd:__pkg__",
"//internal/storage/dolt:__pkg__",
"//internal/storage/embeddeddolt:__pkg__",
],
)

View file

@ -1326,6 +1326,62 @@ sh_test(
tags = ["embedded"],
)
# bd_test as PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs run it: shard k
# runs .github/scripts/proxied-test-shard.sh k+1 15 (the TestProxiedServer*
# functions, assigned by the committed manifest) against this configuration's
# (race) bd_test with BEADS_TEST_PROXIED_SERVER=1 and the non-race
# bd_for_tests as the subprocess binary, like the jobs' /tmp/bd-cmd-test and
# /tmp/bd-proxied. (main.yml's twin job runs the same shard script against
# build-artifacts' non-race bd-cmd-test; race here is the stricter one.)
# The shared proxied harness gets its dolt sql-server from
# internal/testutil's local backend (the pinned dolt in the test wrapper's
# runfiles) instead of the jobs' container, and fails instead of skipping
# when it cannot start one. It repeats bd_test's `env`. shard_count must
# stay equal to the manifest's shard total.
#
# Tags:
# dolt-server-proxied: the proxied-server tier; starts hermetic dolt
# sql-servers; excluded from --config=prcore/ci, selected by
# --config=doltserver-proxied.
sh_test(
name = "bd_proxied_test",
size = "large",
# The script's -test.timeout=15m fires (with a goroutine dump) before
# Bazel's 1200s action limit (eternal under the lane's --test_timeout).
timeout = "eternal",
srcs = ["//tools/bazel:go_test_manifest_shard.sh"],
args = [
"$(rootpath //:.github/scripts/proxied-test-shard.sh)",
"BEADS_TEST_CMD_BINARY",
"$(rootpath :bd_test)",
],
data = [
":bd_for_tests",
":bd_test",
":go_srcs",
"//:.github/scripts/proxied-cmd-test-shards.txt",
"//:.github/scripts/proxied-test-shard.sh",
"@go_sdk//:bin/gofmt",
],
env = {
"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)",
"BEADS_TEST_DOLT_SERVER": "local",
# The jobs' "Configure Git and Dolt identity" global git identity
# (tools/bazel/test_env.sh).
"BEADS_TEST_GIT_IDENTITY": "1",
"BEADS_TEST_GOFMT": "$(rlocationpath @go_sdk//:bin/gofmt)",
"BEADS_TEST_PROXIED_SERVER": "1",
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
# The 4-vCPU runner shape: -test.parallel defaults to GOMAXPROCS, and
# the job runs 4 tests at once, not one per worker core (39
# concurrent `bd init --proxied-server` against one server hung a
# 192-core worker).
"GOMAXPROCS": "4",
},
shard_count = 15,
tags = ["dolt-server-proxied"],
)
# CLI flag registrations pinned by //internal/httpapi spec parity tests.
exports_files(
[

View file

@ -104,6 +104,15 @@ suites of pr.yml's "Test (storage domain + uow)" and "Contract corpus" jobs
on the `local` backend (the `dolt-server` targets); they need no docker and
execute remotely with `--config=remote-exec`. `--config=docker` runs the same
suites on the `container` backend (host docker) as the A/B control.
PR Risk's heavier server tiers have configs of their own, run by bazel.yml
only with remote execution, each in a job of its own (`bazel-proxied`,
`bazel-server-storage`): `--config=doltserver-proxied` is the
proxied-server cmd/bd tier ("Test (Proxied Dolt Cmd N/15)",
`//cmd/bd:bd_proxied_test`), and `--config=doltserver-integration` the
server-Dolt storage tier ("Test (Server Dolt Conformance)", "Test (Server
Dolt Full Suite N/16)", `//internal/storage/dolt:dolt_server_*_test`), which
builds with the integration tag like `--config=integration`. Each shard
runs its CI job's shard script, so Bazel shard k runs the tests of job k+1.
Tests that need a temporary repository or store should use `t.TempDir()` and
`t.Cleanup()`. Temporary repositories must set a repository-local hooks path;

View file

@ -1,4 +1,6 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")
load("@rules_shell//shell:sh_test.bzl", "sh_test")
load("//tools/bazel:race.bzl", "go_test_race_off")
go_library(
name = "dolt",
@ -116,15 +118,13 @@ go_library(
],
)
# Under --config=integration, TestMultiProcessSchemaInit* and
# TestMultiStoreConcurrent_Subprocess build this package with a real
# `go test -c` (integration.SubprocessRunner). That needs the module's source
# tree and every dependency's module (dolt, go-mysql-server, ...) in a module
# cache, which no action has offline, so unlike //internal/testutil/integration
# (whose build fails on purpose) this target gets no BEADS_TEST_GO_BINARY/SDK
# data. Fine while the lane keeps BEADS_TEST_SKIP=dolt (main.yml's integration
# jobs), which skips those tests; a dolt-backed integration variant must build
# the helper binary with Bazel instead.
# TestMultiProcessSchemaInit* and TestMultiStoreConcurrent_Subprocess build
# this package with a real `go test -c` (integration.SubprocessRunner), which
# needs the module's source tree and a module cache no action has. They run
# only against a server: skipped under --config=integration
# (BEADS_TEST_SKIP=dolt, like main.yml's integration jobs), and run by
# dolt_server_full_test, which hands SubprocessRunner its own test binary
# (BEADS_TEST_SUBPROCESS_BINARY) instead.
go_test(
name = "dolt_test",
srcs = [
@ -354,10 +354,12 @@ go_test(
"workspace_config_contract_test.go",
],
# Source-scan guards (journal/version scope completeness, helper sentinel
# census) parse this package and ../issueops at test time.
# census) parse this package and ../issueops at test time; the PR4107 and
# migration replay tests (server-backed) read ../schema/migrations.
data = [
":go_srcs",
"//internal/storage/issueops:go_srcs",
"//internal/storage/schema:migrations",
],
embed = [":dolt"],
deps = [
@ -454,6 +456,102 @@ go_test(
}),
)
# dolt_test without the race detector: PR Risk's server-Dolt jobs run
# build-embedded's /tmp/dolt-conformance-test, built by `go test -c` with the
# integration and gms_pure_go tags but without -race. tools/bazel/race.bzl lands
# its deps in the configuration the non-race //cmd/bd:bd_for_tests is built
# in; under --config=doltserver-integration (the integration lane's build
# flags) that is the configuration --config=integration builds bd_for_tests
# in, so the server tier compiles no third copy of the graph.
#
# Tags:
# manual: only the dolt_server_* variants' input, never a build target of
# its own.
go_test_race_off(
name = "dolt_race_off",
tags = ["manual"],
test = ":dolt_test",
)
# PR Risk's "Test (Server Dolt Conformance)" job: conformance.RunAll against
# server-mode Dolt, with the job's binary flags. TestMain's server is a local
# `dolt sql-server` started from the pinned dolt in the test wrapper's
# runfiles instead of the job's container, and a server that cannot start
# fails the target (BEADS_TEST_REQUIRE_DOLT_CONTAINER=1) instead of skipping
# the suite. It repeats dolt_test's `env` (it has none).
#
# Tags:
# dolt-server-integration: starts hermetic dolt sql-servers and needs the
# integration build tag; excluded from --config=prcore/ci and
# --config=integration, selected by --config=doltserver-integration.
sh_test(
name = "dolt_server_conformance_test",
size = "large",
# The job's -test.timeout=15m fires (with a goroutine dump) well before
# Bazel's 1200s action limit (eternal under the lane's --test_timeout).
timeout = "eternal",
srcs = ["//tools/bazel:go_test_variant.sh"],
args = [
"$(rootpath :dolt_race_off)",
"-test.v",
"-test.count=1",
"-test.timeout=15m",
"-test.run=^TestConformance$$",
],
data = [":dolt_race_off"],
env = {
"BEADS_TEST_DOLT_SERVER": "local",
# The job's "Configure Git" global identity (tools/bazel/test_env.sh).
"BEADS_TEST_GIT_IDENTITY": "1",
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
# The 4-vCPU runner shape: -test.parallel defaults to GOMAXPROCS.
"GOMAXPROCS": "4",
},
tags = ["dolt-server-integration"],
)
# PR Risk's "Test (Server Dolt Full Suite N/16)" jobs: shard k runs
# .github/scripts/server-storage-test-shard.sh k+1 16 (every top-level test
# but TestConformance, assigned by the committed manifest) against
# dolt_race_off, with the jobs' BEADS_TEST_ENV_RUN_DOLT=1 and a local dolt
# sql-server as in dolt_server_conformance_test. SubprocessRunner's helper
# processes re-exec this same binary (BEADS_TEST_SUBPROCESS_BINARY) where the
# job compiles the package again with `go test -tags integration -c`; go.mod
# lets integration.ModuleRoot find the module root it no longer builds in.
# shard_count must stay equal to the manifest's shard total.
#
# Tags:
# dolt-server-integration: as dolt_server_conformance_test.
sh_test(
name = "dolt_server_full_test",
size = "large",
# The script's -test.timeout=15m fires before Bazel's 1200s limit.
timeout = "eternal",
srcs = ["//tools/bazel:go_test_manifest_shard.sh"],
args = [
"$(rootpath //:.github/scripts/server-storage-test-shard.sh)",
"BEADS_TEST_SERVER_TEST_BINARY",
"$(rootpath :dolt_race_off)",
],
data = [
":dolt_race_off",
":go_srcs",
"//:.github/scripts/server-storage-test-shard.sh",
"//:.github/scripts/server-storage-test-shards.txt",
"//:go.mod",
],
env = {
"BEADS_TEST_DOLT_SERVER": "local",
"BEADS_TEST_ENV_RUN_DOLT": "1",
"BEADS_TEST_GIT_IDENTITY": "1",
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath :dolt_race_off)",
"GOMAXPROCS": "4",
},
shard_count = 16,
tags = ["dolt-server-integration"],
)
# --- begin go_srcs (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
filegroup(

View file

@ -12,6 +12,7 @@ import (
"time"
"github.com/steveyegge/beads/internal/doltserver"
"github.com/steveyegge/beads/internal/testutil"
"github.com/steveyegge/beads/internal/testutil/bazeltest"
)
@ -67,6 +68,10 @@ func TestSuiteFixtureLeakFailsOwningSuite(t *testing.T) {
defer writeEnd.Close()
t.Setenv("BEADS_SUITE_LEAK_PROBE", "1")
t.Setenv("BEADS_TEST_SKIP", "dolt")
// The child skips Dolt on purpose, so a lane's fail-closed switch (set
// when this suite runs against a required server) must not turn that
// skip into a FATAL before the child reaches its leak sweep.
t.Setenv(testutil.EnvRequireDoltContainer, "")
t.Setenv(doltserver.AllowLeakEnv, "")
// This child is a complete suite, not one of the schema-init helpers
// which deliberately bypass suite ownership and shutdown.

View file

@ -133,10 +133,32 @@ func goBinary() (string, error) {
return bazeltest.RunfileEnv(GoBinaryEnv)
}
// PrebuiltTestBinaryEnv names a variable that hands SubprocessRunner an
// already-built test binary of the package it would compile, instead of
// running `go test -c`. Under Bazel no action has the module's source tree
// and module cache that `go test -c` needs, but the test binary that is
// running is that package's test binary, integration-tagged, so the target
// passes it (resolved like any other runfile):
//
// env = {"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath <the go_test>)"},
const PrebuiltTestBinaryEnv = "BEADS_TEST_SUBPROCESS_BINARY"
// Build compiles the test binary (once). Returns the path to the binary.
// With BEADS_TEST_SUBPROCESS_BINARY set it compiles nothing and returns that
// binary.
func (r *SubprocessRunner) Build(t *testing.T) string {
t.Helper()
r.once.Do(func() {
if os.Getenv(PrebuiltTestBinaryEnv) != "" {
bin, err := bazeltest.RunfileEnv(PrebuiltTestBinaryEnv)
if err != nil {
r.buildErr = fmt.Errorf("prebuilt test binary: %w", err)
return
}
t.Logf("SubprocessRunner: using prebuilt test binary %s (%s) for %s", bin, PrebuiltTestBinaryEnv, r.pkg)
r.testBin = bin
return
}
r.testBin = filepath.Join(t.TempDir(), "integration.test")
t.Logf("SubprocessRunner: building test binary: go test -tags %s -c -o %s %s", r.tags, r.testBin, r.pkg)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)

View file

@ -679,8 +679,14 @@ var allowedBazelTestTags = map[string]string{
// pinned dolt in its runfiles, so it runs on any worker, and a server that
// cannot start fails it rather than skipping, so its cached result holds.
"dolt-server": "starts hermetic dolt sql-servers (or completes the lane's job without -short); excluded from --config=prcore/ci, run by --config=doltserver",
"embedded": "embedded-Dolt tier variant; excluded from --config=prcore/ci, run by --config=embedded",
"manual": "never part of //...: a repro/bench harness, or a build input only another target needs; excluded from --config=prcore/ci",
// The same rules as dolt-server (hermetic servers, remote, fail-closed:
// checkDoltServerRules), for the tiers bazel.yml runs only with remote
// execution and, for the server storage tier, under the integration
// lane's build flags.
"dolt-server-proxied": "proxied-server cmd/bd tier: starts hermetic dolt sql-servers; excluded from --config=prcore/ci, run by --config=doltserver-proxied",
"dolt-server-integration": "server-Dolt storage tier: starts hermetic dolt sql-servers and needs the integration build tag; excluded from --config=prcore/ci, run by --config=doltserver-integration",
"embedded": "embedded-Dolt tier variant; excluded from --config=prcore/ci, run by --config=embedded",
"manual": "never part of //...: a repro/bench harness, or a build input only another target needs; excluded from --config=prcore/ci",
// For a go_test whose every test file is `//go:build integration`: in any
// other configuration rules_go drops those files and the target runs
// zero tests, which check_testcases.py rejects and equivalence.py can
@ -694,7 +700,7 @@ var allowedBazelTestTags = map[string]string{
// and vice versa (TestBazelPRCoreExcludedTagsMatchTaxonomy), so a new lane
// tag lands here, and through bazelIntegrationExcludedTags in the
// integration lane's filter too.
var bazelPRCoreExcludedTags = []string{"requires-docker", "dolt-server", "embedded", "manual", "integration-only"}
var bazelPRCoreExcludedTags = []string{"requires-docker", "dolt-server", "dolt-server-proxied", "dolt-server-integration", "embedded", "manual", "integration-only"}
// bazelIntegrationRunsTags are the PR-core-excluded tags --config=integration
// runs: the integration lane is main.yml's integration jobs, whose

View file

@ -1471,6 +1471,8 @@ const (
bazelEmbedJobName = "bazel-embedded"
bazelRBEJobName = "rbe"
bazelIntegJobName = "bazel-integration"
bazelProxiedJobName = "bazel-proxied"
bazelServerJobName = "bazel-server-storage"
setupBazelActionDir = ".github/actions/setup-bazel"
uploadArtifactSHA = "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
downloadArtifactSHA = "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c"
@ -1486,11 +1488,11 @@ const (
// bazel.yml's jobs: the rbe job that decides the execution mode, the
// --config=ci lane, and one job per CI job a Bazel config mirrors.
var bazelJobNames = []string{bazelDoltJobName, bazelEmbedJobName, bazelIntegJobName, bazelPureJobName, bazelJobName, bazelRBEJobName}
var bazelJobNames = []string{bazelDoltJobName, bazelEmbedJobName, bazelIntegJobName, bazelProxiedJobName, bazelPureJobName, bazelServerJobName, bazelJobName, bazelRBEJobName}
// The lanes that only run remotely (skipped unless the rbe job chose remote);
// every other lane also runs locally.
var bazelRemoteOnlyJobs = map[string]bool{bazelEmbedJobName: true, bazelIntegJobName: true}
var bazelRemoteOnlyJobs = map[string]bool{bazelEmbedJobName: true, bazelIntegJobName: true, bazelProxiedJobName: true, bazelServerJobName: true}
// The rbe job's decision step reads exactly one secret, and only to test it
// for emptiness: its env value is a boolean, not the secret.
@ -1868,8 +1870,16 @@ var (
doltServerExtraEnv = map[string][]string{
"fix_dolt_test": {`"BEADS_FIX_REQUIRE_DOLT": "1"`},
"protocol_dolt_test": {`"BEADS_PROTOCOL_REQUIRE_DOLT": "1"`},
// Without it every proxied test self-skips (the jobs' step env).
"bd_proxied_test": {`"BEADS_TEST_PROXIED_SERVER": "1"`},
// federation_test.go fails instead of skipping without a server.
"dolt_server_full_test": {`"BEADS_TEST_ENV_RUN_DOLT": "1"`},
}
bazelRuleNameRe = regexp.MustCompile(`(?m)^\s*name\s*=\s*"([^"]+)"`)
// doltServerLaneTags are the tags of the lanes whose targets start
// hermetic dolt sql-servers; checkDoltServerRules holds each of them to
// the same rules.
doltServerLaneTags = []string{"dolt-server", "dolt-server-proxied", "dolt-server-integration"}
bazelRuleNameRe = regexp.MustCompile(`(?m)^\s*name\s*=\s*"([^"]+)"`)
)
// bazelTopRules splits a BUILD file into its top-level calls: each starts at
@ -1893,14 +1903,26 @@ func bazelTopRules(build string) []string {
return rules
}
// checkDoltServerRules checks each rule tagged dolt-server in one package's
// BUILD file on its own (a file-wide search would accept another target's
// env), and that the package has at least one.
// doltServerRuleTag returns the dolt-server lane tag a rule carries, or "".
func doltServerRuleTag(rule string) string {
for _, tag := range doltServerLaneTags {
if strings.Contains(rule, `"`+tag+`"`) {
return tag
}
}
return ""
}
// checkDoltServerRules checks each rule tagged for a dolt-server lane
// (doltServerLaneTags) in one package's BUILD file on its own (a file-wide
// search would accept another target's env), and that the package has at
// least one.
func checkDoltServerRules(pkg, build string) []error {
var errs []error
found := 0
for _, rule := range bazelTopRules(stripStarlarkComments(build)) {
if !strings.Contains(rule, `"dolt-server"`) {
tag := doltServerRuleTag(rule)
if tag == "" {
continue
}
found++
@ -1908,14 +1930,14 @@ func checkDoltServerRules(pkg, build string) []error {
if m := bazelRuleNameRe.FindStringSubmatch(rule); m != nil {
name = m[1]
}
where := "//" + pkg + ":" + name
where := "//" + pkg + ":" + name + " (" + tag + ")"
for _, env := range append(append([]string{}, doltServerRuleEnv...), doltServerExtraEnv[name]...) {
if !strings.Contains(rule, env) {
errs = append(errs, errors.New(where+" (dolt-server) lacks rule env "+env))
errs = append(errs, errors.New(where+" lacks rule env "+env))
}
}
if strings.Contains(rule, `"no-remote-exec"`) {
errs = append(errs, errors.New(where+" (dolt-server) is tagged no-remote-exec; the lane runs on remote workers"))
errs = append(errs, errors.New(where+" is tagged no-remote-exec; the lane runs on remote workers"))
}
}
if found == 0 {
@ -1960,6 +1982,13 @@ sh_test(
"no target": docker,
"fix switch": strings.Replace(good, `"uow_dolt_test"`, `"fix_dolt_test"`, 1),
"protocol switch": strings.Replace(good, `"uow_dolt_test"`, `"protocol_dolt_test"`, 1),
"proxied switch": strings.Replace(strings.Replace(good, `"uow_dolt_test"`, `"bd_proxied_test"`, 1),
`tags = ["dolt-server"]`, `tags = ["dolt-server-proxied"]`, 1),
"run-dolt switch": strings.Replace(strings.Replace(good, `"uow_dolt_test"`, `"dolt_server_full_test"`, 1),
`tags = ["dolt-server"]`, `tags = ["dolt-server-integration"]`, 1),
"proxied container": strings.Replace(strings.Replace(good, `"local"`, `"container"`, 1),
`tags = ["dolt-server"]`, `tags = ["dolt-server-proxied"]`, 1),
"integration no-remote-exec": strings.Replace(good, `tags = ["dolt-server"]`, `tags = ["dolt-server-integration", "no-remote-exec"]`, 1),
} {
if bad == good {
t.Fatalf("%s: mutation did not apply", name)
@ -2054,6 +2083,260 @@ func TestBazelIntegrationJob(t *testing.T) {
}
}
// bazelrcLines returns .bazelrc's trimmed lines as a set.
func bazelrcLines(t *testing.T) map[string]bool {
t.Helper()
lines := map[string]bool{}
for _, line := range strings.Split(readPolicyFile(t, bazelPolicyRoot(t), ".bazelrc"), "\n") {
lines[strings.TrimSpace(line)] = true
}
return lines
}
// assertBazelTierStep checks the test step of a remote-only tier job in
// bazel.yml: it runs exactly `bazel test //... --config=<config>` with a BEP,
// keeps bazel's exit status, has a timeout below the job's, and is followed
// by check_testcases.py on that BEP.
func assertBazelTierStep(t *testing.T, job ciWorkflowJob, jobName, config string) {
t.Helper()
const id, bep, wantIf = "test", "bazel-bep.json", ""
step := job.step(t, "bazel test //... --config="+config)
if step.ID != id || step.If != wantIf || step.TimeoutMinutes == 0 || step.TimeoutMinutes >= job.TimeoutMinutes {
t.Errorf("%s step --config=%s: id=%q if=%q timeout-minutes=%d; want id %q, if %q, a timeout below the job's %d",
jobName, config, step.ID, step.If, step.TimeoutMinutes, id, wantIf, job.TimeoutMinutes)
}
cmd := regexp.MustCompile(`\s*\\\n\s*`).ReplaceAllString(step.Run, " ")
wantCmd := `bazel test //... --config=` + config + ` --build_event_json_file="$RUNNER_TEMP/` + bep + `"`
if !strings.Contains(cmd, wantCmd) || !strings.Contains(step.Run, "set -o pipefail") || strings.Count(step.Run, "bazel test //") != 1 ||
strings.Contains(step.Run, "--config=remote-exec") || strings.Contains(step.Run, "--test_tag_filters") {
t.Errorf("%s step --config=%s does not run exactly %q:\n%s", jobName, config, wantCmd, step.Run)
}
assertTestStepKeepsExitStatus(t, step)
found := false
for _, s := range job.Steps {
if strings.TrimSpace(s.Run) == `python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/`+bep+`"` {
found = true
if s.If != "${{ always() && steps."+id+".outcome != 'skipped' }}" || (s.ContinueOnError != nil && s.ContinueOnError != false) {
t.Errorf("%s: check_testcases.py for %s: if=%q continue-on-error=%v", jobName, bep, s.If, s.ContinueOnError)
}
}
}
if !found {
t.Errorf("%s: no check_testcases.py step on %s (a shard or selector that runs no tests exits 0)", jobName, bep)
}
}
// The proxied-server tier (pr-risk.yml "Test (Proxied Dolt Cmd N/15)"; main.yml's
// twin runs the same shard script on a non-race binary) and the server-Dolt storage tier (pr-risk.yml "Test (Server Dolt
// Conformance)", "Test (Server Dolt Full Suite N/16)") as Bazel variants:
// each manifest-sharded variant runs its jobs' shard script with their shard
// total, the conformance variant the job's exact flags, with the jobs' race
// setting, subprocess binaries and switches; the lanes' configs keep the
// jobs' selection, and the storage lane shares --config=integration's build.
// bazel.yml runs both tiers remotely only.
func TestBazelDoltServerTiersMirrorPRRisk(t *testing.T) {
rc := bazelrcLines(t)
for _, want := range []string{
"test:doltserver-proxied --@rules_go//go/config:race",
"test:doltserver-proxied --test_tag_filters=dolt-server-proxied",
"test:doltserver-proxied --test_timeout=-1,-1,-1,1200",
"test:doltserver-integration --test_tag_filters=dolt-server-integration",
"test:doltserver-integration --test_timeout=-1,-1,-1,1200",
} {
if !rc[want] {
t.Errorf(".bazelrc lacks %q", want)
}
}
var tierTags, integTags map[string]bool
for line := range rc {
for _, config := range []string{"doltserver-proxied", "doltserver-integration"} {
if strings.HasPrefix(line, "test:"+config+" ") && (strings.Contains(line, "-test.short") || strings.Contains(line, "BEADS_TEST_SKIP") ||
strings.Contains(line, "-test.run") || strings.Contains(line, "-test.skip") || strings.Contains(line, "--test_filter")) {
t.Errorf(".bazelrc %q: the jobs run their scripts' selection, without -short or BEADS_TEST_SKIP", line)
}
}
if v, ok := strings.CutPrefix(line, "build:doltserver-integration --@rules_go//go/config:tags="); ok {
tierTags = tagSet(v)
}
if v, ok := strings.CutPrefix(line, "build:integration --@rules_go//go/config:tags="); ok {
integTags = tagSet(v)
}
}
// The same build flags as --config=integration, or the tier compiles the
// tagged graph once more.
if tierTags == nil || !sameTagSet(tierTags, integTags) {
t.Errorf(".bazelrc build:doltserver-integration tags = %v, want build:integration's %v", tierTags, integTags)
}
if rc["test:integration --@rules_go//go/config:race"] != rc["test:doltserver-integration --@rules_go//go/config:race"] {
t.Error(".bazelrc: test:doltserver-integration and test:integration differ in race; they must share their build")
}
risk := readCIWorkflow(t, "pr-risk.yml")
build := risk.job(t, "build-embedded")
for step, want := range map[string]string{
// The proxied jobs' test binary is this race build (--config=
// doltserver-proxied's bd_test), their subprocess bd the non-race one
// (bd_for_tests).
"Build embedded cmd test binary": "go test -tags gms_pure_go -race -c -o /tmp/bd-cmd-test ./cmd/bd/",
"Build proxied bd subprocess binary": "go build -tags gms_pure_go -o /tmp/bd-proxied ./cmd/bd/",
} {
if got := strings.TrimSpace(build.step(t, step).Run); got != want {
t.Errorf("pr-risk.yml %q = %q, want %q", step, got, want)
}
}
// main.yml's twin proxied jobs run build-artifacts' bd-cmd-test, which
// is not race: bd_proxied_test (race, like PR Risk's) is the stricter of
// the two. Pinned so a change there is a decision, not drift.
mainYML := readCIWorkflow(t, "main.yml")
if run := mainYML.job(t, "build-artifacts").step(t, "Build reusable Linux artifacts").Run; !strings.Contains(run, `go test -tags "$BEADS_BUILD_TAGS" -c -o artifacts/bd-cmd-test ./cmd/bd`+"\n") {
t.Errorf("main.yml build-artifacts no longer builds the non-race bd-cmd-test this tier is documented against (.bazelrc, cmd/bd:bd_proxied_test):\n%s", run)
}
if got := mainYML.job(t, "test-proxied-cmd").step(t, "Test proxied-server cmd shard").Env["BEADS_TEST_CMD_BINARY"]; got != "${{ github.workspace }}/ci-build-artifacts/bd-cmd-test" {
t.Errorf("main.yml test-proxied-cmd BEADS_TEST_CMD_BINARY = %q, want build-artifacts' bd-cmd-test", got)
}
// The server jobs' binary: integration-tagged (the lane's build tags) and
// not race (dolt_race_off).
m := regexp.MustCompile(`^go test -tags=(\S+) -c -o /tmp/dolt-conformance-test \./internal/storage/dolt/$`).
FindStringSubmatch(strings.TrimSpace(build.step(t, "Build server Dolt conformance test binary").Run))
if m == nil || !sameTagSet(tagSet(m[1]), tierTags) {
t.Errorf("pr-risk.yml's server Dolt test binary is no longer the non-race `go test -tags=<build:doltserver-integration tags> -c`: %v", m)
}
type shardTier struct {
workflow, job, step, script, binVar, pkg, target, env string
}
tiers := []shardTier{
{"pr-risk.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
{"main.yml", "test-proxied-cmd", "Test proxied-server cmd shard", ".github/scripts/proxied-test-shard.sh", "BEADS_TEST_CMD_BINARY", "cmd/bd", "bd_proxied_test", "BEADS_TEST_PROXIED_SERVER"},
{"pr-risk.yml", "test-server-storage-full", "Test", ".github/scripts/server-storage-test-shard.sh", "BEADS_TEST_SERVER_TEST_BINARY", "internal/storage/dolt", "dolt_server_full_test", "BEADS_TEST_ENV_RUN_DOLT"},
}
for _, c := range tiers {
j := readCIWorkflow(t, c.workflow).job(t, c.job)
shards := len(j.Strategy.Matrix.Shard)
step := j.step(t, c.step)
if want := "bash " + c.script + " ${{ matrix.shard }} " + strconv.Itoa(shards); strings.TrimSpace(step.Run) != want {
t.Errorf("%s %s runs %q, want %q", c.workflow, c.job, step.Run, want)
}
if step.Env[c.env] != "1" {
t.Errorf("%s %s no longer sets %s=1; update %s:%s", c.workflow, c.job, c.env, c.pkg, c.target)
}
if os.Getenv("TEST_SRCDIR") != "" {
continue // scripts_test's runfiles hold no other package's BUILD
}
root := sourceRepoRoot(t)
rule := bazelRuleBlock(readPolicyFile(t, root, c.pkg+"/BUILD.bazel"), c.target)
for _, want := range []string{
`srcs = ["//tools/bazel:go_test_manifest_shard.sh"],`,
`"$(rootpath //:` + c.script + `)",`,
`"` + c.binVar + `",`,
"shard_count = " + strconv.Itoa(shards) + ",",
`timeout = "eternal",`,
} {
if !strings.Contains(rule, want) {
t.Errorf("%s:%s does not contain %q (%s %s):\n%s", c.pkg, c.target, want, c.workflow, c.job, rule)
}
}
mm := shardManifestDefault.FindStringSubmatch(readPolicyFile(t, root, c.script))
if mm == nil {
t.Fatalf("%s has no ${BEADS_TEST_SHARD_MANIFEST:-...} default manifest", c.script)
}
data := bazelAttrBlock(rule, "data")
for _, file := range []string{c.script, mm[1]} {
if !strings.Contains(data, `"//:`+file+`",`) {
t.Errorf("%s:%s data lacks //:%s (without the manifest every shard falls back to hashing):\n%s", c.pkg, c.target, file, data)
}
}
}
conf := risk.job(t, "test-server-storage").step(t, "Test").Run
quoted := regexp.MustCompile(`(-test\.[a-z]+) '([^']*)'|(-test\.[a-z]+=\S+|-test\.v)`)
fields := strings.Fields(conf)
if len(fields) == 0 || fields[0] != "/tmp/dolt-conformance-test" {
t.Fatalf("pr-risk.yml test-server-storage no longer runs /tmp/dolt-conformance-test: %q", conf)
}
var wantArgs []string
for _, m := range quoted.FindAllStringSubmatch(conf, -1) {
if m[1] != "" {
wantArgs = append(wantArgs, `"`+m[1]+"="+strings.ReplaceAll(m[2], "$", "$$")+`",`)
} else {
wantArgs = append(wantArgs, `"`+m[3]+`",`)
}
}
if len(wantArgs) != 4 {
t.Fatalf("parsed %v from pr-risk.yml test-server-storage %q", wantArgs, conf)
}
if os.Getenv("TEST_SRCDIR") == "" {
root := sourceRepoRoot(t)
doltBuild := readPolicyFile(t, root, "internal/storage/dolt/BUILD.bazel")
rule := bazelRuleBlock(doltBuild, "dolt_server_conformance_test")
for _, w := range append(wantArgs, `"$(rootpath :dolt_race_off)",`, `srcs = ["//tools/bazel:go_test_variant.sh"],`) {
if !strings.Contains(bazelAttrBlock(rule, "args")+rule, w) {
t.Errorf("dolt:dolt_server_conformance_test does not contain %q (pr-risk.yml test-server-storage):\n%s", w, rule)
}
}
if strings.Contains(rule, "BEADS_TEST_ENV_RUN_DOLT") {
t.Error("dolt:dolt_server_conformance_test sets BEADS_TEST_ENV_RUN_DOLT; test-server-storage does not")
}
if r := bazelRuleBlock(doltBuild, "dolt_race_off"); !strings.Contains(r, "go_test_race_off(") || !strings.Contains(r, `test = ":dolt_test",`) {
t.Errorf("dolt:dolt_race_off must be go_test_race_off of :dolt_test (the jobs' binary is not race):\n%s", r)
}
full := bazelRuleBlock(doltBuild, "dolt_server_full_test")
for _, w := range []string{`"$(rootpath :dolt_race_off)",`, `"BEADS_TEST_SUBPROCESS_BINARY": "$(rlocationpath :dolt_race_off)"`, `"//:go.mod",`} {
if !strings.Contains(full, w) {
t.Errorf("dolt:dolt_server_full_test lacks %q (SubprocessRunner reuses the test binary; ModuleRoot needs go.mod):\n%s", w, full)
}
}
proxied := bazelRuleBlock(readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "bd_proxied_test")
for _, w := range []string{`"$(rootpath :bd_test)",`, `"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)"`} {
if !strings.Contains(proxied, w) {
t.Errorf("cmd/bd:bd_proxied_test lacks %q (race bd_test, non-race subprocess bd, like the jobs):\n%s", w, proxied)
}
}
for pkg, build := range map[string]string{"cmd/bd": readPolicyFile(t, root, "cmd/bd/BUILD.bazel"), "internal/storage/dolt": doltBuild} {
for _, err := range checkDoltServerRules(pkg, build) {
t.Error(err)
}
}
}
// Each tier is a remote-only job of its own (the job if is pinned by
// TestBazelWorkflowIsAdvisory via bazelRemoteOnlyJobs): not a step of
// bazel-doltserver (which also runs locally, and whose job a gate may
// require) or of bazel-integration (which a caller may switch off,
// although the server tier is a PR-time tier).
workflow := readCIWorkflow(t, bazelWorkflowName)
for _, c := range []struct{ job, config, logs string }{
{bazelProxiedJobName, "doltserver-proxied", "bazel-proxied-testlogs"},
{bazelServerJobName, "doltserver-integration", "bazel-server-storage-testlogs"},
} {
job := workflow.job(t, c.job)
if job.TimeoutMinutes == 0 || job.TimeoutMinutes > 30 {
t.Errorf("%s timeout-minutes = %d; it runs remotely only (longest shard ~2-8 min), keep it at most 30", c.job, job.TimeoutMinutes)
}
assertBazelTierStep(t, job, c.job, c.config)
if n := len(job.Steps); n != 5 {
t.Errorf("%s has %d steps, want checkout, setup-bazel, the tier, check_testcases.py, log upload", c.job, n)
}
logs := job.step(t, "Upload test logs")
if logs.If != "${{ failure() && steps.test.outcome != 'skipped' }}" || logs.With["name"] != c.logs || !strings.HasPrefix(logs.Uses, "actions/upload-artifact@") {
t.Errorf("%s test-log upload: if=%q name=%q uses=%q", c.job, logs.If, logs.With["name"], logs.Uses)
}
}
for name, j := range workflow.Jobs {
for _, step := range j.Steps {
proxied := strings.Contains(step.Run, "--config=doltserver-proxied")
server := strings.Contains(step.Run, "--config=doltserver-integration")
if (proxied && name != bazelProxiedJobName) || (server && name != bazelServerJobName) {
t.Errorf("%s step %q runs a dolt-server tier outside its own job", name, step.Name)
}
if (name == bazelIntegJobName || name == bazelDoltJobName) && (proxied || server || strings.Contains(step.Run, "doltserver-")) {
t.Errorf("%s step %q runs the proxied or server storage tier; each has its own job", name, step.Name)
}
}
}
}
// bazel-embedded replaces pr-risk.yml's embedded-Dolt tier: --config=embedded
// runs the embedded variants race like the jobs' binaries, with the jobs'
// parallelism; each shard variant runs its job's shard script with the job's

View file

@ -1,7 +1,8 @@
#!/usr/bin/env bash
# Runs one Bazel shard of an existing go_test binary through a PR Risk shard
# script (.github/scripts/embedded-test-shard.sh and
# embedded-storage-test-shard.sh), so the Bazel target runs exactly the tests
# script (.github/scripts/embedded-test-shard.sh,
# embedded-storage-test-shard.sh, proxied-test-shard.sh,
# server-storage-test-shard.sh), so the Bazel target runs exactly the tests
# the CI job with the same shard number does: the same discovery, the same
# committed manifest and hash fallback, the same binary flags. Like
# go_test_variant.sh it reuses the go_test's binary instead of compiling the
@ -14,7 +15,8 @@
# and where the script's source globs (cmd/bd/*_embedded_test.go, ...)
# resolve.
# <binary env var>: the variable the script reads the binary path from
# (BEADS_TEST_CMD_BINARY or BEADS_TEST_EMBEDDED_TEST_BINARY); set to an
# (BEADS_TEST_CMD_BINARY, BEADS_TEST_EMBEDDED_TEST_BINARY or
# BEADS_TEST_SERVER_TEST_BINARY); set to an
# absolute path, since tests re-exec it from their package directory.
# The remaining arguments (and any --test_arg) follow the script's own
# binary flags.

View file

@ -55,6 +55,14 @@ trap 'reap_root_processes 2>/dev/null || true; chmod -R u+w "$root" 2>/dev/null
mkdir -p "$root/home" "$root/xdg-config" "$root/dolt-root" "$root/tmp"
: >"$root/gitconfig"
# The global git identity of the CI jobs that run `git config --global
# user.name/user.email` before their tests (PR Risk's server-Dolt and
# proxied-server jobs): a target mirroring such a job sets
# BEADS_TEST_GIT_IDENTITY=1 in its env. Everything else keeps pr-core's empty
# global gitconfig.
if [[ "${BEADS_TEST_GIT_IDENTITY:-}" == 1 ]]; then
printf '[user]\n\tname = CI Bot\n\temail = ci@beads.test\n' >"$root/gitconfig"
fi
# Dolt identity, as beads_test_env_enter sets with `dolt config --global`:
# tests that shell out to dolt commit need an author. Written directly so the
# wrapper does not depend on a dolt binary. The two *.disabled keys stop every