ci(bazel): run the managed-local proxied lifecycle lane under Bazel; retire proxied-local-smoke.yml (#7326)
proxied-local-smoke.yml built bd and the cmd/bd test binary on a GitHub runner, installed the pinned dolt, and ran `-test.run ^TestManagedLocalProxied` under `sudo unshare --net` with only loopback up, to prove the managed-local lifecycle (bd -> loopback proxy -> dolt sql-server) needs no outbound network. //cmd/bd:bd_managed_local_test runs the same tests (BEADS_TEST_PROXIED_LOCAL=1, BEADS_TEST_SKIP=dolt, the non-race bd_for_tests as the bd under test, the pinned dolt from the test wrapper's PATH) through tools/bazel/go_test_offline.sh, which enters an unprivileged user + network namespace with only loopback up and fails if outbound network is reachable: the job's netns, without sudo, on the rbe-west workers. Tagged dolt-server-proxied, so bazel.yml's bazel-proxied lane (remote-only, required by pr.yml's gate) runs it; test result caching replaces the workflow's path filter. The job's "tests are compiled in" assertion becomes scripts/managed_local_lane_test.go, which evaluates cmd/bd's build constraints for a linux cgo build and requires each of the 17 pinned tests. The target joins the pinned retired-lane targets in TestBazelRetiredLanesCannotBeNarrowed. proxied-local-smoke.yml and the policy tests that pinned its job and its dolt version are removed (the Bazel dolt pin is already enforced against install-dolt.sh). Refs: ga-96smfk.19 Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
cb0dcbecf2
commit
c292964571
14 changed files with 189 additions and 231 deletions
4
.bazelrc
4
.bazelrc
|
|
@ -145,7 +145,9 @@ test:doltserver --local_test_jobs=4
|
|||
test:doltserver --remote_download_regex=.*/test\.(log|xml)$
|
||||
|
||||
# Proxied-server tier: PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs, as
|
||||
# //cmd/bd:bd_proxied_test (dolt-server-proxied tag). Kept out of
|
||||
# //cmd/bd:bd_proxied_test and //cmd/bd:bd_managed_local_test (the
|
||||
# managed-local lifecycle in a loopback-only network namespace) (the
|
||||
# dolt-server-proxied tag). Kept out of
|
||||
# --config=doltserver because bazel.yml runs that lane locally too (forks,
|
||||
# rbe=off), where 30 concurrent shards, each with a dolt sql-server busy on
|
||||
# ~4 cores, do not fit regardless of any single shard's wall time; bazel.yml
|
||||
|
|
|
|||
9
.github/workflows/bazel.yml
vendored
9
.github/workflows/bazel.yml
vendored
|
|
@ -1469,9 +1469,12 @@ jobs:
|
|||
# bd_for_tests), shared with bazel-doltserver through the remote cache.
|
||||
# PR Risk builds its bd-cmd-test with -race, as here; main.yml's
|
||||
# test-proxied-cmd twin uses build-artifacts' non-race bd-cmd-test, so
|
||||
# this lane is the stricter of the two. A job of its own, not gated by
|
||||
# any integration switch: it runs in pr.yml's call and its gate requires
|
||||
# it (BAZEL_PROXIED).
|
||||
# this lane is the stricter of the two. It also runs
|
||||
# //cmd/bd:bd_managed_local_test, the managed-local proxied lifecycle
|
||||
# (formerly proxied-local-smoke.yml) in a loopback-only network
|
||||
# namespace, which needs the workers' unprivileged user namespaces. A job
|
||||
# of its own, not gated by any integration switch: it runs in pr.yml's
|
||||
# call and its gate requires it (BAZEL_PROXIED).
|
||||
bazel-proxied:
|
||||
name: proxied-server tier
|
||||
# Remote execution only, like bazel-embedded: 30 concurrent shards, each
|
||||
|
|
|
|||
189
.github/workflows/proxied-local-smoke.yml
vendored
189
.github/workflows/proxied-local-smoke.yml
vendored
|
|
@ -1,189 +0,0 @@
|
|||
# Managed-local proxied lifecycle lane.
|
||||
#
|
||||
# The proxied shards in main.yml exercise command behavior against an
|
||||
# EXTERNALLY managed Dolt testcontainer. This lane covers the production
|
||||
# local-first path those shards cannot: bd launches its own loopback proxy,
|
||||
# the proxy launches a locally installed `dolt sql-server`, and the whole
|
||||
# lifecycle (create/read, loopback-only listeners, idle shutdown,
|
||||
# transparent restart with persistence), identity failures, orphan recovery,
|
||||
# and start/stop convergence run INSIDE a network namespace with only
|
||||
# loopback up — proving no outbound network is needed once the binaries are
|
||||
# installed. Linux first; the test lane defines Windows/macOS parity but does
|
||||
# not implement those OS-specific observation helpers yet.
|
||||
|
||||
name: Proxied Local Lifecycle
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
merge_group:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'cmd/bd/proxied_local_*'
|
||||
- 'cmd/bd/proxied_server*.go'
|
||||
- 'cmd/bd/dolt*.go'
|
||||
- 'cmd/bd/*_proxied_server.go'
|
||||
- 'cmd/bd/backup*.go'
|
||||
- 'cmd/bd/purge*.go'
|
||||
- 'cmd/bd/*managed_local*'
|
||||
- 'cmd/bd/proxied_outage_reconnect_integration_test.go'
|
||||
- 'cmd/bd/db_proxy_child.go'
|
||||
- 'cmd/bd/init*.go'
|
||||
- 'cmd/bd/proxied_integration_helpers_test.go'
|
||||
- 'cmd/bd/store_factory*.go'
|
||||
- 'cmd/bd/test_dolt_server_cgo_test.go'
|
||||
- 'cmd/bd/tracker_proxy_uow_conformance_integration_test.go'
|
||||
- 'cmd/bd/uow_factory.go'
|
||||
- 'internal/configfile/**'
|
||||
- 'internal/doltserver/**'
|
||||
- 'internal/storage/dbproxy/**'
|
||||
- 'internal/storage/uow/**'
|
||||
- 'internal/tracker/conformance/**'
|
||||
- 'internal/tracker/uow_store.go'
|
||||
- 'internal/tracker/engine.go'
|
||||
- 'internal/tracker/store.go'
|
||||
- 'go.mod'
|
||||
- 'go.sum'
|
||||
- '.github/workflows/proxied-local-smoke.yml'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'cmd/bd/proxied_local_*'
|
||||
- 'cmd/bd/proxied_server*.go'
|
||||
- 'cmd/bd/dolt*.go'
|
||||
- 'cmd/bd/*_proxied_server.go'
|
||||
- 'cmd/bd/backup*.go'
|
||||
- 'cmd/bd/purge*.go'
|
||||
- 'cmd/bd/*managed_local*'
|
||||
- 'cmd/bd/proxied_outage_reconnect_integration_test.go'
|
||||
- 'cmd/bd/db_proxy_child.go'
|
||||
- 'cmd/bd/init*.go'
|
||||
- 'cmd/bd/proxied_integration_helpers_test.go'
|
||||
- 'cmd/bd/store_factory*.go'
|
||||
- 'cmd/bd/test_dolt_server_cgo_test.go'
|
||||
- 'cmd/bd/tracker_proxy_uow_conformance_integration_test.go'
|
||||
- 'cmd/bd/uow_factory.go'
|
||||
- 'internal/configfile/**'
|
||||
- 'internal/doltserver/**'
|
||||
- 'internal/storage/dbproxy/**'
|
||||
- 'internal/storage/uow/**'
|
||||
- 'internal/tracker/conformance/**'
|
||||
- 'internal/tracker/uow_store.go'
|
||||
- 'internal/tracker/engine.go'
|
||||
- 'internal/tracker/store.go'
|
||||
- 'go.mod'
|
||||
- 'go.sum'
|
||||
- '.github/workflows/proxied-local-smoke.yml'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
managed-local-smoke:
|
||||
name: Managed-local proxied lifecycle (Linux, offline)
|
||||
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
||||
timeout-minutes: 25
|
||||
env:
|
||||
CGO_ENABLED: "1"
|
||||
# Pinned so the lane tests a known dolt CLI, not whatever the installer
|
||||
# resolves on a given day. Bump deliberately, and only in lockstep with
|
||||
# the measured pin in scripts/ci/install-dolt.sh (see "Which Dolt
|
||||
# version to install" in docs/architecture/dolt.md).
|
||||
DOLT_VERSION: 2.2.0
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Go
|
||||
id: setup-go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: ${{ runner.environment == 'github-hosted' }}
|
||||
|
||||
# F7c review fix S5: this job's own key copy must match main.yml's
|
||||
# seeder key byte-for-byte (TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers),
|
||||
# which now keys by UTC calendar day instead of github.sha so main
|
||||
# pushes don't churn a multi-GB cache entry on every commit; this step
|
||||
# computes that same day value locally so the key expression below
|
||||
# resolves (steps.* references only resolve within the same job).
|
||||
- name: Compute cache date
|
||||
id: cache-date
|
||||
if: runner.environment == 'self-hosted'
|
||||
run: echo "today=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Restore Blacksmith setup-go cache
|
||||
if: runner.environment == 'self-hosted'
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
~/go/pkg/mod
|
||||
~/.cache/go-build
|
||||
key: blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-${{ steps.cache-date.outputs.today }}
|
||||
restore-keys: |
|
||||
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-
|
||||
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Install pinned Dolt CLI
|
||||
run: |
|
||||
curl -fsSL "https://github.com/dolthub/dolt/releases/download/v${DOLT_VERSION}/dolt-linux-amd64.tar.gz" -o /tmp/dolt.tar.gz
|
||||
tar -xzf /tmp/dolt.tar.gz -C /tmp
|
||||
sudo install /tmp/dolt-linux-amd64/bin/dolt /usr/local/bin/dolt
|
||||
dolt version
|
||||
|
||||
- name: Build bd and compile the test binary (online)
|
||||
run: |
|
||||
go build -tags gms_pure_go -o /tmp/bd-under-test ./cmd/bd
|
||||
go test -tags gms_pure_go -c -o /tmp/proxied-local-smoke.test ./cmd/bd
|
||||
|
||||
- name: Assert the managed-local lifecycle tests are compiled in
|
||||
run: |
|
||||
listed="$(
|
||||
BEADS_TEST_PROXIED_LOCAL=1 \
|
||||
BEADS_TEST_SKIP=dolt \
|
||||
/tmp/proxied-local-smoke.test \
|
||||
-test.list '^TestManagedLocalProxied'
|
||||
)"
|
||||
for test_name in \
|
||||
TestManagedLocalProxiedLifecycleSmoke \
|
||||
TestManagedLocalProxiedForeignListenerNotAdopted \
|
||||
TestManagedLocalProxiedReusedPidNeverKilled \
|
||||
TestManagedLocalProxiedLegacyRecordFailClosed \
|
||||
TestManagedLocalProxiedOrphanBackendReaped \
|
||||
TestManagedLocalProxiedNProcessColdStartConvergence \
|
||||
TestManagedLocalProxiedStopStartInterleave \
|
||||
TestManagedLocalProxiedDoltStatusReportsLiveTopology \
|
||||
TestManagedLocalProxiedDoltStartRefusesOverLiveProxy \
|
||||
TestManagedLocalProxiedDoltStartRefusesWithProxyDown \
|
||||
TestManagedLocalProxiedDoltLifecycleLeavesOtherTopologiesAlone \
|
||||
TestManagedLocalProxiedTrackerUOWConformance \
|
||||
TestManagedLocalProxiedBackupRoundTrip \
|
||||
TestManagedLocalProxiedAutoBackup \
|
||||
TestManagedLocalProxiedBackupRestoreRefusedWhileAttached \
|
||||
TestManagedLocalProxiedPurgeWispsPlaneRetention \
|
||||
TestManagedLocalProxiedBackendCleanExitRetiresProxy
|
||||
do
|
||||
if ! grep -qx "$test_name" <<<"$listed"; then
|
||||
echo "managed-local lifecycle test $test_name is missing; got: '$listed'" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Run managed-local lifecycle lane offline (loopback-only netns)
|
||||
run: |
|
||||
sudo env \
|
||||
"PATH=$PATH" \
|
||||
BEADS_TEST_PROXIED_LOCAL=1 \
|
||||
BEADS_TEST_SKIP=dolt \
|
||||
BEADS_TEST_BD_BINARY=/tmp/bd-under-test \
|
||||
unshare --net -- bash -euxc '
|
||||
ip link set lo up
|
||||
# Prove the namespace really has no outbound network before
|
||||
# trusting anything the test reports.
|
||||
if timeout 3 bash -c "exec 3<>/dev/tcp/1.1.1.1/443" 2>/dev/null; then
|
||||
echo "outbound network unexpectedly available inside the netns" >&2
|
||||
exit 1
|
||||
fi
|
||||
cd cmd/bd
|
||||
/tmp/proxied-local-smoke.test \
|
||||
-test.run "^TestManagedLocalProxied" \
|
||||
-test.v -test.timeout 15m
|
||||
'
|
||||
|
|
@ -1488,6 +1488,57 @@ sh_test(
|
|||
tags = ["dolt-server-proxied"],
|
||||
)
|
||||
|
||||
# The managed-local proxied lifecycle lane (formerly proxied-local-smoke.yml):
|
||||
# the TestManagedLocalProxied* tests, where bd launches its own loopback
|
||||
# proxy and the proxy launches the pinned dolt (first on PATH via the test
|
||||
# wrapper) as `dolt sql-server`, run inside a user + network namespace with
|
||||
# only loopback up (tools/bazel/go_test_offline.sh, the job's `sudo unshare
|
||||
# --net`), so they prove the lifecycle needs no outbound network. Against
|
||||
# this configuration's (race) bd_test, with the non-race bd_for_tests as the
|
||||
# bd under test; BEADS_TEST_SKIP=dolt as in the job (no shared test server:
|
||||
# every test supervises its own; BEADS_TEST_DOLT_SERVER/REQUIRE are the
|
||||
# dolt-server lanes' convention and start nothing here). Unsharded: Go's
|
||||
# sharding counts every test in the binary, not the -test.run matches, so
|
||||
# most shards would run none. scripts/managed_local_lane_test.go fails if a
|
||||
# test the job pinned is no longer compiled into a linux cgo build. It repeats
|
||||
# bd_test's `env`.
|
||||
#
|
||||
# Tags:
|
||||
# dolt-server-proxied: the proxied-server tier; starts dolt sql-servers;
|
||||
# excluded from --config=prcore/ci, selected by --config=doltserver-proxied
|
||||
# (bazel.yml's bazel-proxied lane, remote execution only: the namespace
|
||||
# needs the workers' unprivileged user namespaces).
|
||||
sh_test(
|
||||
name = "bd_managed_local_test",
|
||||
size = "large",
|
||||
# -test.timeout=15m (the job's) fires with a goroutine dump before
|
||||
# Bazel's 1200s action limit (eternal under the lane's --test_timeout).
|
||||
timeout = "eternal",
|
||||
srcs = ["//tools/bazel:go_test_offline.sh"],
|
||||
args = [
|
||||
"$(rootpath :bd_test)",
|
||||
"-test.run=^TestManagedLocalProxied",
|
||||
"-test.timeout=15m",
|
||||
],
|
||||
data = [
|
||||
":bd_for_tests",
|
||||
":bd_test",
|
||||
":go_srcs",
|
||||
":preflight_go_fixture",
|
||||
"@go_sdk//:bin/gofmt",
|
||||
],
|
||||
env = {
|
||||
"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)",
|
||||
"BEADS_TEST_DOLT_SERVER": "local",
|
||||
"BEADS_TEST_GOFMT": "$(rlocationpath @go_sdk//:bin/gofmt)",
|
||||
"BEADS_TEST_PREFLIGHT_GO": "$(rlocationpath :preflight_go_fixture)",
|
||||
"BEADS_TEST_PROXIED_LOCAL": "1",
|
||||
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1",
|
||||
"BEADS_TEST_SKIP": "dolt",
|
||||
},
|
||||
tags = ["dolt-server-proxied"],
|
||||
)
|
||||
|
||||
# The whole cmd/bd suite, as --config=doltserver-cmd builds it (exactly
|
||||
# --config=integration's build: the integration and gms_pure_go build tags,
|
||||
# race, so the two lanes share bd_test's compile), against a hermetic dolt sql-server:
|
||||
|
|
|
|||
|
|
@ -29,9 +29,9 @@ import (
|
|||
// gone. Both directions are load-bearing — a restore that changed nothing would
|
||||
// pass the first check alone.
|
||||
//
|
||||
// Named TestManagedLocalProxied* so the proxied-local smoke lane
|
||||
// (.github/workflows/proxied-local-smoke.yml) runs it; that lane is the only
|
||||
// one that sets BEADS_TEST_PROXIED_LOCAL.
|
||||
// Named TestManagedLocalProxied* so the managed-local lane
|
||||
// (//cmd/bd:bd_managed_local_test) runs it; that lane is the only one that
|
||||
// sets BEADS_TEST_PROXIED_LOCAL.
|
||||
func TestManagedLocalProxiedBackupRoundTrip(t *testing.T) {
|
||||
requireManagedLocalProxiedEnv(t)
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@ package main
|
|||
// MANAGED-LOCAL proxied workspace, plus the direct/embedded control that keeps
|
||||
// the proxied guard from leaking into topologies bd really does manage.
|
||||
//
|
||||
// These live in the managed-local lane (TestManagedLocalProxied* runs under
|
||||
// proxied-local-smoke.yml) because they are the only lane with a real proxy
|
||||
// These live in the managed-local lane (TestManagedLocalProxied* runs in
|
||||
// //cmd/bd:bd_managed_local_test) because they are the only lane with a real proxy
|
||||
// supervising a real dolt child: the defects being covered are about which
|
||||
// process record bd reads and which process bd spawns, and neither is
|
||||
// observable against an external testcontainer.
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ package main
|
|||
// -run '^TestManagedLocalProxied' -v
|
||||
//
|
||||
// CI additionally runs it inside a network namespace with only loopback up
|
||||
// (see .github/workflows/proxied-local-smoke.yml), proving the whole
|
||||
// (//cmd/bd:bd_managed_local_test, tools/bazel/go_test_offline.sh), proving the whole
|
||||
// lifecycle needs no outbound network once bd and dolt are installed.
|
||||
// When BEADS_TEST_PROXIED_LOCAL=1 is set, missing prerequisites or a Dolt
|
||||
// child that fails to launch FAIL the test rather than skipping.
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ go_test(
|
|||
"embedded_shard_count_test.go",
|
||||
"host_tools_test.go",
|
||||
"install_dolt_test.go",
|
||||
"managed_local_lane_test.go",
|
||||
"pr_lanes_bazel_coverage_test.go",
|
||||
"pr_preflight_test.go",
|
||||
"pr_risk_bazel_coverage_test.go",
|
||||
|
|
|
|||
|
|
@ -698,7 +698,6 @@ var blacksmithAdvisoryWorkflows = []string{
|
|||
"migration-test.yml",
|
||||
"cross-version-smoke.yml",
|
||||
"docs-mintlify.yml",
|
||||
"proxied-local-smoke.yml",
|
||||
"main.yml",
|
||||
}
|
||||
|
||||
|
|
@ -756,7 +755,6 @@ var advisoryBlacksmithRunnerJobs = []struct {
|
|||
{"cross-version-smoke.yml", "smoke", "blacksmith-4vcpu-ubuntu-2404", "ubuntu-latest"},
|
||||
{"cross-version-smoke.yml", "versions", "blacksmith-2vcpu-ubuntu-2404", "ubuntu-latest"},
|
||||
{"docs-mintlify.yml", "broken-links", "blacksmith-2vcpu-ubuntu-2404", "ubuntu-latest"},
|
||||
{"proxied-local-smoke.yml", "managed-local-smoke", "blacksmith-4vcpu-ubuntu-2404", "ubuntu-latest"},
|
||||
}
|
||||
|
||||
// TestF7cAdvisorySameRepoBlacksmithExpressionSemantics evaluates each F7c
|
||||
|
|
@ -825,7 +823,6 @@ var blacksmithSetupGoCacheConsumers = map[string][]string{
|
|||
"regression.yml": {"regression"},
|
||||
"migration-test.yml": {"historical-upgrades"},
|
||||
"cross-version-smoke.yml": {"smoke"},
|
||||
"proxied-local-smoke.yml": {"managed-local-smoke"},
|
||||
}
|
||||
|
||||
// blacksmithSetupGoCacheKeyNamespace is the self-defined cache key prefix
|
||||
|
|
|
|||
|
|
@ -2464,32 +2464,6 @@ func TestPinnedDoltCLIMatchesContainerImage(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestProxiedLocalSmokeMatchesPinnedDoltVersion keeps the proxied-local-smoke
|
||||
// lane's standalone Dolt CLI install on the same release as the rest of the
|
||||
// suite. That lane downloads its own dolt binary straight from GitHub
|
||||
// releases instead of going through scripts/ci/install-dolt.sh, so nothing
|
||||
// else catches it drifting off the measured pin (see "Which Dolt version to
|
||||
// install" in docs/architecture/dolt.md for why the pin is not just "latest").
|
||||
func TestProxiedLocalSmokeMatchesPinnedDoltVersion(t *testing.T) {
|
||||
root := sourceRepoRoot(t)
|
||||
|
||||
installer, err := os.ReadFile(filepath.Join(root, "scripts", "ci", "install-dolt.sh"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cliVersion := captureOne(t, `(?m)^readonly version="([0-9]+\.[0-9]+\.[0-9]+)"$`, string(installer), "scripts/ci/install-dolt.sh")
|
||||
|
||||
workflow, err := os.ReadFile(filepath.Join(root, ".github", "workflows", "proxied-local-smoke.yml"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
smokeVersion := captureOne(t, `(?m)^\s*DOLT_VERSION:\s*([0-9]+\.[0-9]+\.[0-9]+)\s*$`, string(workflow), "proxied-local-smoke.yml:DOLT_VERSION")
|
||||
|
||||
if cliVersion != smokeVersion {
|
||||
t.Errorf("dolt pins disagree: CLI %s, proxied-local-smoke.yml DOLT_VERSION %s", cliVersion, smokeVersion)
|
||||
}
|
||||
}
|
||||
|
||||
func captureOne(t *testing.T, pattern, body, source string) string {
|
||||
t.Helper()
|
||||
|
||||
|
|
@ -2716,7 +2690,7 @@ const mainWindowsTestBinariesCacheRunsOn = "${{ matrix.runner == 'blacksmith' &&
|
|||
// F7c: used to define its own sameRepoBlacksmith4vcpu here (same expression
|
||||
// as sameRepoBlacksmith2vcpu with the 4 vCPU label, for advisory jobs that
|
||||
// compile Go: regression.yml, migration-test.yml,
|
||||
// cross-version-smoke.yml, proxied-local-smoke.yml). F7a independently
|
||||
// cross-version-smoke.yml). F7a independently
|
||||
// defined the same const; both are now served by the single shared
|
||||
// sameRepoBlacksmith4vcpu in ci_blacksmith_runner_test.go. migration-test.yml's
|
||||
// historical-upgrades job uses sameRepoBlacksmith4vcpuNoble instead (below).
|
||||
|
|
@ -3063,7 +3037,6 @@ func TestSameRepoBlacksmithRunners(t *testing.T) {
|
|||
"migration-test.yml": {"historical-upgrades": sameRepoBlacksmith4vcpuNoble},
|
||||
"cross-version-smoke.yml": {"smoke": sameRepoBlacksmith4vcpu, "versions": sameRepoBlacksmith2vcpu},
|
||||
"docs-mintlify.yml": {"broken-links": sameRepoBlacksmith2vcpu},
|
||||
"proxied-local-smoke.yml": {"managed-local-smoke": sameRepoBlacksmith4vcpu},
|
||||
// main.yml's seeder job (B2, F7c implementation report) is the one
|
||||
// Blacksmith job that is NOT gated by the same-repo-PR expression: it
|
||||
// is push-to-main only (always trusted), so it wraps the literal
|
||||
|
|
@ -3128,7 +3101,7 @@ func TestSameRepoBlacksmithRunners(t *testing.T) {
|
|||
for _, file := range []string{
|
||||
"pr.yml", "pr-risk.yml", "main.yml",
|
||||
"regression.yml", "migration-test.yml",
|
||||
"cross-version-smoke.yml", "docs-mintlify.yml", "proxied-local-smoke.yml",
|
||||
"cross-version-smoke.yml", "docs-mintlify.yml",
|
||||
} {
|
||||
workflow := readCIWorkflow(t, file)
|
||||
allowed := want[file]
|
||||
|
|
|
|||
81
scripts/managed_local_lane_test.go
Normal file
81
scripts/managed_local_lane_test.go
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
package scripts_test
|
||||
|
||||
import (
|
||||
"go/build"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// managedLocalLaneTests are the managed-local proxied lifecycle tests the
|
||||
// retired proxied-local-smoke.yml asserted were compiled into its test binary
|
||||
// before it ran `-test.run ^TestManagedLocalProxied`. They now run in
|
||||
// //cmd/bd:bd_managed_local_test, whose -test.run would silently match fewer
|
||||
// tests if a build constraint dropped one.
|
||||
var managedLocalLaneTests = []string{
|
||||
"TestManagedLocalProxiedLifecycleSmoke",
|
||||
"TestManagedLocalProxiedForeignListenerNotAdopted",
|
||||
"TestManagedLocalProxiedReusedPidNeverKilled",
|
||||
"TestManagedLocalProxiedLegacyRecordFailClosed",
|
||||
"TestManagedLocalProxiedOrphanBackendReaped",
|
||||
"TestManagedLocalProxiedNProcessColdStartConvergence",
|
||||
"TestManagedLocalProxiedStopStartInterleave",
|
||||
"TestManagedLocalProxiedDoltStatusReportsLiveTopology",
|
||||
"TestManagedLocalProxiedDoltStartRefusesOverLiveProxy",
|
||||
"TestManagedLocalProxiedDoltStartRefusesWithProxyDown",
|
||||
"TestManagedLocalProxiedDoltLifecycleLeavesOtherTopologiesAlone",
|
||||
"TestManagedLocalProxiedTrackerUOWConformance",
|
||||
"TestManagedLocalProxiedBackupRoundTrip",
|
||||
"TestManagedLocalProxiedAutoBackup",
|
||||
"TestManagedLocalProxiedBackupRestoreRefusedWhileAttached",
|
||||
"TestManagedLocalProxiedPurgeWispsPlaneRetention",
|
||||
"TestManagedLocalProxiedBackendCleanExitRetiresProxy",
|
||||
}
|
||||
|
||||
// TestManagedLocalLaneTestsCompileIntoBdTest is the job's "Assert the
|
||||
// managed-local lifecycle tests are compiled in" step: each pinned test is
|
||||
// defined in a cmd/bd _test.go file that a linux/amd64 cgo build with
|
||||
// bd_test's tags (gms_pure_go, no integration) compiles.
|
||||
func TestManagedLocalLaneTestsCompileIntoBdTest(t *testing.T) {
|
||||
dir := filepath.Join(sourceRepoRoot(t), "cmd", "bd")
|
||||
ctx := build.Default
|
||||
ctx.GOOS, ctx.GOARCH, ctx.CgoEnabled = "linux", "amd64", true
|
||||
ctx.BuildTags = []string{"gms_pure_go"}
|
||||
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defined := map[string]string{}
|
||||
funcRe := regexp.MustCompile(`(?m)^func (TestManagedLocalProxied\w*)\(`)
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if e.IsDir() || !strings.HasSuffix(name, "_test.go") {
|
||||
continue
|
||||
}
|
||||
match, err := ctx.MatchFile(dir, name)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
if !match {
|
||||
continue
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(dir, name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, m := range funcRe.FindAllStringSubmatch(string(data), -1) {
|
||||
defined[m[1]] = name
|
||||
}
|
||||
}
|
||||
for _, name := range managedLocalLaneTests {
|
||||
if _, ok := defined[name]; !ok {
|
||||
t.Errorf("%s is not defined in any cmd/bd test file a linux cgo build of bd_test compiles", name)
|
||||
}
|
||||
}
|
||||
if len(defined) < len(managedLocalLaneTests) {
|
||||
t.Errorf("found %d TestManagedLocalProxied* tests, want at least %d: %v", len(defined), len(managedLocalLaneTests), defined)
|
||||
}
|
||||
}
|
||||
|
|
@ -1734,6 +1734,14 @@ func TestBazelRetiredLanesCannotBeNarrowed(t *testing.T) {
|
|||
"BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1", "BEADS_TEST_REQUIRE_SOCAT": "1", "GOMAXPROCS": "4",
|
||||
},
|
||||
},
|
||||
"//cmd/bd:bd_managed_local_test": {
|
||||
[]string{"$(rootpath :bd_test)", "-test.run=^TestManagedLocalProxied", "-test.timeout=15m"},
|
||||
map[string]string{
|
||||
"BEADS_TEST_BD_BINARY": "$(rlocationpath :bd_for_tests)", "BEADS_TEST_DOLT_SERVER": "local",
|
||||
"BEADS_TEST_GOFMT": "$(rlocationpath @go_sdk//:bin/gofmt)", "BEADS_TEST_PREFLIGHT_GO": "$(rlocationpath :preflight_go_fixture)",
|
||||
"BEADS_TEST_PROXIED_LOCAL": "1", "BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1", "BEADS_TEST_SKIP": "dolt",
|
||||
},
|
||||
},
|
||||
"//internal/storage/dolt:dolt_server_conformance_test": {
|
||||
[]string{"$(rootpath :dolt_race_off)", "-test.v", "-test.count=1", "-test.timeout=15m", "-test.run=^TestConformance$$"},
|
||||
map[string]string{"BEADS_TEST_DOLT_SERVER": "local", "BEADS_TEST_GIT_IDENTITY": "1", "BEADS_TEST_REQUIRE_DOLT_CONTAINER": "1", "GOMAXPROCS": "4"},
|
||||
|
|
|
|||
|
|
@ -27,6 +27,13 @@ exports_files(
|
|||
visibility = ["//visibility:public"],
|
||||
)
|
||||
|
||||
# sh_test src that runs an existing go_test binary with no network but
|
||||
# loopback (see the script header), e.g. //cmd/bd:bd_managed_local_test.
|
||||
exports_files(
|
||||
["go_test_offline.sh"],
|
||||
visibility = ["//visibility:public"],
|
||||
)
|
||||
|
||||
# Wraps every test process (`test --run_under` in .bazelrc) with the isolated
|
||||
# HOME/XDG/Dolt/git environment that scripts/ci/lib/test-env.sh gives `go test`.
|
||||
# Bazel merges the run_under target's runfiles into every test's runfiles, so
|
||||
|
|
|
|||
24
tools/bazel/go_test_offline.sh
Executable file
24
tools/bazel/go_test_offline.sh
Executable file
|
|
@ -0,0 +1,24 @@
|
|||
#!/usr/bin/env bash
|
||||
# Runs an existing go_test binary (go_test_variant.sh's contract: the first
|
||||
# argument is its runfiles path, the rest go to the binary unchanged) inside a
|
||||
# fresh user and network namespace with only loopback up, as namespace root.
|
||||
# It is the unprivileged equivalent of `sudo unshare --net` plus `ip link set
|
||||
# lo up`: the binary and every process it starts (bd, its loopback proxy, dolt
|
||||
# sql-server) can bind and dial 127.0.0.1 and nothing else, which proves the
|
||||
# code under test needs no outbound network. The workers allow unprivileged
|
||||
# user namespaces; a host that does not fails here rather than running the
|
||||
# test with the network it is meant to be without.
|
||||
set -euo pipefail
|
||||
export GO_TEST_RUN_FROM_BAZEL=1
|
||||
exec unshare --user --map-root-user --net -- bash -euc '
|
||||
ip link set lo up
|
||||
# Prove the namespace really has no outbound network before trusting
|
||||
# anything the test reports.
|
||||
if timeout 3 bash -c "exec 3<>/dev/tcp/1.1.1.1/443" 2>/dev/null; then
|
||||
echo "go_test_offline: outbound network unexpectedly available inside the namespace" >&2
|
||||
exit 1
|
||||
fi
|
||||
bin="$1"
|
||||
shift
|
||||
exec "./$bin" "$@"
|
||||
' go_test_offline "$@"
|
||||
Loading…
Add table
Add a link
Reference in a new issue