Review F1. PR Risk's coverage decision read the mutable RBE_WEST_WORKERS variable in a different run from pr.yml's rbe job, so unsetting it and re-running one workflow could leave both required checks green with neither the legacy tier nor the Bazel lane having run the tier. - Both workflows now commit BAZEL_RETIRES_LEGACY_EMBEDDED (policy-tested equal) and run the identical bazel-embedded-coverage job: covered = flag && pull_request && !fork && executor secret present. RBE_WEST_WORKERS is no longer read by PR Risk at all. - pr.yml's ci-gate requires the job's result (BAZEL_EMBEDDED_COVERAGE) and BAZEL_EMBEDDED_RETIRED, which is red when covered but the Bazel embedded lane did not run remotely and pass (mode skip included). - Revert is a commit of the flag to "false" in both workflows; in-flight PRs pick it up on their next push. The kill-switch runbook now flips the flag first. Tests run both decision steps, bazel.yml's rbe step and pr.yml's real gate step over every event/var/secret/fork/flag combination, with the variable differing between the two runs, and require no double-green without the tier. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1070 lines
51 KiB
YAML
1070 lines
51 KiB
YAML
# Bazel side-by-side lane, gated alongside the legacy go test jobs. Runs
|
|
# `bazel test //... --config=ci`, the PR Core-parity selection (race, -short,
|
|
# dolt skip, -requires-docker), plus one job per mirrored CI job (pure-Go and
|
|
# js/wasm, the dolt-server lane, pr-risk.yml's embedded-Dolt, proxied-server
|
|
# and server-Dolt storage tiers, main.yml's integration jobs).
|
|
#
|
|
# PRs and merge groups run it through pr.yml's `bazel` job (workflow_call),
|
|
# in the same run as the legacy jobs it mirrors, which stay required too,
|
|
# except pr-risk.yml's embedded-Dolt test jobs: while the committed
|
|
# BAZEL_RETIRES_LEGACY_EMBEDDED flag is "true", on a same-repo pull_request
|
|
# with the executor secret (mode remote, or skip with RBE_WEST_WORKERS
|
|
# unset), pr-risk.yml's bazel-embedded-coverage job stands them down and
|
|
# bazel-embedded alone runs the tier; pr.yml's ci-gate then requires that
|
|
# lane to have run remotely and passed, so mode skip is red there
|
|
# (scripts/pr_risk_bazel_coverage_test.go).
|
|
# This file has no pull_request or merge_group trigger of its own, so a PR
|
|
# runs the lane once. Every lane reports its job.status as a workflow_call
|
|
# output (named after the job) from an always() last step. pr.yml's ci-gate
|
|
# requires the call's aggregate result (BAZEL) and bazel-test, bazel-pure,
|
|
# bazel-embedded, bazel-integration, bazel-doltserver, bazel-proxied and
|
|
# bazel-server-storage (BAZEL_TEST, BAZEL_PURE, BAZEL_EMBEDDED,
|
|
# BAZEL_INTEGRATION, BAZEL_DOLTSERVER, BAZEL_PROXIED, BAZEL_SERVER_STORAGE);
|
|
# .github/scripts/bazel-gate.sh turns the rbe job's rbe-mode output into the
|
|
# skips it accepts, so a lane skipped by its `if:` below passes and one that
|
|
# should have run and failed, was cancelled or reported nothing fails the
|
|
# gate. bazel-integration runs in the PR call (pr.yml passes integration:
|
|
# "on") although its legacy counterparts, main.yml's integration jobs, run
|
|
# only on push to main: remotely it is minutes, and it catches on the PR what
|
|
# main.yml would catch after the merge. Like every remote-only lane it skips
|
|
# for fork and Dependabot PRs (mode local); forks get it once the farm has a
|
|
# read-only cache for them. Direct runs (push to main, workflow_dispatch,
|
|
# nightly.yml's call) gate nothing.
|
|
#
|
|
# Execution mode, decided once by the rbe job (every other job needs it and
|
|
# reads its outputs; none re-derives the condition) and passed to
|
|
# .github/actions/setup-bazel:
|
|
# remote: repo variable RBE_WEST_WORKERS=true, rbe not off, not a fork PR
|
|
# (or a fork PR bazel-farm.yml authorized, below), and the secret
|
|
# RBE_WEST_EXECUTOR (grpcs:// endpoint) present, plus
|
|
# RBE_TLS_CERT, RBE_TLS_KEY (base64; RBE_TLS_CA optional): remote
|
|
# execution on rbe-west, instance "oss" (shared with gascity), from the
|
|
# Blacksmith runner.
|
|
# local: fork PRs, rbe=off dispatches, and same-repo runs without the
|
|
# secret (Dependabot PRs: same-repo, but no Actions secrets): local
|
|
# execution on ubuntu-latest, slower but the same results, for every job
|
|
# but the remote-only ones (bazel-embedded, bazel-integration,
|
|
# bazel-proxied, bazel-server-storage), which are skipped.
|
|
# skip: same-repo runs while RBE_WEST_WORKERS is unset (the farm switch),
|
|
# and authorized fork-farm runs without it or without the secret.
|
|
# Trusted-author fork PRs: bazel-farm.yml (pull_request_target, advisory)
|
|
# calls this workflow for fork PRs whose author and triggering user are on
|
|
# the base branch's .github/bazel-farm-allowlist.txt, with fork-farm:
|
|
# authorized and checkout-sha: the event's pinned head SHA. Only that
|
|
# combination, on a pull_request_target event, lets a fork run remotely; a
|
|
# fork's pull_request run (pr.yml, which the fork may edit) stays local
|
|
# whatever it passes, and has no secrets anyway. See
|
|
# engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md, "Trusted-author fork PRs".
|
|
# workflow_dispatch with rbe=off simulates a fork run. The decision is also a
|
|
# workflow_call output (rbe-mode, rbe-enabled), so pr.yml's gate can tell an
|
|
# expected skip from a missing lane without re-deriving it. Callers pass
|
|
# exactly the four RBE secrets below, never `secrets: inherit`.
|
|
#
|
|
# Accepted risk (same model as gascity's bazel-test.yml): on push, merge_group
|
|
# and same-repo pull_request runs, code from the PR head runs with the RBE
|
|
# client certificate. setup-bazel is itself PR code, the key file is readable
|
|
# by every later step and by locally executed Bazel actions, and a .bazelrc
|
|
# edit can point the BEP or the executor anywhere. Anyone who can push a
|
|
# branch here can therefore use or copy the certificate and write to the
|
|
# farm's cache. Fork PRs never get it, except those bazel-farm.yml
|
|
# authorizes (allowlisted author and sender, accepted as same-repo-level
|
|
# trust; that workflow's header lists what it adds): pull_request from a
|
|
# fork receives no secrets, bazel-farm.yml is the only pull_request_target
|
|
# caller (policy-tested), and forks cannot write the runner cache. Nor do
|
|
# Dependabot PRs, which run as same-repo but get no Actions secrets. In place: secrets are referenced by the setup step
|
|
# alone (plus the rbe job's emptiness test of RBE_WEST_EXECUTOR, which yields
|
|
# a boolean) and never by a workflow- or job-level env (policy-tested), decoded key
|
|
# material is masked in logs, and CI uploads only remotely executed results
|
|
# (--noremote_upload_local_results in .bazelrc's remote-exec config and in
|
|
# the generated rc), and the certificate
|
|
# is issued to beads CI alone, so it can be revoked without affecting other
|
|
# clients. Maintainers decided (2026-09-28) that same-repo PRs keep remote
|
|
# execution under this model. Still open: a beads-only RBE_INSTANCE and
|
|
# farm-side denial of client action-cache writes. Nothing downstream consumes
|
|
# artifacts built here yet; in pr.yml's run the bd artifact is
|
|
# bazel-ci-build-artifacts, so it cannot collide with the build-artifacts
|
|
# job's ci-build-artifacts.
|
|
#
|
|
# Runner cache: one entry per (.bazelversion, MODULE.bazel.lock) pair, about
|
|
# 2 GB (Bazel's content-addressable --repository_cache, re-hashed on every
|
|
# hit; never the Bazel binary or the repo contents cache), written
|
|
# only by a push to main that missed it. No --disk_cache: remote runs use the
|
|
# remote cache, and local runs would need about 7 GB more.
|
|
#
|
|
# Workflow-file changes suppress pull_request runs under the org approval
|
|
# policy; validate edits to this file with workflow_dispatch on the branch.
|
|
name: Bazel
|
|
|
|
on:
|
|
push:
|
|
branches: [ main ]
|
|
workflow_dispatch:
|
|
inputs:
|
|
rbe:
|
|
description: "Remote execution: on (use secrets) or off (fork simulation)"
|
|
required: true
|
|
type: choice
|
|
default: "on"
|
|
options: [ "on", "off" ]
|
|
details:
|
|
description: "List every equivalence divergence"
|
|
required: false
|
|
type: boolean
|
|
default: false
|
|
dolt-lane:
|
|
description: "Dolt-backed lane: doltserver (hermetic, remote) or docker (testcontainers A/B control)"
|
|
required: false
|
|
type: choice
|
|
default: "doltserver"
|
|
options: [ "doltserver", "docker" ]
|
|
workflow_call:
|
|
inputs:
|
|
rbe:
|
|
type: string
|
|
required: false
|
|
default: "on"
|
|
details:
|
|
type: boolean
|
|
required: false
|
|
default: true
|
|
go-test-json-artifact:
|
|
description: >
|
|
Artifact (from the calling run) holding go-test.json, PR Core's
|
|
`go-test -json` on this commit; enables the equivalence skip-parity
|
|
check. nightly.yml sets it.
|
|
type: string
|
|
required: false
|
|
default: ""
|
|
# A string, not a boolean: on push and dispatch inputs.integration is
|
|
# null, and null == false in GitHub's loose comparison, so a boolean
|
|
# test would silently stop the lane on main.
|
|
integration:
|
|
description: >
|
|
"off" skips bazel-integration; anything else (pr.yml and
|
|
bazel-farm.yml pass "on") runs it where remote execution is on.
|
|
type: string
|
|
required: false
|
|
default: "on"
|
|
build-artifact-name:
|
|
description: >
|
|
Name of the uploaded bd artifact (ci-build-artifacts layout). pr.yml
|
|
sets bazel-ci-build-artifacts: its build-artifacts job uploads
|
|
ci-build-artifacts in the same run.
|
|
type: string
|
|
required: false
|
|
default: "ci-build-artifacts"
|
|
# Set only by bazel-farm.yml (policy-tested): the fork PR's head SHA
|
|
# from its pull_request_target event, pinned, and "authorized" once
|
|
# that workflow's authorize job has checked the base branch's
|
|
# allowlist. See the header.
|
|
checkout-sha:
|
|
description: >
|
|
Commit every lane checks out (bazel-farm.yml: the trusted fork PR's
|
|
head SHA); empty checks out the event's default ref.
|
|
type: string
|
|
required: false
|
|
default: ""
|
|
fork-farm:
|
|
description: >
|
|
"authorized" (bazel-farm.yml only) lets a pull_request_target fork
|
|
run execute remotely; anything else keeps forks local.
|
|
type: string
|
|
required: false
|
|
default: "off"
|
|
# Only what the rbe job and setup-bazel need; callers pass them
|
|
# explicitly. None is required: fork and Dependabot runs have none.
|
|
secrets:
|
|
RBE_WEST_EXECUTOR:
|
|
required: false
|
|
RBE_TLS_CERT:
|
|
required: false
|
|
RBE_TLS_KEY:
|
|
required: false
|
|
RBE_TLS_CA:
|
|
required: false
|
|
# The rbe job's decision, and each lane's job.status (success, failure or
|
|
# cancelled; empty when the lane was skipped or never reported). pr.yml's
|
|
# ci-gate reads them through .github/scripts/bazel-gate.sh.
|
|
outputs:
|
|
rbe-enabled:
|
|
description: "'true' when the lanes executed remotely on rbe-west, else 'false'"
|
|
value: ${{ jobs.rbe.outputs.enabled }}
|
|
rbe-mode:
|
|
description: "remote, local (no remote-only jobs) or skip (no jobs): the rbe job's decision"
|
|
value: ${{ jobs.rbe.outputs.mode }}
|
|
bazel-test:
|
|
description: "bazel-test's job.status"
|
|
value: ${{ jobs.bazel-test.outputs.result }}
|
|
bazel-pure:
|
|
description: "bazel-pure's job.status"
|
|
value: ${{ jobs.bazel-pure.outputs.result }}
|
|
bazel-embedded:
|
|
description: "bazel-embedded's job.status"
|
|
value: ${{ jobs.bazel-embedded.outputs.result }}
|
|
bazel-integration:
|
|
description: "bazel-integration's job.status"
|
|
value: ${{ jobs.bazel-integration.outputs.result }}
|
|
bazel-doltserver:
|
|
description: "bazel-doltserver's job.status"
|
|
value: ${{ jobs.bazel-doltserver.outputs.result }}
|
|
bazel-proxied:
|
|
description: "bazel-proxied's job.status"
|
|
value: ${{ jobs.bazel-proxied.outputs.result }}
|
|
bazel-server-storage:
|
|
description: "bazel-server-storage's job.status"
|
|
value: ${{ jobs.bazel-server-storage.outputs.result }}
|
|
|
|
# Main runs always finish (they may save the runner cache); PR runs (called
|
|
# from pr.yml or bazel-farm.yml, whose own groups cancel the calling run too)
|
|
# supersede. The group differs from each caller's, as a called workflow's
|
|
# must.
|
|
concurrency:
|
|
group: bazel-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_target' }}
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
env:
|
|
BD_DISABLE_METRICS: "1"
|
|
BD_DISABLE_EVENT_FLUSH: "1"
|
|
# Phase B report-only budget for T1/T2 (tools/bazel/critpath.py).
|
|
BAZEL_BUDGET_SECONDS: "420"
|
|
|
|
jobs:
|
|
# The one place the execution mode is decided (see the header), so every
|
|
# job sees the same answer for the whole run. Its step reads one secret, and
|
|
# only to test it for emptiness: the step's env holds a boolean, never the
|
|
# value, and the step runs no repository code (no checkout). Dependabot
|
|
# runs read vars but get no secrets, so they come out local, not remote.
|
|
rbe:
|
|
name: execution mode
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
outputs:
|
|
enabled: ${{ steps.decide.outputs.enabled }}
|
|
mode: ${{ steps.decide.outputs.mode }}
|
|
steps:
|
|
# The expressions' == is case-insensitive (RBE_WEST_WORKERS=True counts).
|
|
# A push, merge_group or dispatch has no pull_request, so FORK is false.
|
|
- name: Decide remote execution
|
|
id: decide
|
|
env:
|
|
RBE_VAR_ON: ${{ vars.RBE_WEST_WORKERS == 'true' }}
|
|
RBE_INPUT_OFF: ${{ inputs.rbe == 'off' }}
|
|
FORK: ${{ github.event.pull_request.head.repo.fork == true }}
|
|
FORK_FARM: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' && inputs.checkout-sha != '' }}
|
|
HAS_EXECUTOR: ${{ secrets.RBE_WEST_EXECUTOR != '' }}
|
|
run: |
|
|
for v in RBE_VAR_ON RBE_INPUT_OFF FORK FORK_FARM HAS_EXECUTOR; do
|
|
case "${!v}" in
|
|
true|false) ;;
|
|
*) echo "::error::$v is '${!v}', want true or false"; exit 1 ;;
|
|
esac
|
|
done
|
|
# A fork runs locally unless bazel-farm.yml authorized it; an
|
|
# authorized fork run is remote or nothing (pr.yml's call already
|
|
# runs it locally).
|
|
if [ "$RBE_INPUT_OFF" = true ] || { [ "$FORK" = true ] && [ "$FORK_FARM" != true ]; }; then
|
|
mode=local
|
|
elif [ "$RBE_VAR_ON" != true ]; then
|
|
mode=skip
|
|
elif [ "$HAS_EXECUTOR" = true ]; then
|
|
mode=remote
|
|
elif [ "$FORK_FARM" = true ]; then
|
|
mode=skip
|
|
else
|
|
mode=local
|
|
fi
|
|
enabled=false
|
|
if [ "$mode" = remote ]; then enabled=true; fi
|
|
echo "mode=$mode" >> "$GITHUB_OUTPUT"
|
|
echo "enabled=$enabled" >> "$GITHUB_OUTPUT"
|
|
echo "Bazel execution mode: **$mode** (RBE_WEST_WORKERS on: $RBE_VAR_ON, rbe=off: $RBE_INPUT_OFF, fork: $FORK, fork farm: $FORK_FARM, executor secret: $HAS_EXECUTOR)" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
|
|
bazel-test:
|
|
name: test
|
|
# Remote-execution runs use the org's Blacksmith pool (the farm's edge
|
|
# admits it; GitHub-hosted runners are refused) and do not count against
|
|
# GitHub's concurrent-job cap. Local runs (forks, rbe=off, no secrets)
|
|
# build everything on the runner, so they get the larger GitHub-hosted
|
|
# one.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
# Remote execution goes to rbe-west (instance "oss", the Blacksmith-backed
|
|
# worker pool shared with gascity). Skipped only in mode skip (same-repo
|
|
# runs while RBE_WEST_WORKERS is unset).
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 100
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
# Go is for tools/bazel/equivalence.py's `go list` and for make; Bazel
|
|
# uses its own hermetic SDK.
|
|
- name: Set up Go
|
|
id: setup-go
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
# Read-only: main.yml's build-artifacts job owns this cache.
|
|
- name: Restore Go module cache
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/go/pkg/mod
|
|
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
|
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
|
|
|
# Go does not re-hash extracted module directories on use; a restored
|
|
# cache written by an untrusted run is checked against go.sum first.
|
|
- name: Verify restored Go modules
|
|
run: go mod verify
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# T1: remote + restored repository cache; T2: remote, cold runner; local:
|
|
# no remote execution (reported, never budgeted).
|
|
- name: Measurement tier
|
|
id: tier
|
|
env:
|
|
REMOTE: ${{ steps.bazel.outputs.remote }}
|
|
RESTORED: ${{ steps.bazel.outputs.cache-restored }}
|
|
run: |
|
|
if [ "$REMOTE" = "true" ] && [ "$RESTORED" = "true" ]; then tier=T1
|
|
elif [ "$REMOTE" = "true" ]; then tier=T2
|
|
else tier=local; fi
|
|
echo "tier=$tier" >> "$GITHUB_OUTPUT"
|
|
{
|
|
echo "### Bazel / test"
|
|
echo
|
|
echo "Tier **$tier**: remote execution $REMOTE, runner cache restored $RESTORED."
|
|
echo
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# --profile and --build_event_json_file are command options, so they go
|
|
# after `test` (a startup position fails).
|
|
- name: bazel test //... --config=ci
|
|
id: test
|
|
timeout-minutes: 90
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=ci \
|
|
--profile="$RUNNER_TEMP/bazel-profile.json" \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
echo >> "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# pr.yml's build-artifacts job, built by Bazel: the same artifact name
|
|
# and layout (bd-linux-gms-pure, SHA256SUMS, build-manifest.txt), so its
|
|
# consumers (PR Core, domain+uow, the package gates) can switch to it.
|
|
# None does yet. The script documents which binary and how it differs
|
|
# from `go build`'s.
|
|
- name: Package bd (ci-build-artifacts layout)
|
|
id: bd-artifact
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: ./scripts/ci/package-bazel-bd.sh "$RUNNER_TEMP/bd-artifacts"
|
|
|
|
- name: Upload build artifacts
|
|
if: ${{ always() && steps.bd-artifact.outcome == 'success' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: ${{ inputs.build-artifact-name || 'ci-build-artifacts' }}
|
|
path: ${{ runner.temp }}/bd-artifacts/
|
|
retention-days: 1
|
|
if-no-files-found: error
|
|
|
|
- name: Critical-path report
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
env:
|
|
TIER: ${{ steps.tier.outputs.tier }}
|
|
run: |
|
|
budget=()
|
|
if [ "$TIER" != "local" ]; then budget=(--budget "$BAZEL_BUDGET_SECONDS"); fi
|
|
python3 tools/bazel/critpath.py "$RUNNER_TEMP/bazel-profile.json" \
|
|
--tier "$TIER" --log "$RUNNER_TEMP/bazel-test.log" "${budget[@]}"
|
|
|
|
# nightly.yml's PR Core `go test -json` on the same commit.
|
|
- name: Download go-test results
|
|
if: ${{ always() && inputs.go-test-json-artifact != '' && steps.test.outcome != 'skipped' }}
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: ${{ inputs.go-test-json-artifact }}
|
|
path: ${{ runner.temp }}/go-test-json
|
|
|
|
# Fails on any Go test that `go test` runs and Bazel did not (tag filter,
|
|
# manual, missing target, empty shard) unless it is allowlisted. With
|
|
# go test results (nightly) it also fails on tests that passed under go
|
|
# test and were skipped under Bazel; a PR run cannot see those.
|
|
# It reads test.xml from bazel-testlogs, which --config=integration
|
|
# shares with --config=ci (same output directory): if a job ever runs
|
|
# both lanes, run each lane's equivalence right after its own
|
|
# `bazel test`, before the other lane overwrites the logs.
|
|
- name: Go test equivalence
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
timeout-minutes: 15
|
|
env:
|
|
DETAILS: ${{ inputs.details && '--details' || '' }}
|
|
GO_TEST_JSON: ${{ inputs.go-test-json-artifact != '' && format('{0}/go-test-json/go-test.json', runner.temp) || '' }}
|
|
run: |
|
|
args=()
|
|
if [ -n "$DETAILS" ]; then args+=("$DETAILS"); fi
|
|
if [ -n "$GO_TEST_JSON" ]; then args+=(--go-test-json "$GO_TEST_JSON"); fi
|
|
python3 tools/bazel/equivalence.py --bep "$RUNNER_TEMP/bazel-bep.json" "${args[@]}"
|
|
|
|
# Runs last: it rewrites BUILD files in the checkout. main moves BUILD
|
|
# inputs (imports, go.mod) without touching Bazel, so drift shows up here
|
|
# first. On drift the step still fails, and the BUILD.bazel /
|
|
# MODULE.bazel / MODULE.bazel.lock part of the fix is written as a patch
|
|
# for the bazel-sync-patch artifact, which bazel-autofix.yml pushes to
|
|
# same-repo PR branches (or turns into an apply recipe for forks).
|
|
- name: BUILD files in sync (gazelle, go_srcs, MODULE.bazel)
|
|
id: sync
|
|
if: ${{ always() && steps.bazel.outcome == 'success' }}
|
|
timeout-minutes: 15
|
|
env:
|
|
PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
run: |
|
|
start=$(date +%s)
|
|
# A stale go_srcs block is drift like any other: record it, then let
|
|
# bazel-sync fix it so the patch covers it too.
|
|
check=0
|
|
make bazel-sync-check || check=$?
|
|
make bazel-sync
|
|
drift="$(git status --porcelain --untracked-files=all)"
|
|
echo "BUILD sync check: $(( $(date +%s) - start ))s" >> "$GITHUB_STEP_SUMMARY"
|
|
if [ -n "$drift" ]; then
|
|
echo "::error title=Bazel BUILD files out of sync::run 'make bazel-sync' and commit the result"
|
|
echo "$drift"
|
|
git diff | head -200
|
|
echo "BUILD files **out of sync**; run \`make bazel-sync\` and commit the result." >> "$GITHUB_STEP_SUMMARY"
|
|
./scripts/ci/bazel-sync-patch.sh "$RUNNER_TEMP/bazel-sync-patch"
|
|
exit 1
|
|
fi
|
|
exit "$check"
|
|
|
|
# Data only: bazel-autofix.yml validates every path before applying it.
|
|
# Not from bazel-farm.yml's pull_request_target runs: autofix reads only
|
|
# "PR" runs, and a trusted-fork run publishes nothing meant for a
|
|
# privileged consumer.
|
|
- name: Upload BUILD sync patch
|
|
if: ${{ always() && steps.sync.outcome == 'failure' && github.event_name != 'pull_request_target' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-sync-patch
|
|
path: ${{ runner.temp }}/bazel-sync-patch/
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome == 'failure' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Only a push to main that missed the exact key writes the runner cache.
|
|
# Its content (--repository_cache only, never the rc
|
|
# or credentials) is fixed by .bazelversion and MODULE.bazel.lock, so one
|
|
# entry per lock is enough and later pushes do not churn the quota.
|
|
- name: Save Bazel runner cache
|
|
if: ${{ always() && github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.bazel.outcome == 'success' && steps.bazel.outputs.cache-hit != 'true' }}
|
|
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ${{ runner.temp }}/bazel-ci-cache
|
|
key: bazel-repo-v3-${{ runner.os }}-${{ hashFiles('.bazelversion', 'MODULE.bazel.lock') }}
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr.yml's check-cmd-bd-puregeo-tests job ("Check pure-Go and js/wasm
|
|
# boundaries (CGO_ENABLED=0)") with Bazel-built binaries: --config=pure
|
|
# builds bd and the job's three pure test binaries and runs its -short
|
|
# cmd/bd subset (PURE_CMD_BD_TESTS, kept equal to the job's -run selector by
|
|
# scripts/ci_workflow_test.go); --config=js-wasm builds the hooks test for
|
|
# js/wasm, which the Go SDK's go_js_wasm_exec runs under node with the job's
|
|
# exact pass-count guard.
|
|
bazel-pure:
|
|
name: pure-Go and js/wasm
|
|
# Same runner and skip rule as bazel-test.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 90
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
env:
|
|
PURE_CMD_BD_TESTS: '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir|NocgoFactoriesHonorSharedServer|NocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag|NocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag)'
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
# Only for go_js_wasm_exec and wasm_exec_node.js, which must match the
|
|
# Go version Bazel compiles with (go.mod's; policy-tested).
|
|
- name: Set up Go
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v6
|
|
with:
|
|
node-version: '24'
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
- name: Run js/wasm hook boundary
|
|
run: |
|
|
set -euo pipefail
|
|
bazel build --config=js-wasm //internal/hooks:hooks_test
|
|
bin="$(bazel cquery --config=js-wasm --output=files //internal/hooks:hooks_test)"
|
|
cd internal/hooks
|
|
|
|
# Same selector and count guard as pr.yml's job.
|
|
test_status=0
|
|
# wasm_exec.js caps argv+env size; the Blacksmith runner's environment
|
|
# exceeds it, so run with only what the hook test needs.
|
|
wasm_exec="$(go env GOROOT)/lib/wasm/go_js_wasm_exec"
|
|
test_output="$(env -i PATH="$PATH" HOME="$HOME" TMPDIR="$RUNNER_TEMP" \
|
|
BD_DISABLE_METRICS=1 BD_DISABLE_EVENT_FLUSH=1 \
|
|
"$wasm_exec" "$GITHUB_WORKSPACE/$bin" \
|
|
-test.run '^TestRunHookReportsUnsupportedExecution$' \
|
|
-test.v 2>&1)" || test_status=$?
|
|
printf '%s\n' "$test_output"
|
|
(( test_status == 0 )) || exit "$test_status"
|
|
|
|
run_count=0
|
|
pass_count=0
|
|
nonpass_count=0
|
|
pass_pattern='^--- PASS: TestRunHookReportsUnsupportedExecution \([0-9.]+s\)$'
|
|
nonpass_pattern='^[[:space:]]*--- (FAIL|SKIP): '
|
|
while IFS= read -r line; do
|
|
if [[ "$line" == "=== RUN TestRunHookReportsUnsupportedExecution" ]]; then
|
|
((run_count += 1))
|
|
fi
|
|
if [[ "$line" =~ $pass_pattern ]]; then
|
|
((pass_count += 1))
|
|
fi
|
|
if [[ "$line" =~ $nonpass_pattern ]]; then
|
|
((nonpass_count += 1))
|
|
fi
|
|
done <<< "$test_output"
|
|
if (( run_count != 1 || pass_count != 1 || nonpass_count != 0 )); then
|
|
printf 'expected one exact js/wasm hook test run and pass with no failures or skips; got runs=%s passes=%s nonpasses=%s\n' \
|
|
"$run_count" "$pass_count" "$nonpass_count" >&2
|
|
exit 1
|
|
fi
|
|
|
|
- name: Build cmd/bd and pure-Go test binaries (--config=pure)
|
|
run: >-
|
|
bazel build --config=pure
|
|
//cmd/bd:bd
|
|
//cmd/bd:bd_test
|
|
//internal/storage/embeddeddolt:embeddeddolt_test
|
|
//internal/tracker:tracker_test
|
|
|
|
# `go build` with CGO_ENABLED=0 rejects a pure binary whose imports reach
|
|
# dolt's cgo-only store/nbs (gozstd). Bazel compiles gozstd's pure stubs
|
|
# instead, whose init panics (third_party/patches/gozstd_nocgo.patch), so
|
|
# the check happens here: every pure artifact is started once. bd_test
|
|
# starts in the next step. -test.run=^$ runs no test (TestMain still
|
|
# runs; BEADS_TEST_SKIP=dolt keeps tracker's from starting a container).
|
|
- name: Start every pure-Go artifact (gozstd contamination check)
|
|
run: |
|
|
set -euo pipefail
|
|
bazel run --config=pure //cmd/bd:bd -- version
|
|
bazel test --config=pure \
|
|
//internal/storage/embeddeddolt:embeddeddolt_test \
|
|
//internal/tracker:tracker_test \
|
|
--test_sharding_strategy=disabled \
|
|
'--test_arg=-test.run=^$' \
|
|
--test_env=BEADS_TEST_SKIP=dolt
|
|
|
|
# One unsharded process, like `go test -run`; the test.xml check keeps a
|
|
# selector that matches nothing from passing.
|
|
- name: Run pure-Go cmd/bd test subset (--config=pure)
|
|
run: |
|
|
set -euo pipefail
|
|
bazel test --config=pure //cmd/bd:bd_test \
|
|
--test_sharding_strategy=disabled \
|
|
"--test_arg=-test.run=$PURE_CMD_BD_TESTS"
|
|
n="$(grep -c '<testcase ' bazel-testlogs/cmd/bd/bd_test/test.xml || true)"
|
|
echo "pure cmd/bd subset: $n test cases"
|
|
(( n > 0 ))
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr-risk.yml's embedded-Dolt tier ("Test (Embedded Dolt Cmd N/20)",
|
|
# "Test (Embedded Dolt Storage N/5)", "Test (Embedded Dolt Conformance -
|
|
# core/audit)") as --config=embedded: the embedded-tagged variants of
|
|
# //cmd/bd:bd_test and //internal/storage/embeddeddolt:embeddeddolt_test.
|
|
# The shard variants run the jobs' own shard scripts and manifests, so Bazel
|
|
# shard k runs exactly the tests of job k. A job of its own rather than a
|
|
# second invocation in bazel-test: the tier's long race shards stay off
|
|
# bazel-test's critical path and budget, and it compiles nothing bazel-test
|
|
# does not (same race configuration, shared through the remote cache).
|
|
bazel-embedded:
|
|
name: embedded Dolt tier
|
|
# Remote execution only: skipped unless the rbe job chose remote (no
|
|
# fork, rbe=off or secret-less run). Locally the tier is 27 race test
|
|
# processes (~7-10k test-seconds) on one runner, an hour or more per run
|
|
# on the pool whose job cap this track exists to relieve, and it proves
|
|
# nothing pr-risk.yml's embedded jobs (which do run for forks) do not.
|
|
# On same-repo pull_requests those jobs stand down (pr-risk.yml's
|
|
# bazel-embedded-coverage), so this lane is then the tier's only PR run
|
|
# and pr.yml's gate requires it to have run here and passed.
|
|
# The runs-on expression is the shared one and always picks the
|
|
# Blacksmith runner under this if.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 60
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# --build_event_json_file is a command option, so it goes after `test`.
|
|
- name: bazel test //... --config=embedded
|
|
id: test
|
|
timeout-minutes: 50
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=embedded \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=embedded: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# A shard the manifest assigns nothing, or a selector that matches
|
|
# nothing, exits 0: fail if any target's or shard's test.xml lists no
|
|
# tests (or is missing).
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-embedded-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# main.yml's "Main Linux integration packages (N/6)" and "Main Linux
|
|
# integration cmd/bd (N/8)" jobs (nightly.yml's Full Test Suite runs the
|
|
# same go test in one job) as --config=integration: every package with
|
|
# -tags=integration,gms_pure_go, race, no -short, BEADS_TEST_SKIP=dolt (see
|
|
# .bazelrc). A job of its own: ci and integration share bazel-testlogs, so
|
|
# bazel-test's equivalence step would read this lane's test.xml if both ran
|
|
# in one job, and the tagged race build is a second full compile.
|
|
# No equivalence step: equivalence.py's default mode is the --config=ci
|
|
# selection (go list without the integration tag, -short, ^TestEmbedded
|
|
# skipped), and its job mode needs the integration jobs' go test -json,
|
|
# which no run here has. check_testcases.py still catches a target or
|
|
# shard that ran no tests. pr.yml's gate requires it (BAZEL_INTEGRATION)
|
|
# on same-repo PRs; see the header.
|
|
bazel-integration:
|
|
name: integration
|
|
# Remote execution only, like bazel-embedded: locally the tagged race
|
|
# build of //... plus 14 jobs' worth of tests on one runner is well over
|
|
# the timeout, for tests main.yml's integration jobs already run. A
|
|
# caller may turn it off with integration: "off"; pr.yml and
|
|
# bazel-farm.yml pass "on".
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' && inputs.integration != 'off' }}
|
|
timeout-minutes: 55
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# Remote: a cold compile (a PR touching a widely imported package; the
|
|
# first cold run took 17 minutes end to end, warm ones ~3.5) plus the
|
|
# longest test action (1200s cap, see .bazelrc) if it starts last.
|
|
# scripts/ci_workflow_test.go keeps this at or above that sum.
|
|
- name: bazel test //... --config=integration
|
|
id: test
|
|
timeout-minutes: 45
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=integration \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded. dolt_version_test is a plain sh_test (no Go
|
|
# testcases in its test.xml), which --config=integration's tag filter
|
|
# keeps, unlike the embedded one.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json" --not-go //tools/bazel:dolt_version_test
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-integration-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr.yml's container-backed jobs, "Test (storage domain + uow)" and
|
|
# "Contract corpus", as Bazel's dolt-server lane: --config=doltserver runs
|
|
# the dolt-server variants (domain/..., uow in 8 shards, tracker/...,
|
|
# doctor/fix, protocol in 8 shards), each of which starts its own dolt
|
|
# sql-server from the pinned dolt, so they execute on the remote workers
|
|
# like any other test (locally for forks and rbe=off). The docker lane
|
|
# (--config=docker: the same suites against testcontainers on this runner,
|
|
# no-remote-exec) stays the A/B control: dispatch with dolt-lane=docker.
|
|
bazel-doltserver:
|
|
name: dolt-server lane
|
|
# Same runner and skip rule as bazel-test.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 90
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
env:
|
|
BAZEL_DOLT_LANE: ${{ inputs.dolt-lane || 'doltserver' }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# Docker lane only (the A/B control), as in the mirrored jobs; its
|
|
# tests fail rather than skip without the image.
|
|
- name: Pull Dolt sql-server image
|
|
if: ${{ env.BAZEL_DOLT_LANE == 'docker' }}
|
|
run: ./scripts/ci/pull-dolt-image.sh
|
|
|
|
- name: bazel test //... --config=doltserver
|
|
id: test
|
|
timeout-minutes: 80
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... "--config=$BAZEL_DOLT_LANE" 2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=$BAZEL_DOLT_LANE: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome == 'failure' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-${{ env.BAZEL_DOLT_LANE }}-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)", as
|
|
# --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the job's
|
|
# shard script in 15 Bazel shards, so shard k runs exactly job k's tests.
|
|
# Same build as --config=doltserver (race bd_test, non-race bd_for_tests),
|
|
# shared with bazel-doltserver through the remote cache. PR Risk builds
|
|
# its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd twin
|
|
# uses build-artifacts' non-race bd-cmd-test, so this lane is the stricter
|
|
# of the two. A job of its own, not gated by any integration switch: it
|
|
# runs in pr.yml's call and its gate requires it (BAZEL_PROXIED).
|
|
bazel-proxied:
|
|
name: proxied-server tier
|
|
# Remote execution only, like bazel-embedded: 15 shards of ~5-10
|
|
# minutes, each with a dolt sql-server busy on ~4 cores, do not fit a
|
|
# GitHub-hosted runner.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# Remote: the longest shard takes ~5-8 minutes (each shard's Go
|
|
# timeout is its job's 15m, 1200s Bazel cap), plus a compile.
|
|
- name: bazel test //... --config=doltserver-proxied
|
|
id: test
|
|
timeout-minutes: 25
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=doltserver-proxied \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=doltserver-proxied: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-proxied-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr-risk.yml's server-Dolt storage tier, "Test (Server Dolt Conformance)"
|
|
# and "Test (Server Dolt Full Suite N/16)", as --config=doltserver-integration:
|
|
# the conformance and 16-shard full-suite variants of
|
|
# //internal/storage/dolt over a non-race dolt_test build, each shard
|
|
# running the job's own shard script. --config=doltserver-integration has
|
|
# exactly --config=integration's build flags, so it compiles nothing
|
|
# bazel-integration does not (shared through the remote cache). A job of
|
|
# its own, not gated by any integration switch: this is a PR-time tier,
|
|
# unlike main.yml's integration jobs, and pr.yml's gate requires it
|
|
# (BAZEL_SERVER_STORAGE).
|
|
bazel-server-storage:
|
|
name: server-Dolt storage tier
|
|
# Remote execution only, like bazel-integration: locally the tagged race
|
|
# build of //... alone is most of the timeout.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.enabled == 'true' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.enabled == 'true' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.enabled == 'true' && 'oss' || '' }}
|
|
|
|
# Remote: the longest shard takes ~2-3 minutes (the storage suite's Go
|
|
# timeout is 15m per shard, 1200s Bazel cap), plus a compile.
|
|
- name: bazel test //... --config=doltserver-integration
|
|
id: test
|
|
timeout-minutes: 25
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=doltserver-integration \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=doltserver-integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: bazel-server-storage-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|