fast-checks, PR Policy, and the Linux leg of check-doc-freshness-platforms ran repository guards on GitHub runners. They now run as Bazel tests in bazel.yml's `test` lane (remote, cached), each over only the files it reads: - tools/bazel/repo_subset.bzl picks a narrow, path-selected slice of //:repo_files at analysis time. scripts/repochecks/run_check.sh copies that slice into a throwaway git checkout and runs the unchanged guard there. - //scripts/repochecks: build_tags, workflow_tags, go_install_guidance, winget_portable_alias, migration_hygiene (checks A and B), fmt (registered SDK gofmt), doc_freshness, workapi_frontend_boundary, types_gen_drift (`make api-check` drift, oapi-codegen from go.mod). - //scripts:doc_freshness_required_test (integration lane, -required-suite) and //scripts/gitattributespolicy:gitattributespolicy_required_host_test cover the Linux doc-freshness leg. - check-versions stays covered by //internal/versioncheck:versioncheck_test. check-testing-short stays covered by //scripts:scripts_test. pr-policy-wrapper is deleted. fast-checks keeps only the two checks that need the PR base: migration hygiene's delta half (MIGRATION_HYGIENE_SCOPE= delta) and the .beads diff. check-doc-flags stays, because it builds bd from the pinned release tag. The example Go modules leave .bazelignore so gofmt and the grep guards see them. check_testcases.py now accepts --not-go //pkg:all. Refs: ga-96smfk.20 Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2066 lines
109 KiB
YAML
2066 lines
109 KiB
YAML
# Bazel side-by-side lane, gated alongside the legacy go test jobs. Runs
|
|
# `bazel test //... --config=ci`, the PR Core-parity selection (race, -short,
|
|
# dolt skip, -requires-docker), plus one job per mirrored CI job (pure-Go and
|
|
# js/wasm, the dolt-server lane, pr-risk.yml's embedded-Dolt, proxied-server
|
|
# and server-Dolt storage tiers, main.yml's integration jobs).
|
|
#
|
|
# PRs and merge groups run it through pr.yml's `bazel` job (workflow_call),
|
|
# in the same run as the legacy jobs it mirrors, which stay required too,
|
|
# except where their D2 flag retires them: while the committed
|
|
# BAZEL_RETIRES_LEGACY_EMBEDDED / BAZEL_RETIRES_LEGACY_DOLT_SERVER_TIERS /
|
|
# BAZEL_RETIRES_LEGACY_PR_LANES flag is "true", on a same-repo,
|
|
# non-Dependabot pull_request or a merge_group run (whatever the mode), the
|
|
# bazel-coverage job stands down pr-risk.yml's embedded-Dolt test jobs (bazel-embedded runs
|
|
# the tier), its proxied-server and server-Dolt storage test jobs
|
|
# (bazel-proxied, bazel-server-storage), or pr.yml's PR Core,
|
|
# build-artifacts, pure-Go/js-wasm, domain+uow and contract corpus jobs
|
|
# (bazel-test, bazel-pure, bazel-doltserver; their `bazel test`s then add
|
|
# --config=sole-run); pr.yml's ci-gate then requires those lanes to have run
|
|
# remotely and passed, so modes skip, local and cache are red there
|
|
# (scripts/pr_risk_bazel_coverage_test.go, pr_lanes_bazel_coverage_test.go).
|
|
# Test results are cached (read and written) on every run but nightly's: a
|
|
# test whose action key is unchanged reuses the PASS the farm recorded for
|
|
# it, on a PR, a merge group (merge queue) or a push to main alike, and only
|
|
# what changed runs. nightly.yml passes fresh-test-results: true, so every
|
|
# `bazel test` here appends --config=fresh (BAZEL_FRESH below;
|
|
# --nocache_test_results) and re-executes every test once a night, which
|
|
# keeps sampling flaky and -race failures. A cached failure is never reused.
|
|
# Policy: scripts/ci_merge_queue_test.go.
|
|
# This file has no pull_request or merge_group trigger of its own, so a PR
|
|
# runs the lane once. Every lane reports its job.status as a workflow_call
|
|
# output (named after the job) from an always() last step. pr.yml's ci-gate
|
|
# requires the call's aggregate result (BAZEL) and bazel-test, bazel-pure,
|
|
# bazel-embedded, bazel-integration, bazel-doltserver, bazel-proxied and
|
|
# bazel-server-storage (BAZEL_TEST, BAZEL_PURE, BAZEL_EMBEDDED,
|
|
# BAZEL_INTEGRATION, BAZEL_DOLTSERVER, BAZEL_PROXIED, BAZEL_SERVER_STORAGE);
|
|
# bazel-cmd-dolt (the cmd/bd Dolt-server tier, BAZEL_CMD_DOLT) is advisory:
|
|
# job-level continue-on-error, and pr.yml requires it only while its
|
|
# BAZEL_CMD_DOLT_REQUIRED flag is "true" (pr.yml then also passes
|
|
# cmd-dolt-required: true, and bazel-integration leaves //cmd/bd:bd_test,
|
|
# which bazel-cmd-dolt runs too, to it);
|
|
# .github/scripts/bazel-gate.sh turns the rbe job's rbe-mode output into the
|
|
# skips it accepts, so a lane skipped by its `if:` below passes and one that
|
|
# should have run and failed, was cancelled or reported nothing fails the
|
|
# gate. bazel-integration runs in the PR call (pr.yml passes integration:
|
|
# "on") although its legacy counterparts, main.yml's integration jobs, run
|
|
# only on push to main: remotely it is minutes, and it catches on the PR what
|
|
# main.yml would catch after the merge. Fork and Dependabot PRs run it
|
|
# remotely in modes fork-ro/fork-rw like every lane, and in mode cache
|
|
# (rbe-fork closed) unlike the remote-only lanes: with the read-only cache
|
|
# only what the PR changed is built and tested on the runner, and it is
|
|
# their only PR-time integration run (no legacy job runs it on PRs). Mode
|
|
# local (rbe=off) still skips it. Direct runs (push to main,
|
|
# workflow_dispatch, nightly.yml's call) gate nothing.
|
|
#
|
|
# Execution mode, decided once by the rbe job (every other job needs it and
|
|
# reads its outputs; none re-derives the condition) and passed to
|
|
# .github/actions/setup-bazel:
|
|
# remote: repo variable RBE_WEST_WORKERS=true, rbe not off, not a fork or
|
|
# Dependabot PR (or a fork PR bazel-farm.yml authorized, below), and the
|
|
# secret RBE_WEST_EXECUTOR (grpcs:// endpoint) present, plus
|
|
# RBE_TLS_CERT, RBE_TLS_KEY (base64; RBE_TLS_CA optional): remote
|
|
# execution on rbe-west, instance "oss" (shared with gascity), from the
|
|
# Blacksmith runner.
|
|
# fork-ro, fork-rw: fork and Dependabot pull_request runs while rbe-west's
|
|
# certificate mint (rbe-fork-mint, infra nativelink-cas/west README
|
|
# "rbe-fork") says open for this run. Such runs get no secrets, no OIDC
|
|
# token and no repository variables, so the farm decides: the rbe job
|
|
# asks the mint's /v1/status, and every lane then proves it is a
|
|
# running job of this run (a CSR uploaded as an artifact of it) and gets
|
|
# its own certificate of about two hours (setup-bazel,
|
|
# fork-credential.sh). fork-ro: instance "oss-fork", the fork pool,
|
|
# nothing cached for anyone; fork-rw (PRs from an allowlisted author's
|
|
# own fork, triggered by an allowlisted user; the mint reads the
|
|
# default branch's .github/bazel-farm-allowlist.txt): instance "oss",
|
|
# the trusted OSS pool, results cached by its workers. Every lane runs,
|
|
# remotely, as in mode remote, from a GitHub-hosted runner (the client
|
|
# only). A lane whose certificate says another tier than this job fails
|
|
# ("re-run all jobs").
|
|
# cache: rbe=cache dispatches, fork and Dependabot PRs while rbe-fork is
|
|
# closed, unreachable or refuses the run (the mint's canary list, its
|
|
# switches, the farm's gate), and same-repo runs without the secret:
|
|
# like local (ubuntu-latest, bazel-embedded, bazel-proxied and
|
|
# bazel-server-storage skipped), plus rbe-west's anonymous read-only
|
|
# cache (.bazelrc --config=fork-cache): hits for every action trusted CI
|
|
# already executed, no credentials, no uploads; and bazel-integration
|
|
# runs too (a warm run hits all but what the PR changed). The farm can
|
|
# close the cache at any time; Bazel then runs everything locally.
|
|
# local: rbe=off dispatches (fork simulation without the cache): local
|
|
# execution on ubuntu-latest, slower but the same results, for every job
|
|
# but the remote-only ones (bazel-embedded, bazel-proxied,
|
|
# bazel-server-storage) and bazel-integration, which are skipped.
|
|
# skip: same-repo runs while RBE_WEST_WORKERS is unset (the farm switch),
|
|
# and authorized fork-farm runs without it or without the secret. Fork
|
|
# and Dependabot pull_request runs cannot read the variable; rbe-fork's
|
|
# own switches decide for them.
|
|
# Trusted-author fork PRs: bazel-farm.yml (pull_request_target, advisory)
|
|
# calls this workflow for fork PRs whose author and triggering user are on
|
|
# the base branch's .github/bazel-farm-allowlist.txt, with fork-farm:
|
|
# authorized and checkout-sha: the event's pinned head SHA. Only that
|
|
# combination, on a pull_request_target event, lets a fork run remotely; a
|
|
# fork's pull_request run (pr.yml, which the fork may edit) stays in mode
|
|
# cache (or local, or fork-ro/fork-rw as the mint decides) whatever it
|
|
# passes, and has no secrets anyway. See engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
|
|
# "Trusted-author fork PRs". (rbe-fork's tier rw replaces this workflow
|
|
# once allowlisted forks have run it.)
|
|
# workflow_dispatch with rbe=cache simulates a fork run; rbe=off also drops
|
|
# the cache. The decision is also a workflow_call output (rbe-mode,
|
|
# rbe-enabled), so pr.yml's gate can tell an expected skip from a missing
|
|
# lane without re-deriving it. Callers pass secrets explicitly, never
|
|
# `secrets: inherit`: pr.yml and nightly.yml pass all five declared secrets
|
|
# (the four RBE ones plus rbe-prewarm's bazel-allocator app private key);
|
|
# bazel-farm.yml passes only the four RBE secrets (its pull_request_target
|
|
# run never gets the app credential - see "rbe-west Pre-warm" below and B1
|
|
# in this task's security review).
|
|
#
|
|
# Accepted risk (same model as gascity's bazel-test.yml): on push, merge_group
|
|
# and same-repo pull_request runs, code from the PR head runs with the RBE
|
|
# client certificate. setup-bazel is itself PR code, the key file is readable
|
|
# by every later step and by locally executed Bazel actions, and a .bazelrc
|
|
# edit can point the BEP or the executor anywhere. Anyone who can push a
|
|
# branch here can therefore use or copy the certificate and write to the
|
|
# farm's cache. Fork PRs never get it on their own runs, except those
|
|
# bazel-farm.yml authorizes (allowlisted author and sender, accepted as
|
|
# same-repo-level trust; that workflow's header lists what it adds), and
|
|
# except once queued: a merge_group run loads every workflow file and
|
|
# script from the queue entry's commit, so a fork (or Dependabot) PR that a
|
|
# maintainer queues runs its own edits to .github/, .bazelrc, tools/bazel,
|
|
# MODULE.bazel, BUILD files and CI scripts with all CI secrets, the
|
|
# pre-warm app key included, even if the entry is later ejected. That is
|
|
# the trust of merging it (push to main would run the same code), so the
|
|
# maintainer rule is: review a fork PR's changes to those paths before
|
|
# queueing it, as before merging (engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md,
|
|
# "Merge Queue"). Otherwise: pull_request from a
|
|
# fork receives no secrets, bazel-farm.yml is the only pull_request_target
|
|
# caller (policy-tested), and forks cannot write the runner cache. Nor do
|
|
# Dependabot PRs, which run as same-repo but get no Actions secrets. Fork
|
|
# and Dependabot runs in modes fork-ro/fork-rw hold a certificate of their
|
|
# own instead, worth what their run gets anyway (remote execution under the
|
|
# farm's quotas, AC_OSS reads, hash-verified CAS uploads; no action-cache
|
|
# writes) for about two hours: anyone who copies it from the run gets no
|
|
# more than the PR's author already has. In place: the four RBE secrets are
|
|
# referenced by the setup-bazel step alone (plus the rbe job's emptiness test
|
|
# of RBE_WEST_EXECUTOR, and rbe-prewarm's own has-app/mint steps for the
|
|
# app private key, below), never by a workflow- or job-level env (policy-tested),
|
|
# decoded key material is masked in logs, and CI uploads only remotely
|
|
# executed results (--noremote_upload_local_results in .bazelrc's
|
|
# remote-exec config and in the generated rc), and the certificate
|
|
# is issued to beads CI alone, so it can be revoked without affecting other
|
|
# clients. Maintainers decided (2026-09-28) that same-repo PRs keep remote
|
|
# execution under this model. Still open: a beads-only RBE_INSTANCE. The farm
|
|
# refuses client action-cache writes (only its workers write; rbe-west D2).
|
|
#
|
|
# Accepted risk, rbe-prewarm's credential (see "rbe-west Pre-warm",
|
|
# engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md, and B1 in this task's security
|
|
# review): the bazel-allocator private key is shared with gascity's own
|
|
# rbe-west scaler (a deliberate, user-approved reuse, not a beads-only
|
|
# credential), so same-repo pull_request, merge_group, push and nightly.yml
|
|
# runs - the only contexts that ever receive RBE_POOL_APP_PRIVATE_KEY - each
|
|
# reach that shared key for the duration of the mint step alone (nothing
|
|
# executes from the checked-out repository tree while it, or the token it
|
|
# produces, is in scope: see the rbe-prewarm job below). That holds only for
|
|
# this file as committed: a merge_group run uses the queued commit's copy,
|
|
# so a queued fork PR that edits this job (or pr.yml) reaches the key with
|
|
# whatever it wrote. Queueing a fork PR is therefore merging-level trust;
|
|
# review its .github/ changes first (see the header above). Unlike the RBE
|
|
# client certificate above, this key cannot be rotated or revoked for beads
|
|
# alone without also breaking gascity's scaler; the kill switch
|
|
# (RBE_POOL_APP_PRIVATE_KEY unset, or RBE_PREWARM_WORKERS=0) disables
|
|
# pre-warming without touching the key itself, so it is the first response to
|
|
# a suspected leak, not a rotation.
|
|
# In pr.yml's run the bd artifact is bazel-ci-build-artifacts, so it cannot
|
|
# collide with the build-artifacts job's ci-build-artifacts. F3: the package
|
|
# gates no longer consume this artifact (they build their own bd); it is
|
|
# kept for the F3.5.3 SHA256SUMS comparison and for debugging.
|
|
#
|
|
# Runner cache: one entry per (.bazelversion, MODULE.bazel.lock) pair, about
|
|
# 2 GB (Bazel's content-addressable --repository_cache, re-hashed on every
|
|
# hit; never the Bazel binary or the repo contents cache), written
|
|
# only by a push to main that missed it. No --disk_cache: remote runs use the
|
|
# remote cache, and local runs would need about 7 GB more.
|
|
#
|
|
# Workflow-file changes suppress pull_request runs under the org approval
|
|
# policy (merge_group runs are not suppressed: they run the queued copy);
|
|
# validate edits to this file with workflow_dispatch on the branch.
|
|
name: Bazel
|
|
|
|
on:
|
|
push:
|
|
branches: [ main ]
|
|
workflow_dispatch:
|
|
inputs:
|
|
rbe:
|
|
description: "Remote execution: on (use secrets), cache (fork simulation: anonymous read-only cache) or off (local only)"
|
|
required: true
|
|
type: choice
|
|
default: "on"
|
|
options: [ "on", "cache", "off" ]
|
|
details:
|
|
description: "List every equivalence divergence"
|
|
required: false
|
|
type: boolean
|
|
default: false
|
|
dolt-lane:
|
|
description: "Dolt-backed lane: doltserver (hermetic, remote) or docker (testcontainers A/B control)"
|
|
required: false
|
|
type: choice
|
|
default: "doltserver"
|
|
options: [ "doltserver", "docker" ]
|
|
fresh-test-results:
|
|
description: "Re-execute every test (--config=fresh: no cached test results), as nightly does"
|
|
required: false
|
|
type: boolean
|
|
default: false
|
|
workflow_call:
|
|
inputs:
|
|
rbe:
|
|
type: string
|
|
required: false
|
|
default: "on"
|
|
details:
|
|
type: boolean
|
|
required: false
|
|
default: true
|
|
go-test-json-artifact:
|
|
description: >
|
|
Artifact (from the calling run) holding go-test.json, PR Core's
|
|
`go-test -json` on this commit; enables the equivalence skip-parity
|
|
check. nightly.yml sets it.
|
|
type: string
|
|
required: false
|
|
default: ""
|
|
# A string, not a boolean: on push and dispatch inputs.integration is
|
|
# null, and null == false in GitHub's loose comparison, so a boolean
|
|
# test would silently stop the lane on main.
|
|
integration:
|
|
description: >
|
|
"off" skips bazel-integration; anything else (pr.yml and
|
|
bazel-farm.yml pass "on") runs it where remote execution is on.
|
|
type: string
|
|
required: false
|
|
default: "on"
|
|
# true (pr.yml only, policy-tested to equal its committed
|
|
# BAZEL_CMD_DOLT_REQUIRED flag): pr.yml's ci-gate requires
|
|
# bazel-cmd-dolt, which runs the whole integration-tagged cmd/bd suite
|
|
# (//cmd/bd:bd_dolt_server_test wraps the integration build's
|
|
# bd_test), so bazel-integration leaves //cmd/bd:bd_test to it and
|
|
# each of those tests runs once. A boolean: on push and dispatch it is
|
|
# null, which is not == true, so bazel-farm.yml, nightly.yml, push and
|
|
# dispatch (bazel-cmd-dolt advisory there) keep bd_test in
|
|
# bazel-integration.
|
|
cmd-dolt-required:
|
|
description: >
|
|
true (pr.yml only) drops //cmd/bd:bd_test from bazel-integration:
|
|
the required bazel-cmd-dolt lane runs those tests.
|
|
type: boolean
|
|
required: false
|
|
default: false
|
|
# true (nightly.yml only): every `bazel test` re-executes every test
|
|
# (--config=fresh), the run that samples flaky and -race failures.
|
|
# Everyone else reuses cached results for unchanged tests. A boolean:
|
|
# null on push, so push to main reuses too.
|
|
fresh-test-results:
|
|
description: >
|
|
true (nightly.yml) re-executes every test: no cached test results.
|
|
type: boolean
|
|
required: false
|
|
default: false
|
|
build-artifact-name:
|
|
description: >
|
|
Name of the uploaded bd artifact (ci-build-artifacts layout). pr.yml
|
|
sets bazel-ci-build-artifacts: its build-artifacts job uploads
|
|
ci-build-artifacts in the same run.
|
|
type: string
|
|
required: false
|
|
default: "ci-build-artifacts"
|
|
# Set only by bazel-farm.yml (policy-tested): the fork PR's head SHA
|
|
# from its pull_request_target event, pinned, and "authorized" once
|
|
# that workflow's authorize job has checked the base branch's
|
|
# allowlist. See the header.
|
|
checkout-sha:
|
|
description: >
|
|
Commit every lane checks out (bazel-farm.yml: the trusted fork PR's
|
|
head SHA); empty checks out the event's default ref.
|
|
type: string
|
|
required: false
|
|
default: ""
|
|
fork-farm:
|
|
description: >
|
|
"authorized" (bazel-farm.yml only) lets a pull_request_target fork
|
|
run execute remotely; anything else keeps forks local.
|
|
type: string
|
|
required: false
|
|
default: "off"
|
|
# "on" (pr.yml only) runs package-mcp and package-npm in this call,
|
|
# which waits on nothing but the rbe job (F3: the legacy pr.yml jobs
|
|
# those gates used to need: detect-package-gates, build-artifacts,
|
|
# bazel-coverage, and the whole bazel call). bazel-farm.yml, nightly.yml
|
|
# and direct runs (push, dispatch) keep the default "off": main.yml
|
|
# keeps its own package jobs, and a fork-farm run has no bd artifact
|
|
# consumer waiting on these.
|
|
package-gates:
|
|
description: >
|
|
"on" (pr.yml only) runs the MCP and npm package gates in this call.
|
|
type: string
|
|
required: false
|
|
default: "off"
|
|
# Only what the rbe job and setup-bazel need; callers pass them
|
|
# explicitly. None is required: fork and Dependabot runs have none.
|
|
secrets:
|
|
RBE_WEST_EXECUTOR:
|
|
required: false
|
|
RBE_TLS_CERT:
|
|
required: false
|
|
RBE_TLS_KEY:
|
|
required: false
|
|
RBE_TLS_CA:
|
|
required: false
|
|
# rbe-prewarm only (never read by setup-bazel or the rbe job): the
|
|
# "bazel-allocator" GitHub App's private key (its public client ID is
|
|
# a literal in the mint step, not a secret). The app is installed on
|
|
# gastownhall/gascity alone with Actions (read/write) and Metadata
|
|
# (read) permissions, nothing else; rbe-prewarm's mint step reads the
|
|
# private key directly (it IS a long-lived credential, held in the
|
|
# runner's memory for that one step) to produce a short-lived
|
|
# installation token, which is all any later step ever sees. Unset
|
|
# disables pre-warming. Declared here so fork-rw could in principle
|
|
# request it, but no caller ever does: pr.yml and nightly.yml pass it
|
|
# straight through (both are same-repo/trusted triggers only);
|
|
# bazel-farm.yml's pull_request_target call does not, so no pre-warm
|
|
# credential reaches a fork-authored run before a maintainer queues
|
|
# it (B1, security review of bdef342d5 - accepted risk is same-repo
|
|
# runs reaching the shared gascity scaler key, not forks). Exception:
|
|
# a merge_group run executes the queued commit's workflows, so a
|
|
# queued fork PR's edits run with this key (merging-level trust; see
|
|
# the header). See
|
|
# engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md, "rbe-west Pre-warm".
|
|
RBE_POOL_APP_PRIVATE_KEY:
|
|
required: false
|
|
# The rbe job's decision, and each lane's job.status (success, failure or
|
|
# cancelled; empty when the lane was skipped or never reported). pr.yml's
|
|
# ci-gate reads them through .github/scripts/bazel-gate.sh.
|
|
outputs:
|
|
rbe-enabled:
|
|
description: "'true' when the lanes executed remotely on rbe-west, else 'false'"
|
|
value: ${{ jobs.rbe.outputs.enabled }}
|
|
rbe-mode:
|
|
description: "remote, fork-ro or fork-rw (every lane runs remotely), cache (no remote-only jobs but integration), local (no remote-only jobs), or skip (no jobs): the rbe job's decision"
|
|
value: ${{ jobs.rbe.outputs.mode }}
|
|
bazel-test:
|
|
description: "bazel-test's job.status"
|
|
value: ${{ jobs.bazel-test.outputs.result }}
|
|
bazel-pure:
|
|
description: "bazel-pure's job.status"
|
|
value: ${{ jobs.bazel-pure.outputs.result }}
|
|
bazel-embedded:
|
|
description: "bazel-embedded's job.status"
|
|
value: ${{ jobs.bazel-embedded.outputs.result }}
|
|
bazel-integration:
|
|
description: "bazel-integration's job.status"
|
|
value: ${{ jobs.bazel-integration.outputs.result }}
|
|
bazel-doltserver:
|
|
description: "bazel-doltserver's job.status"
|
|
value: ${{ jobs.bazel-doltserver.outputs.result }}
|
|
bazel-proxied:
|
|
description: "bazel-proxied's job.status"
|
|
value: ${{ jobs.bazel-proxied.outputs.result }}
|
|
bazel-server-storage:
|
|
description: "bazel-server-storage's job.status"
|
|
value: ${{ jobs.bazel-server-storage.outputs.result }}
|
|
bazel-cmd-dolt:
|
|
description: "bazel-cmd-dolt's job.status (advisory unless pr.yml's BAZEL_CMD_DOLT_REQUIRED is \"true\")"
|
|
value: ${{ jobs.bazel-cmd-dolt.outputs.result }}
|
|
package-mcp:
|
|
description: "package-mcp's job.status (empty unless package-gates is 'on')"
|
|
value: ${{ jobs.package-mcp.outputs.result }}
|
|
package-npm:
|
|
description: "package-npm's job.status (empty unless package-gates is 'on')"
|
|
value: ${{ jobs.package-npm.outputs.result }}
|
|
rbe-prewarm:
|
|
description: "rbe-prewarm's job.status (advisory: continue-on-error, never gates or blocks a lane)"
|
|
value: ${{ jobs.rbe-prewarm.outputs.result }}
|
|
|
|
# Main runs always finish (they may save the runner cache); PR runs (called
|
|
# from pr.yml or bazel-farm.yml, whose own groups cancel the calling run too)
|
|
# supersede. The group differs from each caller's, as a called workflow's
|
|
# must.
|
|
concurrency:
|
|
group: bazel-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_target' }}
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
env:
|
|
BD_DISABLE_METRICS: "1"
|
|
BD_DISABLE_EVENT_FLUSH: "1"
|
|
# Appended last to every `bazel test` (after the lane's config and
|
|
# sole-run): --config=fresh (.bazelrc, --nocache_test_results) only when
|
|
# the caller asks for fresh results (nightly.yml, or a dispatch); empty
|
|
# otherwise, so results are cached and reused. See the header.
|
|
BAZEL_FRESH: ${{ inputs.fresh-test-results && '--config=fresh' || '' }}
|
|
# Phase B report-only budget for T1/T2 (tools/bazel/critpath.py).
|
|
BAZEL_BUDGET_SECONDS: "420"
|
|
|
|
jobs:
|
|
# The one place the execution mode is decided (see the header), so every
|
|
# job sees the same answer for the whole run. Its step reads one secret, and
|
|
# only to test it for emptiness: the step's env holds a boolean, never the
|
|
# value, and the step runs no repository code (no checkout). Fork and
|
|
# Dependabot pull_request runs get no secrets: rbe-fork-mint decides
|
|
# theirs (fork-ro, fork-rw, or cache while it is closed).
|
|
rbe:
|
|
name: execution mode
|
|
# Same-repo PRs (and merge_group/push/dispatch/schedule runs, which are
|
|
# never forks) get the org's Blacksmith pool; a pull_request_target farm
|
|
# run and any fork or Dependabot PR stay on GitHub-hosted runners. This
|
|
# job reads no repository code and holds no secret value (see below), so
|
|
# Blacksmith sees nothing sensitive either way.
|
|
runs-on: ${{ (github.event_name == 'push' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
timeout-minutes: 5
|
|
outputs:
|
|
enabled: ${{ steps.decide.outputs.enabled }}
|
|
mode: ${{ steps.decide.outputs.mode }}
|
|
tier: ${{ steps.decide.outputs.tier }}
|
|
steps:
|
|
# The expressions' == is case-insensitive (RBE_WEST_WORKERS=True counts).
|
|
# A push, merge_group or dispatch has no pull_request, so FORK is false.
|
|
# Fork and Dependabot pull_request runs see no secrets and no vars: for
|
|
# them rbe-west decides (rbe-fork-mint's status: open, and the tier from
|
|
# the PR's author and this run's triggering actor). Its answer is the
|
|
# whole run's mode; every lane then gets its own certificate
|
|
# (setup-bazel, BAZEL_FORK_REMOTE) and fails if the tier changed. A
|
|
# mint that is closed, refuses or cannot be reached leaves mode cache,
|
|
# as before rbe-fork.
|
|
- name: Decide remote execution
|
|
id: decide
|
|
env:
|
|
RBE_VAR_ON: ${{ vars.RBE_WEST_WORKERS == 'true' }}
|
|
RBE_INPUT_OFF: ${{ inputs.rbe == 'off' }}
|
|
RBE_INPUT_CACHE: ${{ inputs.rbe == 'cache' }}
|
|
PULL_REQUEST: ${{ github.event_name == 'pull_request' }}
|
|
FORK: ${{ github.event.pull_request.head.repo.fork == true }}
|
|
DEPENDABOT: ${{ github.actor == 'dependabot[bot]' }}
|
|
FORK_FARM: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' && inputs.checkout-sha != '' }}
|
|
HAS_EXECUTOR: ${{ secrets.RBE_WEST_EXECUTOR != '' }}
|
|
PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
run: |
|
|
for v in RBE_VAR_ON RBE_INPUT_OFF RBE_INPUT_CACHE PULL_REQUEST FORK DEPENDABOT FORK_FARM HAS_EXECUTOR; do
|
|
case "${!v}" in
|
|
true|false) ;;
|
|
*) echo "::error::$v is '${!v}', want true or false"; exit 1 ;;
|
|
esac
|
|
done
|
|
# A fork or Dependabot pull_request run executes remotely only with
|
|
# rbe-fork's certificate (mint open for this run), else locally with
|
|
# the read-only cache. Any other fork run (pull_request_target)
|
|
# stays local unless bazel-farm.yml authorized it; an authorized
|
|
# fork run is remote or nothing (pr.yml's call already runs it).
|
|
tier=
|
|
if [ "$RBE_INPUT_OFF" = true ]; then
|
|
mode=local
|
|
elif [ "$RBE_INPUT_CACHE" = true ]; then
|
|
mode=cache
|
|
elif [ "$PULL_REQUEST" = true ] && { [ "$FORK" = true ] || [ "$DEPENDABOT" = true ]; }; then
|
|
[[ "$PR_NUMBER" =~ ^[1-9][0-9]{0,8}$ ]] || { echo "::error::PR number '$PR_NUMBER'"; exit 1; }
|
|
url="https://rbe-mint.ops.gascity.com:8444/v1/status?repo=${GITHUB_REPOSITORY#*/}&run=$GITHUB_RUN_ID&attempt=$GITHUB_RUN_ATTEMPT&pr=$PR_NUMBER"
|
|
st=$(curl -sS --connect-timeout 5 --max-time 30 --retry 3 --retry-all-errors "$url" || echo '{}')
|
|
tier=$(jq -r 'if .open == true then .tier else "" end' <<<"$st" 2>/dev/null || true)
|
|
case "$tier" in
|
|
ro|rw) mode="fork-$tier" ;;
|
|
*)
|
|
tier=
|
|
mode=cache
|
|
echo "::warning title=rbe-fork closed for this run::$(jq -r '.reason // .error // "rbe-fork-mint unreachable"' <<<"$st" 2>/dev/null || echo unreachable)"
|
|
;;
|
|
esac
|
|
elif [ "$FORK" = true ] && [ "$FORK_FARM" != true ]; then
|
|
mode=cache
|
|
elif [ "$RBE_VAR_ON" != true ]; then
|
|
mode=skip
|
|
elif [ "$HAS_EXECUTOR" = true ]; then
|
|
mode=remote
|
|
elif [ "$FORK_FARM" = true ]; then
|
|
mode=skip
|
|
else
|
|
mode=cache
|
|
fi
|
|
enabled=false
|
|
case "$mode" in remote|fork-ro|fork-rw) enabled=true ;; esac
|
|
{
|
|
echo "mode=$mode"
|
|
echo "enabled=$enabled"
|
|
echo "tier=$tier"
|
|
} >> "$GITHUB_OUTPUT"
|
|
echo "Bazel execution mode: **$mode** (RBE_WEST_WORKERS on: $RBE_VAR_ON, rbe=off: $RBE_INPUT_OFF, rbe=cache: $RBE_INPUT_CACHE, fork: $FORK, dependabot: $DEPENDABOT, fork farm: $FORK_FARM, executor secret: $HAS_EXECUTOR)" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
|
|
# Best-effort pre-warm of gastownhall/gascity's rbe-west OSS worker pool
|
|
# (one Blacksmith-hosted NativeLink worker, rbe-worker-pool.yml) so it is
|
|
# already booting instead of the lanes below waiting ~120s at "N actions, 0
|
|
# running" while the pool scales from zero. A job of its own (not a step in
|
|
# `rbe`), so it never delays the mode decision every other job needs, and
|
|
# it is never a `needs` of any lane and never in pr.yml's ci-gate required
|
|
# list (advisory; .github/scripts/bazel-gate.sh does not read it). See
|
|
# engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md, "rbe-west Pre-warm".
|
|
#
|
|
# Only when the run's lanes will actually execute on instance "oss": mode
|
|
# remote (the CI secrets) or fork-rw (rbe-fork-mint's trusted-author tier,
|
|
# cached by the same OSS workers per the header above). Mode fork-ro targets
|
|
# instance "oss-fork", a separate, uncached pool rbe-worker-pool.yml does
|
|
# not serve, so it is excluded (TestBazelCacheModeReachesTheRC's one named
|
|
# exception to "every lane runs remotely" in a fork mode). Like every other
|
|
# lane, the `if:` reads only needs.rbe.outputs - TestBazelRBEJobDecidesOnce
|
|
# forbids re-deriving fork/event facts outside the rbe job itself - and
|
|
# runs-on is the same mode-ternary every other lane uses, so a genuine fork
|
|
# or Dependabot PR that reaches mode fork-rw still never touches Blacksmith:
|
|
# the runner is only a client there, like every other fork-mode lane (see
|
|
# bazel-test's comment above). That PR also carries no repository secrets
|
|
# regardless of runs-on (GitHub strips them from a fork's pull_request
|
|
# event), so the credential-check step below finds nothing to mint even if
|
|
# it runs. The app secret (secrets.RBE_POOL_APP_PRIVATE_KEY) is never
|
|
# read in this `if:` or anywhere else in this file - only inside this job's own step env/with, same as the
|
|
# rbe job's HAS_EXECUTOR pattern - so a missing credential (the app not
|
|
# provisioned yet, or the kill switch) fails soft (no mint attempt, then
|
|
# the script's own ::warning::, exit 0) rather than changing whether the
|
|
# job runs.
|
|
rbe-prewarm:
|
|
name: pre-warm rbe-west pool
|
|
needs: rbe
|
|
# Only mode remote (B1, security review of bdef342d5: was remote or
|
|
# fork-rw). fork-rw is reached only by a fork or Dependabot
|
|
# pull_request run, which GitHub never hands a workflow_call secret
|
|
# regardless of this job's `if:`, so that half of the old condition
|
|
# always found no-app and no-op'd; bazel-farm.yml (the one caller with a
|
|
# distinct pull_request_target trust tier that could otherwise have
|
|
# reached fork-rw with secrets attached) no longer forwards the app
|
|
# secrets either, below. See "rbe-west Pre-warm",
|
|
# engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md.
|
|
if: ${{ needs.rbe.outputs.mode == 'remote' }}
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
timeout-minutes: 5
|
|
# Job-level, not step-level: the only continue-on-error in this workflow
|
|
# (TestBazelWorkflowSecretsAndFailureSurface and
|
|
# TestBazelWorkflowJobsAndExecutionMode both carve out this one path by
|
|
# name). Every other job here is a gated lane whose real failure must
|
|
# reach pr.yml's ci-gate; this one is advisory and must never fail the
|
|
# call bazel.yml's own run represents (a job's failure fails the whole
|
|
# reusable-workflow run regardless of `needs`, which would otherwise turn
|
|
# pr.yml's ci-gate red for a pre-warm hiccup nothing requires).
|
|
continue-on-error: true
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# No checkout in this job (B1, security review of bdef342d5; policy-
|
|
# tested that it has none): a step that checked out PR-head content
|
|
# and then ran a script from that tree while this job's credential was
|
|
# live would let that content - or any same-repo branch push, since
|
|
# editing a .github/scripts/*.sh file does not trip the org's
|
|
# workflow-file approval gate the way editing this YAML does - read
|
|
# the shared gascity scaler key. The dispatch logic therefore lives
|
|
# entirely in this trusted workflow definition's own `run:` below,
|
|
# never a checked-out file.
|
|
|
|
# Same env-boolean pattern as the rbe job's HAS_EXECUTOR: the secret is
|
|
# tested for emptiness, never referenced in an `if:` directly, and the
|
|
# next step's `if:` reads this step's output (a job-level env can't be
|
|
# read from another step's `if:`, only its own `env:`).
|
|
- name: Check for the pre-warm app credential
|
|
id: has-app
|
|
env:
|
|
HAS_POOL_APP: ${{ secrets.RBE_POOL_APP_PRIVATE_KEY != '' }}
|
|
run: echo "has-app=$HAS_POOL_APP" >> "$GITHUB_OUTPUT"
|
|
|
|
# Mints a short-lived installation token for the "bazel-allocator"
|
|
# GitHub App (installed on gastownhall/gascity only, Actions
|
|
# read/write + Metadata read): skipped entirely while the app's
|
|
# private key is not provisioned, leaving steps.mint.outputs.token empty, which
|
|
# the dispatch step below treats the same as any other missing-token
|
|
# case (::warning::, exit 0). permission-actions narrows the minted
|
|
# token to Actions even if the app is ever granted more. This is the
|
|
# only step that ever reads the long-lived private key directly
|
|
# (below); every step after it sees at most the short-lived token this
|
|
# one produces. client-id is the app's public Client ID (GET
|
|
# /apps/bazel-allocator; not a credential), committed as a literal so
|
|
# it is reviewed and policy-tested with the rest of this step rather
|
|
# than living in an unversioned repository variable; it replaces the
|
|
# action's deprecated app-id input.
|
|
- name: Mint gastownhall/gascity installation token
|
|
id: mint
|
|
if: ${{ steps.has-app.outputs.has-app == 'true' }}
|
|
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3
|
|
with:
|
|
client-id: Iv23ligrqVEhlamZnoPU
|
|
private-key: ${{ secrets.RBE_POOL_APP_PRIVATE_KEY }}
|
|
owner: gastownhall
|
|
repositories: gascity
|
|
permission-actions: write
|
|
|
|
# Inlined rather than checked out and run from a script file (B1
|
|
# above): every byte this step executes comes from this workflow
|
|
# definition, which pull_request_target always loads from the base
|
|
# branch and which editing suppresses pull_request runs under the
|
|
# org's approval policy (see the header). Not on merge_group: a queue
|
|
# entry runs its own commit's copy of this file with no approval
|
|
# gate, so a queued fork PR's edit here runs with the key (review
|
|
# .github/ changes before queueing). steps.mint.outputs.token
|
|
# appears nowhere else in this file (policy-tested): this is the only
|
|
# step with the credential in its environment, and no step after it
|
|
# touches a repository path or runs a local composite action.
|
|
- name: Pre-warm rbe-west OSS worker pool (gastownhall/gascity)
|
|
env:
|
|
GH_TOKEN: ${{ steps.mint.outputs.token }}
|
|
RBE_PREWARM_WORKERS: ${{ vars.RBE_PREWARM_WORKERS }}
|
|
run: |
|
|
set -u
|
|
|
|
POOL_REPO="gastownhall/gascity"
|
|
POOL_WORKFLOW="rbe-worker-pool.yml"
|
|
POOL_REF="main"
|
|
|
|
n="${RBE_PREWARM_WORKERS:-1}"
|
|
case "$n" in
|
|
'') n=1 ;;
|
|
*[!0-9]*) n=1 ;;
|
|
# Two or more digits is over the cap unless they are all zeros
|
|
# (still the kill switch); deciding it here keeps a string too
|
|
# long for the shell's integer compare from slipping past the
|
|
# -gt below unclamped.
|
|
??*)
|
|
case "$n" in
|
|
*[!0]*) n=4 ;;
|
|
*) n=0 ;;
|
|
esac
|
|
;;
|
|
esac
|
|
if [ "$n" -gt 4 ]; then
|
|
n=4
|
|
fi
|
|
|
|
if [ "$n" -le 0 ]; then
|
|
echo "RBE_PREWARM_WORKERS=0: rbe-west pre-warm disabled (kill switch)"
|
|
exit 0
|
|
fi
|
|
|
|
if [ -z "${GH_TOKEN:-}" ]; then
|
|
echo "::warning title=rbe-west pre-warm::no dispatch token (bazel-allocator app not provisioned, or the mint step failed); skipping"
|
|
exit 0
|
|
fi
|
|
|
|
active="$(gh run list -R "$POOL_REPO" --workflow "$POOL_WORKFLOW" --limit 50 \
|
|
--json status --jq '[.[] | select(.status != "completed")] | length' 2>/dev/null)" || true
|
|
case "$active" in
|
|
'' | *[!0-9]*)
|
|
echo "::warning title=rbe-west pre-warm::could not read $POOL_REPO/$POOL_WORKFLOW run status; skipping dispatch"
|
|
exit 0
|
|
;;
|
|
esac
|
|
|
|
missing=$((n - active))
|
|
if [ "$missing" -le 0 ]; then
|
|
echo "${active} $POOL_REPO/$POOL_WORKFLOW worker(s) already active (want ${n}); nothing to dispatch"
|
|
exit 0
|
|
fi
|
|
|
|
dispatched=0
|
|
i=1
|
|
while [ "$i" -le "$missing" ]; do
|
|
if gh workflow run "$POOL_WORKFLOW" -R "$POOL_REPO" --ref "$POOL_REF"; then
|
|
dispatched=$((dispatched + 1))
|
|
else
|
|
echo "::warning title=rbe-west pre-warm::gh workflow run $POOL_WORKFLOW -R $POOL_REPO failed (attempt $i/$missing)"
|
|
fi
|
|
i=$((i + 1))
|
|
done
|
|
|
|
echo "dispatched ${dispatched}/${missing} $POOL_REPO/$POOL_WORKFLOW worker(s) (active before: ${active}, want: ${n})"
|
|
exit 0
|
|
|
|
# Every lane reports its job.status from an always() last step
|
|
# (TestBazelLaneIsGatedAlongsideLegacy). The dispatch step above always
|
|
# exits 0, so this normally reads success regardless of what it found;
|
|
# the job-level continue-on-error above is the backstop for a step
|
|
# that fails outside it (the mint action).
|
|
- name: Record result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
bazel-test:
|
|
name: test
|
|
# Remote-execution runs use the org's Blacksmith pool (the farm's edge
|
|
# admits any runner holding the client certificate; there is no IP
|
|
# allowlist) and do not count against GitHub's concurrent-job cap. Local
|
|
# and cache runs (forks, rbe=off/cache, no secrets) build everything
|
|
# they miss on the runner, so they get the larger GitHub-hosted one.
|
|
# Fork modes (fork-ro/fork-rw) execute remotely but stay GitHub-hosted,
|
|
# like every fork job: the runner is only a client.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
# Remote execution goes to rbe-west (instance "oss", the Blacksmith-backed
|
|
# worker pool shared with gascity). Skipped only in mode skip (same-repo
|
|
# runs while RBE_WEST_WORKERS is unset).
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 100
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
env:
|
|
# D2 step 3: on covered PRs and merge groups (always mode remote) this
|
|
# lane is the only pre-merge run of pr.yml's legacy PR Core job, which
|
|
# never retried: --config=sole-run then fails on a remote cache eviction
|
|
# instead of re-running the whole invocation (whose retry could turn a
|
|
# failed test green). Fork modes do the same (with BAZEL_COVERS_FORKS
|
|
# "true" it is their only run too). Test results are cached and reused
|
|
# like every lane's; only nightly re-executes them all (BAZEL_FRESH).
|
|
BAZEL_SOLE_RUN: ${{ needs.rbe.outputs.enabled == 'true' && '--config=sole-run' || '' }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
# Go is for tools/bazel/equivalence.py's `go list` and for make; Bazel
|
|
# uses its own hermetic SDK.
|
|
- name: Set up Go
|
|
id: setup-go
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
# Read-only: main.yml's build-artifacts job owns this cache.
|
|
- name: Restore Go module cache
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/go/pkg/mod
|
|
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
|
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
|
|
|
# Go does not re-hash extracted module directories on use; a restored
|
|
# cache written by an untrusted run is checked against go.sum first.
|
|
- name: Verify restored Go modules
|
|
run: go mod verify
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# T1: remote + restored repository cache; T2: remote, cold runner; local:
|
|
# no remote execution (reported, never budgeted).
|
|
- name: Measurement tier
|
|
id: tier
|
|
env:
|
|
REMOTE: ${{ steps.bazel.outputs.remote }}
|
|
RESTORED: ${{ steps.bazel.outputs.cache-restored }}
|
|
run: |
|
|
if [ "$REMOTE" = "true" ] && [ "$RESTORED" = "true" ]; then tier=T1
|
|
elif [ "$REMOTE" = "true" ]; then tier=T2
|
|
else tier=local; fi
|
|
echo "tier=$tier" >> "$GITHUB_OUTPUT"
|
|
{
|
|
echo "### Bazel / test"
|
|
echo
|
|
echo "Tier **$tier**: remote execution $REMOTE, runner cache restored $RESTORED."
|
|
echo
|
|
} >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
# --profile and --build_event_json_file are command options, so they go
|
|
# after `test` (a startup position fails).
|
|
- name: bazel test //... --config=ci
|
|
id: test
|
|
timeout-minutes: 90
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=ci ${BAZEL_SOLE_RUN:+"$BAZEL_SOLE_RUN"} ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--profile="$RUNNER_TEMP/bazel-profile.json" \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
echo >> "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# pr.yml's build-artifacts job, built by Bazel: the same artifact name
|
|
# and layout (bd-linux-gms-pure, SHA256SUMS, build-manifest.txt). Where
|
|
# build-artifacts stands down (D2 step 3), its other consumers (PR
|
|
# Core, domain+uow) stand down there too. F3: the package gates build
|
|
# their own bd and no longer consume this artifact; it is kept for the
|
|
# F3.5.3 SHA256SUMS comparison and for debugging. The script documents
|
|
# which binary and how it differs from `go build`'s.
|
|
- name: Package bd (ci-build-artifacts layout)
|
|
id: bd-artifact
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: ./scripts/ci/package-bazel-bd.sh "$RUNNER_TEMP/bd-artifacts"
|
|
|
|
- name: Upload build artifacts
|
|
if: ${{ always() && steps.bd-artifact.outcome == 'success' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: ${{ inputs.build-artifact-name || 'ci-build-artifacts' }}
|
|
path: ${{ runner.temp }}/bd-artifacts/
|
|
retention-days: 1
|
|
if-no-files-found: error
|
|
|
|
- name: Critical-path report
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
env:
|
|
TIER: ${{ steps.tier.outputs.tier }}
|
|
run: |
|
|
budget=()
|
|
if [ "$TIER" != "local" ]; then budget=(--budget "$BAZEL_BUDGET_SECONDS"); fi
|
|
python3 tools/bazel/critpath.py "$RUNNER_TEMP/bazel-profile.json" \
|
|
--tier "$TIER" --log "$RUNNER_TEMP/bazel-test.log" "${budget[@]}"
|
|
|
|
# nightly.yml's PR Core `go test -json` on the same commit.
|
|
- name: Download go-test results
|
|
if: ${{ always() && inputs.go-test-json-artifact != '' && steps.test.outcome != 'skipped' }}
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: ${{ inputs.go-test-json-artifact }}
|
|
path: ${{ runner.temp }}/go-test-json
|
|
|
|
# Fails on any Go test that `go test` runs and Bazel did not (tag filter,
|
|
# manual, missing target, empty shard) unless it is allowlisted. With
|
|
# go test results (nightly) it also fails on tests that passed under go
|
|
# test and were skipped under Bazel; a PR run cannot see those.
|
|
# It reads test.xml from bazel-testlogs, which --config=integration
|
|
# shares with --config=ci (same output directory): if a job ever runs
|
|
# both lanes, run each lane's equivalence right after its own
|
|
# `bazel test`, before the other lane overwrites the logs.
|
|
- name: Go test equivalence
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
timeout-minutes: 15
|
|
env:
|
|
DETAILS: ${{ inputs.details && '--details' || '' }}
|
|
GO_TEST_JSON: ${{ inputs.go-test-json-artifact != '' && format('{0}/go-test-json/go-test.json', runner.temp) || '' }}
|
|
run: |
|
|
args=()
|
|
if [ -n "$DETAILS" ]; then args+=("$DETAILS"); fi
|
|
if [ -n "$GO_TEST_JSON" ]; then args+=(--go-test-json "$GO_TEST_JSON"); fi
|
|
python3 tools/bazel/equivalence.py --bep "$RUNNER_TEMP/bazel-bep.json" "${args[@]}"
|
|
|
|
# Runs last: it rewrites BUILD files in the checkout. main moves BUILD
|
|
# inputs (imports, go.mod) without touching Bazel, so drift shows up here
|
|
# first. On drift the step still fails, and the BUILD.bazel /
|
|
# MODULE.bazel / MODULE.bazel.lock part of the fix is written as a patch
|
|
# for the bazel-sync-patch artifact, which bazel-autofix.yml pushes to
|
|
# same-repo PR branches (or turns into an apply recipe for forks).
|
|
- name: BUILD files in sync (gazelle, go_srcs, MODULE.bazel)
|
|
id: sync
|
|
if: ${{ always() && steps.bazel.outcome == 'success' }}
|
|
timeout-minutes: 15
|
|
env:
|
|
PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
run: |
|
|
start=$(date +%s)
|
|
# A stale go_srcs block is drift like any other: record it, then let
|
|
# bazel-sync fix it so the patch covers it too.
|
|
check=0
|
|
make bazel-sync-check || check=$?
|
|
make bazel-sync
|
|
drift="$(git status --porcelain --untracked-files=all)"
|
|
echo "BUILD sync check: $(( $(date +%s) - start ))s" >> "$GITHUB_STEP_SUMMARY"
|
|
if [ -n "$drift" ]; then
|
|
echo "::error title=Bazel BUILD files out of sync::run 'make bazel-sync' and commit the result"
|
|
echo "$drift"
|
|
git diff | head -200
|
|
echo "BUILD files **out of sync**; run \`make bazel-sync\` and commit the result." >> "$GITHUB_STEP_SUMMARY"
|
|
./scripts/ci/bazel-sync-patch.sh "$RUNNER_TEMP/bazel-sync-patch"
|
|
exit 1
|
|
fi
|
|
exit "$check"
|
|
|
|
# Data only: bazel-autofix.yml validates every path before applying it.
|
|
# Not from bazel-farm.yml's pull_request_target runs: autofix reads only
|
|
# "PR" runs, and a trusted-fork run publishes nothing meant for a
|
|
# privileged consumer.
|
|
- name: Upload BUILD sync patch
|
|
if: ${{ always() && steps.sync.outcome == 'failure' && github.event_name != 'pull_request_target' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-sync-patch
|
|
path: ${{ runner.temp }}/bazel-sync-patch/
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome == 'failure' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Only a push to main that missed the exact key writes the runner cache.
|
|
# Its content (--repository_cache only, never the rc
|
|
# or credentials) is fixed by .bazelversion and MODULE.bazel.lock, so one
|
|
# entry per lock is enough and later pushes do not churn the quota.
|
|
- name: Save Bazel runner cache
|
|
if: ${{ always() && github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.bazel.outcome == 'success' && steps.bazel.outputs.cache-hit != 'true' }}
|
|
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ${{ runner.temp }}/bazel-ci-cache
|
|
key: bazel-repo-v3-${{ runner.os }}-${{ hashFiles('.bazelversion', 'MODULE.bazel.lock') }}
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr.yml's check-cmd-bd-puregeo-tests job ("Check pure-Go and js/wasm
|
|
# boundaries (CGO_ENABLED=0)") with Bazel-built binaries: --config=pure
|
|
# builds bd and the job's three pure test binaries and runs its -short
|
|
# cmd/bd subset (PURE_CMD_BD_TESTS, kept equal to the job's -run selector by
|
|
# scripts/ci_workflow_test.go); --config=js-wasm builds the hooks test for
|
|
# js/wasm, which the Go SDK's go_js_wasm_exec runs under node with the job's
|
|
# exact pass-count guard.
|
|
bazel-pure:
|
|
name: pure-Go and js/wasm
|
|
# Same runner and skip rule as bazel-test.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 90
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
env:
|
|
PURE_CMD_BD_TESTS: '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir|NocgoFactoriesHonorSharedServer|NocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag|NocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag)'
|
|
# D2 step 3: on covered PRs and merge groups (always mode remote) this
|
|
# lane is the only pre-merge run of pr.yml's legacy pure-Go and js/wasm
|
|
# job, which never retried: --config=sole-run then fails on a remote
|
|
# cache eviction instead of re-running the whole invocation (whose retry
|
|
# could turn a failed test green). Fork modes do the same (with
|
|
# BAZEL_COVERS_FORKS "true" it is their only run too). Test results are
|
|
# cached and reused like every lane's; only nightly re-executes them all
|
|
# (BAZEL_FRESH).
|
|
BAZEL_SOLE_RUN: ${{ needs.rbe.outputs.enabled == 'true' && '--config=sole-run' || '' }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
# Only for go_js_wasm_exec and wasm_exec_node.js, which must match the
|
|
# Go version Bazel compiles with (go.mod's; policy-tested).
|
|
- name: Set up Go
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
- name: Set up Node.js
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v6
|
|
with:
|
|
node-version: '24'
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
- name: Run js/wasm hook boundary
|
|
run: |
|
|
set -euo pipefail
|
|
bazel build --config=js-wasm //internal/hooks:hooks_test
|
|
bin="$(bazel cquery --config=js-wasm --output=files //internal/hooks:hooks_test)"
|
|
cd internal/hooks
|
|
|
|
# Same selector and count guard as pr.yml's job.
|
|
test_status=0
|
|
# wasm_exec.js caps argv+env size; the Blacksmith runner's environment
|
|
# exceeds it, so run with only what the hook test needs.
|
|
wasm_exec="$(go env GOROOT)/lib/wasm/go_js_wasm_exec"
|
|
test_output="$(env -i PATH="$PATH" HOME="$HOME" TMPDIR="$RUNNER_TEMP" \
|
|
BD_DISABLE_METRICS=1 BD_DISABLE_EVENT_FLUSH=1 \
|
|
"$wasm_exec" "$GITHUB_WORKSPACE/$bin" \
|
|
-test.run '^TestRunHookReportsUnsupportedExecution$' \
|
|
-test.v 2>&1)" || test_status=$?
|
|
printf '%s\n' "$test_output"
|
|
(( test_status == 0 )) || exit "$test_status"
|
|
|
|
run_count=0
|
|
pass_count=0
|
|
nonpass_count=0
|
|
pass_pattern='^--- PASS: TestRunHookReportsUnsupportedExecution \([0-9.]+s\)$'
|
|
nonpass_pattern='^[[:space:]]*--- (FAIL|SKIP): '
|
|
while IFS= read -r line; do
|
|
if [[ "$line" == "=== RUN TestRunHookReportsUnsupportedExecution" ]]; then
|
|
((run_count += 1))
|
|
fi
|
|
if [[ "$line" =~ $pass_pattern ]]; then
|
|
((pass_count += 1))
|
|
fi
|
|
if [[ "$line" =~ $nonpass_pattern ]]; then
|
|
((nonpass_count += 1))
|
|
fi
|
|
done <<< "$test_output"
|
|
if (( run_count != 1 || pass_count != 1 || nonpass_count != 0 )); then
|
|
printf 'expected one exact js/wasm hook test run and pass with no failures or skips; got runs=%s passes=%s nonpasses=%s\n' \
|
|
"$run_count" "$pass_count" "$nonpass_count" >&2
|
|
exit 1
|
|
fi
|
|
|
|
- name: Build cmd/bd and pure-Go test binaries (--config=pure)
|
|
run: >-
|
|
bazel build --config=pure
|
|
//cmd/bd:bd
|
|
//cmd/bd:bd_test
|
|
//internal/storage/embeddeddolt:embeddeddolt_test
|
|
//internal/tracker:tracker_test
|
|
|
|
# `go build` with CGO_ENABLED=0 rejects a pure binary whose imports reach
|
|
# dolt's cgo-only store/nbs (gozstd). Bazel compiles gozstd's pure stubs
|
|
# instead, whose init panics (third_party/patches/gozstd_nocgo.patch), so
|
|
# the check happens here: every pure artifact is started once. bd_test
|
|
# starts in the next step. -test.run=^$ runs no test (TestMain still
|
|
# runs; BEADS_TEST_SKIP=dolt keeps tracker's from starting a container).
|
|
- name: Start every pure-Go artifact (gozstd contamination check)
|
|
run: |
|
|
set -euo pipefail
|
|
bazel run --config=pure //cmd/bd:bd -- version
|
|
bazel test --config=pure ${BAZEL_SOLE_RUN:+"$BAZEL_SOLE_RUN"} ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
//internal/storage/embeddeddolt:embeddeddolt_test \
|
|
//internal/tracker:tracker_test \
|
|
--test_sharding_strategy=disabled \
|
|
'--test_arg=-test.run=^$' \
|
|
--test_env=BEADS_TEST_SKIP=dolt
|
|
|
|
# One unsharded process, like `go test -run`; the test.xml check keeps a
|
|
# selector that matches nothing from passing.
|
|
- name: Run pure-Go cmd/bd test subset (--config=pure)
|
|
run: |
|
|
set -euo pipefail
|
|
bazel test --config=pure ${BAZEL_SOLE_RUN:+"$BAZEL_SOLE_RUN"} ${BAZEL_FRESH:+"$BAZEL_FRESH"} //cmd/bd:bd_test \
|
|
--test_sharding_strategy=disabled \
|
|
"--test_arg=-test.run=$PURE_CMD_BD_TESTS"
|
|
n="$(grep -c '<testcase ' bazel-testlogs/cmd/bd/bd_test/test.xml || true)"
|
|
echo "pure cmd/bd subset: $n test cases"
|
|
(( n > 0 ))
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr-risk.yml's embedded-Dolt tier ("Test (Embedded Dolt Cmd N/20)",
|
|
# "Test (Embedded Dolt Storage N/5)", "Test (Embedded Dolt Conformance -
|
|
# core/audit)") as --config=embedded: the embedded-tagged variants of
|
|
# //cmd/bd:bd_test and //internal/storage/embeddeddolt:embeddeddolt_test.
|
|
# The shard variants run the jobs' own shard scripts and manifests, so Bazel
|
|
# shard k runs exactly the tests of job k. A job of its own rather than a
|
|
# second invocation in bazel-test: the tier's long race shards stay off
|
|
# bazel-test's critical path and budget, and it compiles nothing bazel-test
|
|
# does not (same race configuration, shared through the remote cache).
|
|
bazel-embedded:
|
|
name: embedded Dolt tier
|
|
# Remote execution only: skipped unless the rbe job chose remote,
|
|
# fork-ro or fork-rw (enabled; not in modes cache, local or skip).
|
|
# Locally the tier is 67 race test
|
|
# processes (50 cmd/bd shards + 15 embeddeddolt storage shards + 2
|
|
# conformance partitions; ~24k test-seconds measured locally -- see
|
|
# scripts/ci/embedded_cmd_test_durations.json and
|
|
# scripts/ci/embedded_storage_test_durations.json -- up from an earlier,
|
|
# smaller test surface's ~7-10k estimate) on one runner, several hours
|
|
# per run on the pool whose job cap this track exists to relieve, and it
|
|
# proves nothing pr-risk.yml's embedded jobs (which do run for forks) do
|
|
# not.
|
|
#
|
|
# Shard counts: storage (15) is sized the way both tiers were originally
|
|
# meant to be -- the smallest count whose LPT-packed heaviest shard
|
|
# (scripts/ci/gen_embedded_storage_shard_manifest.py --weights=duration
|
|
# --repack) lands at or below the tier's own heaviest single top-level
|
|
# test (211-225s). cmd cannot reach that floor: Bazel hard-caps
|
|
# shard_count at 50 per sh_test-like rule ("Having more than 50 shards is
|
|
# indicative of poor test organization. Please reduce the number of
|
|
# shards." -- a native analysis-time error, not a style lint), and at 50
|
|
# shards the cmd suite's LPT-packed heaviest shard is ~409.6s -- above
|
|
# its own single heaviest test's 239.7s, because the suite's total
|
|
# duration (~20145 test-seconds across 242 tests) is too large to flatten
|
|
# to the single-test floor within 50 bins. Clearing more shards would
|
|
# need either further per-test speedups spread across the suite or
|
|
# splitting bd_embedded_test into multiple <=50-shard sh_test rules with
|
|
# shard-index-offset plumbing through tools/bazel/go_test_manifest_shard.sh
|
|
# and check_shard_coverage.py; neither is done here, so this lane's floor
|
|
# is cmd's ~409.6s, not the ~240s a 5-minute lane (including this job's
|
|
# ~60s fixed cost) would need -- tracked as a follow-up, not silently
|
|
# hidden in the shard count. On RBE this makes the tier's wall time ~=
|
|
# that floor + fixed cost (checkout, build, shard dispatch), not ~=
|
|
# sum(durations)/shard_count the way a CPU-core-bound local sweep is --
|
|
# each shard is its own remote action and runs concurrently with every
|
|
# other shard, farm capacity permitting.
|
|
# On same-repo pull_requests those jobs stand down (pr-risk.yml's
|
|
# bazel-coverage), so this lane is then the tier's only PR run
|
|
# and pr.yml's gate requires it to have run here and passed.
|
|
# The runs-on expression is the shared one: Blacksmith in mode remote,
|
|
# GitHub-hosted in the fork modes (a client only).
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 60
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Bazel retries a flaky = True test (up to three attempts) and reports a
|
|
# later pass as PASSED. The policy test greps BUILD files for it; this
|
|
# asks Bazel itself, so a macro or a variable cannot hide one. Cheap:
|
|
# loading only, with the repository cache the test step needs anyway.
|
|
- name: No test target is marked flaky
|
|
run: |
|
|
flaky="$(bazel query 'attr(flaky, 1, tests(//...))')"
|
|
if [ -n "$flaky" ]; then
|
|
echo "::error::flaky = True on ${flaky//$'\n'/ }: the gated lanes run each test once"
|
|
exit 1
|
|
fi
|
|
|
|
# --build_event_json_file is a command option, so it goes after `test`.
|
|
- name: bazel test //... --config=embedded
|
|
id: test
|
|
timeout-minutes: 50
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=embedded ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=embedded: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# A shard the manifest assigns nothing, or a selector that matches
|
|
# nothing, exits 0: fail if any target's or shard's test.xml lists no
|
|
# tests (or is missing).
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
# Since D2 step 1 this lane is the tier's only pre-merge run on
|
|
# same-repo PRs. The shard scripts discover tests from source, but each
|
|
# Bazel binary holds only its target's srcs: a listed test the binary
|
|
# lacks matches nothing and passes silently. Fail unless every Bazel
|
|
# shard ran exactly the tests its script lists (list-only mode, cheap),
|
|
# and unless every shard and conformance partition ran at least one
|
|
# top-level test that was not skipped (an injected -test.short or
|
|
# BEADS_TEST_SKIP turns the tier into t.Skip calls).
|
|
- name: Every listed test ran in its shard
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: >-
|
|
python3 tools/bazel/check_shard_coverage.py
|
|
--bep "$RUNNER_TEMP/bazel-bep.json"
|
|
--suite //cmd/bd:bd_embedded_test .github/scripts/embedded-test-shard.sh 50
|
|
--suite //internal/storage/embeddeddolt:embeddeddolt_embedded_test .github/scripts/embedded-storage-test-shard.sh 15
|
|
--whole //internal/storage/embeddeddolt:embeddeddolt_conformance_core_test
|
|
--whole //internal/storage/embeddeddolt:embeddeddolt_conformance_audit_test
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-embedded-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# main.yml's "Main Linux integration packages (N/6)" and "Main Linux
|
|
# integration cmd/bd (N/8)" jobs (nightly.yml's Full Test Suite runs the
|
|
# same go test in one job) as --config=integration: every package with
|
|
# -tags=integration,gms_pure_go, race, no -short, BEADS_TEST_SKIP=dolt (see
|
|
# .bazelrc). A job of its own: ci and integration share bazel-testlogs, so
|
|
# bazel-test's equivalence step would read this lane's test.xml if both ran
|
|
# in one job, and the tagged race build is a second full compile.
|
|
# No equivalence step: equivalence.py's default mode is the --config=ci
|
|
# selection (go list without the integration tag, -short, ^TestEmbedded
|
|
# skipped), and its job mode needs the integration jobs' go test -json,
|
|
# which no run here has. check_testcases.py still catches a target or
|
|
# shard that ran no tests. pr.yml's gate requires it (BAZEL_INTEGRATION)
|
|
# on same-repo PRs; see the header.
|
|
# Exactly-once cmd/bd: where the caller says bazel-cmd-dolt is required
|
|
# (inputs.cmd-dolt-required true: pr.yml while its
|
|
# BAZEL_CMD_DOLT_REQUIRED is "true"), the target set is //... minus
|
|
# //cmd/bd:bd_test. bazel-cmd-dolt has this job's `if:`, so it runs in
|
|
# every mode this one does (remote, fork-ro, fork-rw, cache; Dependabot
|
|
# too), bazel-gate.sh accepts the two lanes' skips in the same modes, and
|
|
# its //cmd/bd:bd_dolt_server_test runs that same integration-build
|
|
# bd_test without BEADS_TEST_SKIP=dolt (a superset: the skip only skips).
|
|
# Rollback: commit BAZEL_CMD_DOLT_REQUIRED "false" and pr.yml's
|
|
# cmd-dolt-required: false together (policy-tested equal); bd_test is
|
|
# back here on the next run.
|
|
bazel-integration:
|
|
name: integration
|
|
# Remote (modes remote, fork-ro, fork-rw), or local with the read-only
|
|
# cache (mode cache: fork and Dependabot PRs while rbe-fork is closed,
|
|
# rbe=cache dispatches), which serves every action
|
|
# trusted CI already ran, so only what the PR changed builds and tests
|
|
# on the GitHub-hosted runner. Never plain local (rbe=off): the tagged
|
|
# race build of //... plus 14 jobs' worth of tests on one runner, cold.
|
|
# A caller may turn it off with integration: "off"; pr.yml and
|
|
# bazel-farm.yml pass "on".
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ (needs.rbe.outputs.enabled == 'true' || needs.rbe.outputs.mode == 'cache') && inputs.integration != 'off' }}
|
|
timeout-minutes: 85
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Remote: a cold compile (a PR touching a widely imported package; the
|
|
# first cold run took 17 minutes end to end, warm ones ~3.5) plus the
|
|
# longest test action (1200s cap, see .bazelrc) if it starts last.
|
|
# scripts/ci_workflow_test.go keeps this at or above that sum. Mode
|
|
# cache on a 4-CPU runner (simulated, 2026-10-02): ~2.5 minutes warm,
|
|
# 3-5 for a cmd/bd change, plus up to ~11 more when slow blob reads
|
|
# make Bazel restart the build ("Lost inputs"), and ~30 with the cache
|
|
# closed (the farm's kill switch: everything local, cold), which must
|
|
# still fit so closing the cache does not turn fork PRs red.
|
|
# Where bazel-cmd-dolt is required, //cmd/bd:bd_test is excluded (see
|
|
# above): on a PR that changes cmd/bd or what it imports, its race
|
|
# link and 16 shards were this step's last ~2-3.5 minutes (runs of
|
|
# 2026-10-05: 215-314s wall, 113-208s of it after every other test
|
|
# had finished).
|
|
- name: bazel test //... --config=integration
|
|
id: test
|
|
timeout-minutes: 75
|
|
env:
|
|
EXCLUDE_TARGETS: ${{ inputs.cmd-dolt-required == true && '-//cmd/bd:bd_test' || '' }}
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
if [[ -n "$EXCLUDE_TARGETS" ]]; then
|
|
echo "bazel-cmd-dolt is required: excluding $EXCLUDE_TARGETS (it runs there)" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
fi
|
|
bazel test //... --config=integration ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
${EXCLUDE_TARGETS:+-- "$EXCLUDE_TARGETS"} \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded. dolt_version_test and the repository guards in
|
|
# //scripts/repochecks are plain sh_tests (no Go testcases in their
|
|
# test.xml), which --config=integration's tag filter keeps, unlike the
|
|
# embedded one.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json" --not-go //tools/bazel:dolt_version_test --not-go //scripts/repochecks:all
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-integration-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr.yml's container-backed jobs, "Test (storage domain + uow)" and
|
|
# "Contract corpus", as Bazel's dolt-server lane: --config=doltserver runs
|
|
# the dolt-server variants (domain/..., uow in 8 shards, tracker/...,
|
|
# doctor/fix, protocol in 8 shards), each of which starts its own dolt
|
|
# sql-server from the pinned dolt, so they execute on the remote workers
|
|
# like any other test (locally for forks and rbe=off). The docker lane
|
|
# (--config=docker: the same suites against testcontainers on this runner,
|
|
# no-remote-exec) stays the A/B control: dispatch with dolt-lane=docker.
|
|
bazel-doltserver:
|
|
name: dolt-server lane
|
|
# Same runner and skip rule as bazel-test.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.mode != 'skip' }}
|
|
timeout-minutes: 90
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
env:
|
|
BAZEL_DOLT_LANE: ${{ inputs.dolt-lane || 'doltserver' }}
|
|
# D2 step 3: on covered PRs and merge groups (always mode remote) this
|
|
# lane is the only pre-merge run of pr.yml's legacy domain+uow and
|
|
# contract corpus jobs, which never retried: --config=sole-run then
|
|
# fails on a remote cache eviction instead of re-running the whole
|
|
# invocation (whose retry could turn a failed test green). Fork modes do
|
|
# the same (with BAZEL_COVERS_FORKS "true" it is their only run too).
|
|
# Test results are cached and reused like every lane's; only nightly
|
|
# re-executes them all (BAZEL_FRESH).
|
|
BAZEL_SOLE_RUN: ${{ needs.rbe.outputs.enabled == 'true' && '--config=sole-run' || '' }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Docker lane only (the A/B control), as in the mirrored jobs; its
|
|
# tests fail rather than skip without the image.
|
|
- name: Pull Dolt sql-server image
|
|
if: ${{ env.BAZEL_DOLT_LANE == 'docker' }}
|
|
run: ./scripts/ci/pull-dolt-image.sh
|
|
|
|
- name: bazel test //... --config=doltserver
|
|
id: test
|
|
timeout-minutes: 80
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... "--config=$BAZEL_DOLT_LANE" ${BAZEL_SOLE_RUN:+"$BAZEL_SOLE_RUN"} ${BAZEL_FRESH:+"$BAZEL_FRESH"} 2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=$BAZEL_DOLT_LANE: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome == 'failure' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-${{ env.BAZEL_DOLT_LANE }}-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# Like pr-risk.yml's proxied-server tier, "Test (Proxied Dolt Cmd N/15)",
|
|
# as --config=doltserver-proxied: //cmd/bd:bd_proxied_test runs the shard
|
|
# script, so shard k runs exactly that manifest entry's tests. This lane
|
|
# runs 30 shards of its own (.github/scripts/proxied-cmd-test-shards.txt's
|
|
# duration-balanced 30-block; see scripts/ci/proxied_test_durations.json),
|
|
# not the legacy jobs' 15 — shard k here is not the same split as their
|
|
# shard k. Same build as --config=doltserver (race bd_test, non-race
|
|
# bd_for_tests), shared with bazel-doltserver through the remote cache.
|
|
# PR Risk builds its bd-cmd-test with -race, as here; main.yml's
|
|
# test-proxied-cmd twin uses build-artifacts' non-race bd-cmd-test, so
|
|
# this lane is the stricter of the two. It also runs
|
|
# //cmd/bd:bd_managed_local_test, the managed-local proxied lifecycle
|
|
# (formerly proxied-local-smoke.yml) in a loopback-only network
|
|
# namespace, which needs the workers' unprivileged user namespaces. A job
|
|
# of its own, not gated by any integration switch: it runs in pr.yml's
|
|
# call and its gate requires it (BAZEL_PROXIED).
|
|
bazel-proxied:
|
|
name: proxied-server tier
|
|
# Remote execution only, like bazel-embedded: 30 concurrent shards, each
|
|
# running its own dolt sql-server busy on ~4 cores, do not fit a
|
|
# GitHub-hosted runner regardless of any single shard's wall time.
|
|
# On same-repo pull_requests the legacy jobs stand down (pr-risk.yml's
|
|
# bazel-coverage; D2 step 2), so this lane is then the tier's only PR
|
|
# run and pr.yml's gate requires it to have run here and passed.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Bazel retries a flaky = True test (up to three attempts) and reports a
|
|
# later pass as PASSED. The policy test greps BUILD files for it; this
|
|
# asks Bazel itself, so a macro or a variable cannot hide one. Cheap:
|
|
# loading only, with the repository cache the test step needs anyway.
|
|
- name: No test target is marked flaky
|
|
run: |
|
|
flaky="$(bazel query 'attr(flaky, 1, tests(//...))')"
|
|
if [ -n "$flaky" ]; then
|
|
echo "::error::flaky = True on ${flaky//$'\n'/ }: the gated lanes run each test once"
|
|
exit 1
|
|
fi
|
|
|
|
# Remote: each shard's Go timeout is its job's 15m, 1200s Bazel cap,
|
|
# plus a compile. scripts/ci/proxied_test_durations.json's 176 entries
|
|
# are each a clean, single-job (no concurrent heavy bazel/dolt job
|
|
# intentionally started on the box; see that file's header for its
|
|
# ambient-contention caveat), -test.parallel=4 top-level-test
|
|
# duration — the exact config this job uses — from a single fresh,
|
|
# manifest-aligned 30-shard sweep's own <testsuite time> (errors="0",
|
|
# so a failing-but-complete suite's time is still trustworthy); every
|
|
# one of the 176 discovered tests completed inside its shard run, so
|
|
# no `bazel run` leaf-summation workaround was needed this time. See
|
|
# that file's header for the full methodology, including why an
|
|
# earlier -test.parallel=1 (serial) capture was replaced: serial
|
|
# timings don't model this lane (which always runs parallel=4), and
|
|
# an independent re-review measured near-zero correlation (r=0.08)
|
|
# between the serial-derived predicted shard load and actual
|
|
# parallel=4 wall time.
|
|
#
|
|
# Measured per-shard wall time for the full 30-shard local sweep that
|
|
# produced this data: shards ranged from roughly 32s to 341s, with
|
|
# most clustering well under this job's 25m step timeout / 30m job
|
|
# timeout. After repacking with these corrected weights, the LPT
|
|
# estimate clusters every shard within a few seconds of ~355-360s
|
|
# (sum of all 176 measured durations / 30 shards) — no single test or
|
|
# small group of tests dominates a shard the way the discarded serial
|
|
# data wrongly implied. This is a local, single-job measurement, not
|
|
# a verified RBE forecast (this job has only ever run remote); if a
|
|
# shard's RBE wall time diverges sharply from this local estimate,
|
|
# that is RBE throughput differing from local execution, not
|
|
# necessarily a regression — compare against this job's own recent
|
|
# run history before assuming a hang.
|
|
- name: bazel test //... --config=doltserver-proxied
|
|
id: test
|
|
timeout-minutes: 25
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=doltserver-proxied ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=doltserver-proxied: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
# Since D2 step 2 this lane is the tier's only pre-merge run on
|
|
# same-repo PRs: as in bazel-embedded, every Bazel shard must run
|
|
# exactly the tests its shard script lists, and not only skips.
|
|
- name: Every listed test ran in its shard
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: >-
|
|
python3 tools/bazel/check_shard_coverage.py
|
|
--bep "$RUNNER_TEMP/bazel-bep.json"
|
|
--suite //cmd/bd:bd_proxied_test .github/scripts/proxied-test-shard.sh 30
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-proxied-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# pr-risk.yml's server-Dolt storage tier, "Test (Server Dolt Conformance)"
|
|
# and "Test (Server Dolt Full Suite N/16)", as --config=doltserver-integration:
|
|
# the conformance and 16-shard full-suite variants of
|
|
# //internal/storage/dolt over a non-race dolt_test build, each shard
|
|
# running the job's own shard script. --config=doltserver-integration has
|
|
# exactly --config=integration's build flags, so it compiles nothing
|
|
# bazel-integration does not (shared through the remote cache). A job of
|
|
# its own, not gated by any integration switch: this is a PR-time tier,
|
|
# unlike main.yml's integration jobs, and pr.yml's gate requires it
|
|
# (BAZEL_SERVER_STORAGE).
|
|
bazel-server-storage:
|
|
name: server-Dolt storage tier
|
|
# Remote execution only, like bazel-integration: locally the tagged race
|
|
# build of //... alone is most of the timeout.
|
|
# On same-repo pull_requests the legacy jobs stand down (pr-risk.yml's
|
|
# bazel-coverage; D2 step 2), so this lane is then the tier's only PR
|
|
# run and pr.yml's gate requires it to have run here and passed.
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ needs.rbe.outputs.enabled == 'true' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Bazel retries a flaky = True test (up to three attempts) and reports a
|
|
# later pass as PASSED. The policy test greps BUILD files for it; this
|
|
# asks Bazel itself, so a macro or a variable cannot hide one. Cheap:
|
|
# loading only, with the repository cache the test step needs anyway.
|
|
- name: No test target is marked flaky
|
|
run: |
|
|
flaky="$(bazel query 'attr(flaky, 1, tests(//...))')"
|
|
if [ -n "$flaky" ]; then
|
|
echo "::error::flaky = True on ${flaky//$'\n'/ }: the gated lanes run each test once"
|
|
exit 1
|
|
fi
|
|
|
|
# Remote: the longest shard takes ~2-3 minutes (the storage suite's Go
|
|
# timeout is 15m per shard, 1200s Bazel cap), plus a compile.
|
|
- name: bazel test //... --config=doltserver-integration
|
|
id: test
|
|
timeout-minutes: 25
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=doltserver-integration ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=doltserver-integration: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded.
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
# Since D2 step 2 this lane is the tier's only pre-merge run on
|
|
# same-repo PRs: as in bazel-embedded, every full-suite shard must run
|
|
# exactly the tests its shard script lists, and neither it nor the
|
|
# conformance target only skips.
|
|
- name: Every listed test ran in its shard
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: >-
|
|
python3 tools/bazel/check_shard_coverage.py
|
|
--bep "$RUNNER_TEMP/bazel-bep.json"
|
|
--suite //internal/storage/dolt:dolt_server_full_test .github/scripts/server-storage-test-shard.sh 16
|
|
--whole //internal/storage/dolt:dolt_server_conformance_test
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-server-storage-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# The cmd/bd Dolt-server tier, as --config=doltserver-cmd: the whole
|
|
# integration-tagged cmd/bd suite (//cmd/bd:bd_dolt_server_test, 16
|
|
# shards) against a hermetic dolt sql-server, i.e. the Dolt-gated cmd/bd
|
|
# tests (TestCLI_*, TestInit*, the store-backed suites) that every other
|
|
# lane and legacy job skips with BEADS_TEST_SKIP=dolt or leaves out of its
|
|
# manifest. --config=doltserver-cmd has exactly --config=integration's
|
|
# build flags, so it shares bazel-integration's compile through the
|
|
# remote cache. Runs exactly where bazel-integration runs (the same if):
|
|
# remotely (modes remote, fork-ro, fork-rw), or locally with the
|
|
# read-only cache (mode cache: fork and Dependabot PRs while rbe-fork is
|
|
# closed), never plain local (rbe=off), and not when a caller passes
|
|
# integration: "off".
|
|
# Advisory: job-level continue-on-error keeps a failure out of this
|
|
# call's aggregate result (BAZEL), and pr.yml's ci-gate requires the
|
|
# lane's own result (BAZEL_CMD_DOLT) only while pr.yml's committed
|
|
# BAZEL_CMD_DOLT_REQUIRED flag is "true" (scripts/ci_workflow_test.go,
|
|
# TestBazelCmdDoltJob); pr.yml then passes cmd-dolt-required: true and
|
|
# this lane is the integration build's only bd_test run
|
|
# (TestBazelIntegrationExcludesBdTestOnlyWhereCmdDoltCovers).
|
|
bazel-cmd-dolt:
|
|
name: cmd/bd Dolt-server tier
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ (needs.rbe.outputs.enabled == 'true' || needs.rbe.outputs.mode == 'cache') && inputs.integration != 'off' }}
|
|
timeout-minutes: 85
|
|
continue-on-error: true
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
# inputs.checkout-sha: see the header (empty: the event's default ref).
|
|
# checkout v7 refuses a fork head on pull_request_target unless opted
|
|
# in; only bazel-farm.yml's authorized call opts in (false elsewhere).
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
BAZEL_FORK_REMOTE: ${{ startsWith(needs.rbe.outputs.mode, 'fork-') && 'true' || '' }}
|
|
RBE_FORK_TIER: ${{ needs.rbe.outputs.tier }}
|
|
RBE_FORK_PR: ${{ github.event.pull_request.number }}
|
|
|
|
# Remote: a compile shared with bazel-integration plus the longest
|
|
# of 16 shards (~3 minutes measured; each shard pays ~30-40 s of
|
|
# TestMain and its own dolt sql-server; Go's -test.timeout=19m,
|
|
# Bazel's 1200s cap). Mode cache on a 4-CPU runner: the compile hits
|
|
# the read-only cache, and a PR that changes cmd/bd or what it imports
|
|
# runs every shard locally, four at a time (~2,200 test-seconds
|
|
# measured unloaded, slower four-up: ~20-30 minutes); with the cache
|
|
# closed, add the cold race build (~30).
|
|
- name: bazel test //... --config=doltserver-cmd
|
|
id: test
|
|
timeout-minutes: 75
|
|
run: |
|
|
set -o pipefail
|
|
start=$(date +%s)
|
|
rc=0
|
|
bazel test //... --config=doltserver-cmd ${BAZEL_FRESH:+"$BAZEL_FRESH"} \
|
|
--build_event_json_file="$RUNNER_TEMP/bazel-bep.json" \
|
|
2>&1 | tee "$RUNNER_TEMP/bazel-test.log" || rc=$?
|
|
echo "bazel test --config=doltserver-cmd: exit $rc, $(( $(date +%s) - start ))s wall" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
exit "$rc"
|
|
|
|
# As in bazel-embedded: every shard of the target ran tests (the Go
|
|
# binary shards itself over every top-level test, so together the
|
|
# shards run each exactly once).
|
|
- name: Every target and shard ran tests
|
|
if: ${{ always() && steps.test.outcome != 'skipped' }}
|
|
run: python3 tools/bazel/check_testcases.py --bep "$RUNNER_TEMP/bazel-bep.json"
|
|
|
|
- name: Upload test logs
|
|
if: ${{ failure() && steps.test.outcome != 'skipped' }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
overwrite: true # a job re-run is the same run: replace, not 409
|
|
name: bazel-cmd-dolt-testlogs
|
|
path: |
|
|
bazel-testlogs/**/test.log
|
|
bazel-testlogs/**/test.xml
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too; job.status is the steps' status, unaffected by the
|
|
# job-level continue-on-error.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# F3: moved here from pr.yml so the gate depends on nothing but the rbe
|
|
# job's decision (not detect-package-gates, build-artifacts, bazel-coverage
|
|
# or the whole bazel call). Only pr.yml opts in (package-gates: "on");
|
|
# bazel-farm.yml and nightly.yml keep the default "off" and this job does
|
|
# not run for them. Same-repo PRs get a Bazel-built bd (no --config=ci:
|
|
# only test:ci is defined in .bazelrc, so bazel build //cmd/bd:bd_for_tests
|
|
# is run directly); forks, Dependabot (whatever rbe-fork says: these gates
|
|
# take no fork certificate) and rbe-off/skip/cache runs fall back to
|
|
# `go build`, same as package-mcp.sh's own prepare_bd_binary() does when
|
|
# BEADS_TEST_BD_BINARY is unset. Hence mode == 'remote' below, not enabled
|
|
# (the fork modes are enabled too).
|
|
package-mcp:
|
|
name: Package Gate (MCP)
|
|
needs: rbe
|
|
# 4 vCPU on purpose: pytest-xdist -n 8 measured ~2.7-3x on 4 CPUs
|
|
# (tools/f3 timing, 2026-10), well past 2 vCPU's ceiling.
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ inputs.package-gates == 'on' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Decide applicability
|
|
id: detect
|
|
env:
|
|
PR_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
|
|
PUSH_BEFORE_SHA: ${{ github.event.before }}
|
|
PUSH_AFTER_SHA: ${{ github.sha }}
|
|
run: ./scripts/ci/detect-package-gates.sh
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
|
|
# No --config=ci: only test:ci is defined in .bazelrc, and `bazel
|
|
# build --config=foo` fails when only test:foo exists. The explicit
|
|
# --@rules_go//go/config:race matches test:ci's top-level build
|
|
# setting (test:ci -> test:prcore), so bd_for_tests' race="off"
|
|
# transition lands in the same k8-fastbuild-ST-... output directory
|
|
# and action-cache key `bazel test --config=ci` already populated,
|
|
# instead of plain k8-fastbuild's; race itself still builds a non-race
|
|
# bd (verified with cquery: both configs cquery to the identical
|
|
# bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path).
|
|
- name: bazel build //cmd/bd:bd_for_tests
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$'
|
|
|
|
- name: Package the Bazel-built bd
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
run: |
|
|
./scripts/ci/package-bazel-bd.sh "$RUNNER_TEMP/bd-artifacts"
|
|
echo "BEADS_TEST_BD_BINARY=$RUNNER_TEMP/bd-artifacts/bd-linux-gms-pure" >> "$GITHUB_ENV"
|
|
# Defense in depth: nothing after this step needs the Bazel server
|
|
# or its RBE client key (uv sync and pytest run PR-controlled
|
|
# code). Shutting down also frees the JVM's memory for the 8
|
|
# xdist workers that follow on this 4 vCPU runner.
|
|
bazel shutdown
|
|
rm -rf "$RUNNER_TEMP/bazel-ci-secret"
|
|
|
|
- name: Set up Go
|
|
id: setup-go
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
- name: Restore Go module cache
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/go/pkg/mod
|
|
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
|
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
|
|
|
- name: Verify Go modules
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
run: go mod verify
|
|
|
|
- name: Restore non-race Go build cache
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ${{ runner.temp }}/go-cache/non-race
|
|
key: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-${{ github.sha }}
|
|
restore-keys: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-
|
|
|
|
- name: Set up Python
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' }}
|
|
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
with:
|
|
python-version: '3.14'
|
|
|
|
- name: Install uv
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' }}
|
|
run: python -m pip install uv==0.11.16
|
|
|
|
- name: Run MCP package gate
|
|
if: ${{ steps.detect.outputs.mcp_package == 'true' }}
|
|
env:
|
|
GOCACHE: ${{ runner.temp }}/go-cache/non-race
|
|
run: make ci-package-mcp
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too; package-gates == 'on' is the only skip condition
|
|
# (the job itself), so PACKAGE_MCP is never "skipped" on pr.yml.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|
|
|
|
# F3: npm's twin of package-mcp above. Structurally identical apart from
|
|
# the language setup (Node instead of Python/uv) and the `make` target.
|
|
package-npm:
|
|
name: Package Gate (npm)
|
|
needs: rbe
|
|
runs-on: ${{ needs.rbe.outputs.mode == 'remote' && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
if: ${{ inputs.package-gates == 'on' }}
|
|
timeout-minutes: 30
|
|
outputs:
|
|
result: ${{ steps.result.outputs.result }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
with:
|
|
ref: ${{ inputs.checkout-sha }}
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
allow-unsafe-pr-checkout: ${{ inputs.fork-farm == 'authorized' && github.event_name == 'pull_request_target' }}
|
|
|
|
- name: Decide applicability
|
|
id: detect
|
|
env:
|
|
PR_BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
|
|
PUSH_BEFORE_SHA: ${{ github.event.before }}
|
|
PUSH_AFTER_SHA: ${{ github.sha }}
|
|
run: ./scripts/ci/detect-package-gates.sh
|
|
|
|
- name: Set up Bazel
|
|
id: bazel
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
uses: ./.github/actions/setup-bazel
|
|
env:
|
|
BAZEL_REMOTE_EXECUTOR: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_WEST_EXECUTOR || '' }}
|
|
RBE_TLS_CERT: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CERT || '' }}
|
|
RBE_TLS_KEY: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_KEY || '' }}
|
|
RBE_TLS_CA: ${{ needs.rbe.outputs.mode == 'remote' && secrets.RBE_TLS_CA || '' }}
|
|
RBE_INSTANCE: ${{ needs.rbe.outputs.mode == 'remote' && 'oss' || '' }}
|
|
BAZEL_FORK_CACHE: ${{ needs.rbe.outputs.mode == 'cache' && 'true' || '' }}
|
|
|
|
# No --config=ci: only test:ci is defined in .bazelrc, and `bazel
|
|
# build --config=foo` fails when only test:foo exists. The explicit
|
|
# --@rules_go//go/config:race matches test:ci's top-level build
|
|
# setting (test:ci -> test:prcore), so bd_for_tests' race="off"
|
|
# transition lands in the same k8-fastbuild-ST-... output directory
|
|
# and action-cache key `bazel test --config=ci` already populated,
|
|
# instead of plain k8-fastbuild's; race itself still builds a non-race
|
|
# bd (verified with cquery: both configs cquery to the identical
|
|
# bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path).
|
|
- name: bazel build //cmd/bd:bd_for_tests
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$'
|
|
|
|
- name: Package the Bazel-built bd
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
|
run: |
|
|
./scripts/ci/package-bazel-bd.sh "$RUNNER_TEMP/bd-artifacts"
|
|
echo "BEADS_TEST_BD_BINARY=$RUNNER_TEMP/bd-artifacts/bd-linux-gms-pure" >> "$GITHUB_ENV"
|
|
# Defense in depth: nothing after this step needs the Bazel server
|
|
# or its RBE client key (npm install runs PR-controlled code).
|
|
bazel shutdown
|
|
rm -rf "$RUNNER_TEMP/bazel-ci-secret"
|
|
|
|
- name: Set up Go
|
|
id: setup-go
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: false
|
|
|
|
- name: Restore Go module cache
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ~/go/pkg/mod
|
|
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
|
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
|
|
|
- name: Verify Go modules
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
run: go mod verify
|
|
|
|
- name: Restore non-race Go build cache
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode != 'remote' }}
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
|
with:
|
|
path: ${{ runner.temp }}/go-cache/non-race
|
|
key: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-${{ github.sha }}
|
|
restore-keys: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-
|
|
|
|
- name: Set up Node.js
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' }}
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v6
|
|
with:
|
|
node-version: '24'
|
|
|
|
- name: Run npm package gate
|
|
if: ${{ steps.detect.outputs.npm_package == 'true' }}
|
|
env:
|
|
GOCACHE: ${{ runner.temp }}/go-cache/non-race
|
|
run: make ci-package-npm
|
|
|
|
# Last step: the job's result as a workflow_call output for pr.yml's
|
|
# ci-gate. always() runs it after a failure, a timeout or a
|
|
# cancellation too; package-gates == 'on' is the only skip condition
|
|
# (the job itself), so PACKAGE_NPM is never "skipped" on pr.yml.
|
|
- name: Record job result
|
|
id: result
|
|
if: ${{ always() }}
|
|
env:
|
|
JOB_STATUS: ${{ job.status }}
|
|
run: echo "result=$JOB_STATUS" >> "$GITHUB_OUTPUT"
|