beads/.github/scripts/bazel-gate.sh
Julian Knutsen 143c418148
ci(bazel): cut per-lane fixed cost — LLVM .tar.zst slice, release cross-compile lane, no protoc build (#7379)
* ci(bazel): fetch the LLVM toolchain as a 149 MB slice; re-key the runner cache

Port of gascity #7279. @llvm_dist_linux_x86_64 downloaded the 1.94 GB
LLVM-22.1.8-Linux-X64.tar.xz and ran a single-threaded `tar -xJf` over
11 GiB to keep a 688 MB slice. That took 94-147 s in every Bazel lane,
and every cgo, race and stdlib action waits on it.

- llvm_dist.bzl gains `sliced`: download_and_extract of a members-only
  .tar.zst, plus a missing-member check. The xz path stays.
- MODULE.bazel pins the toolchain-llvm-22.1.8-slice-1 asset by sha256
  (de614777...6bec5). The URL is gascity's release asset for now; the
  list form leaves room for a beads mirror of the same bytes.
- tools/cc_toolchain/repack_llvm.sh and llvm_members.txt rebuild the slice
  from the pinned upstream release.
- Runner cache v4, keyed on MODULE.bazel too. MODULE.bazel.lock does not
  record llvm_dist, so the v3 key never changed and kept restoring the
  2.9 GB entry that holds the xz archive.
- bazel_hermetic_cc_test: a sliced llvm_dist may only download from a
  gastownhall/beads or gastownhall/gascity release, and its members must
  equal llvm_members.txt.

Verified locally:
- Every file in the extracted slice (2009 entries: types, modes,
  symlink targets, sha256) is identical to the xz extraction.
- aquery action keys are identical, base vs head, at the same workspace
  path: 7565/7565 over deps(//cmd/bd:bd_for_tests) and 7146/7146 over
  the race deps(//internal/storage/embeddeddolt:embeddeddolt_test).
- Extraction time: 178 s (xz) vs 1.9 s (zstd).

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci(bazel): run the release cross-compile as its own lane

After the LLVM slice, bazel-pure is the longest cached Bazel lane (about
170 s). Its "Cross-compile every release target" step takes 54-58 s of
that (78.9k configured targets, 25k cache-hit lookups) and shares no
analysis with the pure invocations, because release_cross is a split
transition.

The step moves, unchanged, into a new job, bazel-release-cross ("release
cross-compile"). The job uses bazel-pure's runner, `needs: rbe` and skip
rule, so it runs in every mode except skip. It reports job.status like
every lane: the workflow_call output is bazel-release-cross, and pr.yml's
CI Gate requires it as BAZEL_RELEASE_CROSS. bazel-gate.sh skips it only in
mode skip, as it does BAZEL_PURE. Merge groups use the same pr.yml gate.
Push and dispatch runs gate nothing. Existing gate ids are unchanged.

The policy test now pins the step to bazel-release-cross, requires the new
gate id, and rejects any other job that runs the script (no double runs).
The new job is listed as having no CI analytics step. It captured no BEP as
a bazel-pure step either, and the analytics steps are unchanged. Docs:
LINTING.md, TESTING.md, CI_REQUIRED_CHECK_TOPOLOGY.md.

Expected: bazel-pure about 110 s on cached runs; the new lane about 85 s,
in parallel. Cost: one more 2-vCPU Blacksmith runner per run.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): stop building protoc for protoc-gen-validate

github.com/envoyproxy/protoc-gen-validate ships validate/validate.pb.go,
but its own validate/BUILD makes the Go library a go_proto_library. So
every lane built protoc from source just to regenerate that file. The
module reaches the graph only through grpc xds and the cloud SDKs.

A gazelle_override now regenerates the module's BUILD files with
`gazelle:proto disable`. It sets build_file_generation = "on" and adds
`gazelle:build_file_name BUILD.bazel`, so gazelle writes BUILD.bazel
(which Bazel prefers) instead of keeping the upstream BUILD. The proto
directive alone changes nothing, because the upstream BUILD files win.
validate is now a plain go_library over validate.pb.go.

Measured locally with aquery:
- //... minus the manual release_cross target: base has 2096 CppCompile,
  612 CppArchive, 314 CppLink and 1 GoProtocGen action; head has 0.
- cquery somepath(//..., @protobuf//:protoc): base goes through
  grpc xds -> envoy core/v3 -> validate_go_proto -> protoc; head is
  empty.
- deps(//cmd/bd:bd_for_tests): 7565 actions become 3077. The race
  deps(//internal/storage/embeddeddolt:embeddeddolt_test): 7146 become
  2658.

MODULE.bazel.lock does not change. The v4 runner cache key includes
MODULE.bazel, so it still moves. Expect one cold compile of the Go
packages above validate on the first run after merge.

Agent-Signature: claude-code-claude-opus-5-5-medium on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(bazel): bazel mod tidy formatting

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 03:13:34 +00:00

62 lines
2.7 KiB
Bash
Executable file

#!/usr/bin/env bash
# pr.yml's ci-gate view of its bazel.yml call (the `bazel` job). bazel.yml's
# rbe job decides the execution mode once and exports it as the call's
# rbe-mode / rbe-enabled outputs; this script reads those outputs and never
# re-derives the decision (repo variable, fork, secrets) itself.
#
# bazel-gate.sh skips the Bazel gate ids whose job is skipped by design
# in this mode, for CI_GATE_SKIPPED_OK:
# skip: every lane and the aggregate (BAZEL)
# local: the remote-only BAZEL_EMBEDDED,
# BAZEL_PROXIED, BAZEL_SERVER_STORAGE,
# BAZEL_INTEGRATION and BAZEL_CMD_DOLT
# cache: BAZEL_EMBEDDED, BAZEL_PROXIED,
# BAZEL_SERVER_STORAGE (integration
# and cmd-dolt run with the read-only
# cache)
# remote, fork-ro, fork-rw: none
# bazel-gate.sh aggregate the value to gate on for BAZEL: the call's
# aggregate result, or, when the mode is missing
# or invalid (the rbe job failed, the call never
# started), an unexpected value that ci-gate.sh
# rejects.
#
# Any other skip, a failure, a cancellation, or a lane that should run but
# reported nothing (pr.yml maps an empty output to skipped) fails the gate.
#
# Inputs (environment): BAZEL_RBE_MODE, BAZEL_RBE_ENABLED (the call's
# rbe-mode / rbe-enabled outputs), BAZEL_CALL (needs.bazel.result).
set -euo pipefail
mode="${BAZEL_RBE_MODE:-}"
enabled="${BAZEL_RBE_ENABLED:-}"
valid=false
case "$mode/$enabled" in
remote/true | fork-ro/true | fork-rw/true | local/false | cache/false | skip/false) valid=true ;;
esac
case "${1:-}" in
skips)
skips=()
if [[ "$valid" == true ]]; then
case "$mode" in
skip) skips+=(BAZEL BAZEL_TEST BAZEL_PURE BAZEL_RELEASE_CROSS BAZEL_EMBEDDED BAZEL_INTEGRATION BAZEL_DOLTSERVER BAZEL_PROXIED BAZEL_SERVER_STORAGE BAZEL_CMD_DOLT) ;;
local) skips+=(BAZEL_EMBEDDED BAZEL_INTEGRATION BAZEL_PROXIED BAZEL_SERVER_STORAGE BAZEL_CMD_DOLT) ;;
cache) skips+=(BAZEL_EMBEDDED BAZEL_PROXIED BAZEL_SERVER_STORAGE) ;;
esac
fi
echo "${skips[*]-}"
;;
aggregate)
if [[ "$valid" != true ]]; then
echo "invalid-rbe-mode:$(printf '%s/%s' "$mode" "$enabled" | tr -cd 'a-z/')"
else
echo "${BAZEL_CALL:-}"
fi
;;
*)
echo "usage: $0 skips|aggregate" >&2
exit 2
;;
esac