ci(bazel): cross-compile every release target with Bazel, retire the go build job (#7332)
pr.yml's check-release-target-cross-compilation ran `CGO_ENABLED=0 GOOS/GOARCH go build -tags gms_pure_go ./...` for the eight release targets on two GitHub/Blacksmith runners. It is now a step of bazel.yml's pure-Go lane (BAZEL_PURE, already required by ci-gate): scripts/ci/bazel-release-cross-compile.sh builds //tools/bazel:release_cross once, remotely, with --//tools/bazel:release_platforms set from scripts/ci/release-targets.txt (still the single source of truth, pinned to .goreleaser.yml). release_cross's split transition configures every go_library/go_binary for each rules_go platform with cgo off, so all eight platforms build in parallel in one invocation. - tools/bazel/release_cross.bzl: the split-transition rule and the string-list flag. - tools/bazel/go_srcs.py (make bazel-sync, checked by bazel-sync-check and TestBazelGoSrcsBlocksCurrent) keeps the release_cross target list; the script checks with bazel query that every Bazel package holding Go targets is reached before it builds. - //tools/bazel:pure_bd_has_no_cgo_only_deps: Bazel compiles gozstd's pure stubs where `go build` rejects them, so the pure bd of every platform must not link gozstd. - //internal/storage/embeddeddolt/cmd:cmd (cgo-only main, skipped by `go build ./...` under CGO_ENABLED=0) is tagged cgo-only and incompatible with pure builds. - A failed build is re-run per platform (cache hits) to name each failing GOOS/GOARCH. Same cgo-off coverage boundary as the retired job. Refs ga-96smfk.20. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
bb9b9e5020
commit
5e73670866
16 changed files with 703 additions and 333 deletions
10
.github/workflows/bazel.yml
vendored
10
.github/workflows/bazel.yml
vendored
|
|
@ -1049,6 +1049,16 @@ jobs:
|
|||
//internal/storage/embeddeddolt:embeddeddolt_test
|
||||
//internal/tracker:tracker_test
|
||||
|
||||
# Every release target (scripts/ci/release-targets.txt) on every PR, so
|
||||
# a platform-gating mistake is caught here instead of at tag time
|
||||
# (#5662): the whole tree, cgo off, one remote `bazel build
|
||||
# --platforms=...` per target (the script's header has the coverage
|
||||
# boundary: the cgo release builds are not compiled here).
|
||||
- name: Cross-compile every release target (--config=release-cross)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
./scripts/ci/bazel-release-cross-compile.sh
|
||||
|
||||
# `go build` with CGO_ENABLED=0 rejects a pure binary whose imports reach
|
||||
# dolt's cgo-only store/nbs (gozstd). Bazel compiles gozstd's pure stubs
|
||||
# instead, whose init panics (third_party/patches/gozstd_nocgo.patch), so
|
||||
|
|
|
|||
62
.github/workflows/pr.yml
vendored
62
.github/workflows/pr.yml
vendored
|
|
@ -233,8 +233,8 @@ jobs:
|
|||
#
|
||||
# D2 step 3: stands down where bazel-coverage says pr_lanes; Bazel's
|
||||
# `pure-Go and js/wasm` lane (bazel-pure) runs the same boundaries, and
|
||||
# check-release-target-cross-compilation still `go build`s ./... with
|
||||
# CGO_ENABLED=0 on every PR.
|
||||
# also builds the whole tree with cgo off for every release target
|
||||
# (scripts/ci/bazel-release-cross-compile.sh).
|
||||
check-cmd-bd-puregeo-tests:
|
||||
name: Check pure-Go and js/wasm boundaries (CGO_ENABLED=0)
|
||||
runs-on: ubuntu-latest
|
||||
|
|
@ -318,61 +318,6 @@ jobs:
|
|||
run: |
|
||||
go test -tags gms_pure_go -count=1 -short -run '^Test(Help|CheckRemoteSafety|FormatDestroyToken|ShouldWireInitRemote|ExtractPrefix|IsNumericID|GetWorktreeGitDir|DriftItemStatuses|RunDriftChecks|IsServerProbablyRunning|CheckHooksDriftNotGitRepo|CheckServerDriftNoBeadsDir|CheckRemoteDriftNoBeadsDir|NocgoFactoriesHonorSharedServer|NocgoNewDoltStoreFromConfig_ErrorSuggestsCorrectFlag|NocgoNewReadOnlyStoreFromConfig_ErrorSuggestsCorrectFlag)' ./cmd/bd
|
||||
|
||||
# Compile every shipped release target on every PR, so a platform-gating
|
||||
# mistake is caught here instead of at tag time (#5662).
|
||||
#
|
||||
# Coverage boundary, recorded so the green is not over-read: every leg below
|
||||
# builds CGO_ENABLED=0, but .goreleaser.yml builds bd-linux-amd64,
|
||||
# bd-linux-arm64 and bd-windows-amd64 with CGO_ENABLED=1, and the darwin pair
|
||||
# is built natively at CGO_ENABLED=1 by release.yml's goreleaser-macos job.
|
||||
# Five of these eight legs are therefore a weaker proxy than the binary they
|
||||
# stand for: the cgo half of the tree (beads_cgo.go ->
|
||||
# internal/storage/embeddeddolt) is never compiled here, so a break confined
|
||||
# to it still passes this check. Only android/arm64, windows/arm64 and
|
||||
# freebsd/amd64 replicate their release configuration exactly.
|
||||
#
|
||||
# The build scope is deliberately ./... and not the ./cmd/bd that every
|
||||
# .goreleaser.yml build id declares as main:. Compiling the whole tree per
|
||||
# platform is intentional extra coverage, not an oversight.
|
||||
#
|
||||
# F7a: folded from 8 per-target jobs into 2 per-group jobs. The target list
|
||||
# moved out of this matrix and into scripts/ci/release-targets.txt, which
|
||||
# scripts/ci/check-release-cross-compile.sh reads and
|
||||
# TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser pins against
|
||||
# .goreleaser.yml, so the manifest (not this YAML) is now the single
|
||||
# source of truth for the target set. Moved to Blacksmith for same-repo
|
||||
# PRs/merge_group: this job restores no GitHub-saved build cache (only a
|
||||
# best-effort module-cache restore, whose misses are cheap network
|
||||
# fetches, not lost compute). Runner choice must match the
|
||||
# TestSameRepoBlacksmithRunners pin.
|
||||
check-release-target-cross-compilation:
|
||||
name: Check release target cross-compilation (${{ matrix.group }})
|
||||
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
group: [unix, desktop]
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Go
|
||||
id: setup-go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: false
|
||||
|
||||
- name: Restore Go module cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ~/go/pkg/mod
|
||||
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
||||
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Build complete release target tree for group ${{ matrix.group }}
|
||||
run: ./scripts/ci/check-release-cross-compile.sh ${{ matrix.group }}
|
||||
|
||||
# The lingering-handle liveness regression test is //go:build windows: the
|
||||
# Ubuntu job above selects it by name but compiles it out, and main.yml's
|
||||
# Windows job runs only on pushes to main (build/smoke, no go test). Run
|
||||
|
|
@ -2290,7 +2235,6 @@ jobs:
|
|||
- build-artifacts
|
||||
- fast-checks
|
||||
- check-cmd-bd-puregeo-tests
|
||||
- check-release-target-cross-compilation
|
||||
- test-windows-liveness
|
||||
- worktree-remove-windows
|
||||
- test-windows-liveness-prebuilt
|
||||
|
|
@ -2319,7 +2263,6 @@ jobs:
|
|||
CHECK_MIGRATION_HYGIENE
|
||||
CHECK_NO_BEADS_CHANGES
|
||||
CHECK_CMD_BD_PUREGEO_TESTS
|
||||
CHECK_RELEASE_TARGET_CROSS_COMPILATION
|
||||
CHECK_DOC_FLAGS
|
||||
CHECK_DOC_FRESHNESS_PLATFORMS
|
||||
PR_PREFLIGHT_PLATFORMS
|
||||
|
|
@ -2359,7 +2302,6 @@ jobs:
|
|||
# job.
|
||||
FAST_CHECKS: ${{ needs.fast-checks.result }}
|
||||
CHECK_CMD_BD_PUREGEO_TESTS: ${{ needs.check-cmd-bd-puregeo-tests.result }}
|
||||
CHECK_RELEASE_TARGET_CROSS_COMPILATION: ${{ needs.check-release-target-cross-compilation.result }}
|
||||
TEST_WINDOWS_LIVENESS: ${{ needs.test-windows-liveness.result }}
|
||||
WORKTREE_REMOVE_WINDOWS: ${{ needs.worktree-remove-windows.result }}
|
||||
# F4 side-by-side rollout: always evaluated, like the native pair
|
||||
|
|
|
|||
21
RELEASING.md
21
RELEASING.md
|
|
@ -143,9 +143,9 @@ Before starting a release:
|
|||
- [ ] **Upgrade smoke tests pass** (`make test-upgrade`) — see [Release Stability Gate](engdocs/RELEASE-STABILITY-GATE.md)
|
||||
- [ ] **Regression tests pass** (`make test-regression`)
|
||||
- [ ] **Every release target cross-compiles** — see
|
||||
[Cross-compile before tagging](#cross-compile-before-tagging). PR CI does
|
||||
not build them ([#5662](https://github.com/gastownhall/beads/issues/5662)),
|
||||
and a target that fails at tag time burns the tag.
|
||||
[Cross-compile before tagging](#cross-compile-before-tagging). PR CI
|
||||
builds them only with cgo off, and a target that fails at tag time
|
||||
burns the tag.
|
||||
- [ ] **CHANGELOG.md updated with release notes** (see format below)
|
||||
- [ ] **CHANGELOG rollup checked** — nothing left under `[Unreleased]` that
|
||||
belongs in this release, and nothing filed under a *previous* release's
|
||||
|
|
@ -163,12 +163,15 @@ Before starting a release:
|
|||
|
||||
### Cross-compile before tagging
|
||||
|
||||
`.goreleaser.yml` builds darwin, linux, windows **and freebsd**, but PR CI
|
||||
builds none of the cross targets — the v1.2.0 tag burned on a freebsd
|
||||
compilation failure that no pre-tag gate could have caught
|
||||
([#5661](https://github.com/gastownhall/beads/pull/5661) fixed the break,
|
||||
[#5662](https://github.com/gastownhall/beads/issues/5662) tracks the CI gap).
|
||||
Until that gap is closed, build them by hand before tagging:
|
||||
`.goreleaser.yml` builds darwin, linux, windows **and freebsd**; the v1.2.0
|
||||
tag burned on a freebsd compilation failure no pre-tag gate caught
|
||||
([#5661](https://github.com/gastownhall/beads/pull/5661),
|
||||
[#5662](https://github.com/gastownhall/beads/issues/5662)). PR CI now builds
|
||||
the whole tree for every target in `scripts/ci/release-targets.txt` with Bazel
|
||||
(`scripts/ci/bazel-release-cross-compile.sh`, in bazel.yml's pure-Go lane),
|
||||
but with cgo off, while linux/amd64, linux/arm64, windows/amd64 and the darwin
|
||||
pair ship with `CGO_ENABLED=1`. Build the real release configuration by hand
|
||||
before tagging:
|
||||
|
||||
```bash
|
||||
goreleaser build --snapshot --clean
|
||||
|
|
|
|||
|
|
@ -639,8 +639,6 @@ Do not require these existing check names directly:
|
|||
[Repository Guards as Bazel Tests](#repository-guards-as-bazel-tests))
|
||||
- `Check pure-Go and js/wasm boundaries (CGO_ENABLED=0)`
|
||||
- `Check doc flags freshness`
|
||||
- `Check release target cross-compilation (unix)` and `(desktop)` (F7a fold
|
||||
of the former eight-target `check-release-target-cross-compilation` matrix)
|
||||
- `Windows Make shell (native, msys2, cygwin)` (F7a fold of the former
|
||||
`host: [native, msys2, cygwin]` three-job matrix)
|
||||
- `Test (Windows) small packages (doltversion, dbproxy server)` (F7a fold of
|
||||
|
|
@ -925,8 +923,16 @@ manifests. On those PRs they are the tiers' only pre-merge run, and
|
|||
(`Test (Dolt server fingerprint)`, `TEST_DOLT_SERVER_FINGERPRINT`)
|
||||
pulled a docker image the workers do not have, so it is retired; the
|
||||
image tag stays pinned to the dolt release by
|
||||
`TestPinnedDoltCLIMatchesContainerImage`. `check-release-target-cross-compilation`
|
||||
still `go build`s `./...` with `CGO_ENABLED=0` on every PR.
|
||||
`TestPinnedDoltCLIMatchesContainerImage`. The release-target
|
||||
cross-compilation (formerly pr.yml's
|
||||
`check-release-target-cross-compilation`, `go build ./...` with
|
||||
`CGO_ENABLED=0` per target) is a step of bazel.yml's `bazel-pure` lane
|
||||
(`BAZEL_PURE`): `scripts/ci/bazel-release-cross-compile.sh` runs one
|
||||
remote `bazel build //tools/bazel:release_cross`, every `go_library` and
|
||||
`go_binary` for each row of `scripts/ci/release-targets.txt`, plus
|
||||
`//tools/bazel:pure_bd_has_no_cgo_only_deps` (a pure bd must not link
|
||||
gozstd, which `go build` rejects with cgo off). Same cgo-off boundary as
|
||||
the job it replaces.
|
||||
- Also kept on every PR, in the required job `scripts-go-checks`
|
||||
(`SCRIPTS_GO_CHECKS`, one leg, `Go checks (vet)`): `go test`'s own vet
|
||||
checks (cmd/go's `defaultVetFlags`, policy-tested equal to the
|
||||
|
|
@ -1219,7 +1225,8 @@ scope, not this slice's.
|
|||
`scripts/ci/check-release-cross-compile.sh <group>`, which builds every
|
||||
target in its group sequentially and reports every failure before exiting
|
||||
non-zero, so a PR touching two platforms at once sees both failures in one
|
||||
log instead of needing a per-target re-run.
|
||||
log instead of needing a per-target re-run. (Since retired: bazel.yml's `bazel-pure`
|
||||
lane builds the same manifest with Bazel, `--platforms` per target.)
|
||||
- **`advisory-reports` fold.** `build-examples` and `complexity-report` (both
|
||||
already advisory: neither was in ci-gate's `needs`/`CI_GATE_REQUIRED`)
|
||||
became one job's steps. Each keeps its pre-fold step-level
|
||||
|
|
@ -1531,7 +1538,6 @@ PR timings, 2026-10): `fast-checks` (~40 s), `pr-policy-wrapper`
|
|||
(native/darwin/windows, up to ~4 min), `check-doc-flags` (~1.7 min),
|
||||
`check-doc-freshness-platforms` and `pr-preflight-platforms` (Linux,
|
||||
Windows and macOS legs, up to ~5 min on Windows),
|
||||
`check-release-target-cross-compilation` (~5 min),
|
||||
`windows-make-shell` (~2.7 min),
|
||||
`windows-test-binaries` plus the prebuilt Windows pair (~7 min end to
|
||||
end), the advisory native Windows pair (~5 min; the gate waits for them),
|
||||
|
|
|
|||
|
|
@ -8,9 +8,17 @@ go_library(
|
|||
deps = ["//internal/storage/embeddeddolt"],
|
||||
)
|
||||
|
||||
# main.go is //go:build cgo: without cgo the package is empty, which
|
||||
# `go build ./...` skips and a go_binary cannot link. cgo-only: left out of
|
||||
# //tools/bazel:release_cross (tools/bazel/go_srcs.py).
|
||||
go_binary(
|
||||
name = "cmd",
|
||||
embed = [":cmd_lib"],
|
||||
tags = ["cgo-only"],
|
||||
target_compatible_with = select({
|
||||
"//tools/bazel:pure": ["@platforms//:incompatible"],
|
||||
"//conditions:default": [],
|
||||
}),
|
||||
visibility = ["//:__subpackages__"],
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -852,6 +852,12 @@ var allowedBazelTestTags = map[string]string{
|
|||
// integration tag, as releases are, and running it again under the
|
||||
// integration build would only repeat it against a test-shaped bd.
|
||||
"pr-core-only": "runs only in --config=prcore/ci; excluded from --config=integration",
|
||||
// For a go_binary whose every source file is `//go:build cgo`: it also
|
||||
// sets target_compatible_with incompatible under //tools/bazel:pure, and
|
||||
// tools/bazel/go_srcs.py and scripts/ci/bazel-release-cross-compile.sh
|
||||
// leave it out of //tools/bazel:release_cross by this tag (`go build
|
||||
// ./...` skips the package with CGO_ENABLED=0).
|
||||
"cgo-only": "a go_binary with only cgo sources: incompatible with pure builds and left out of //tools/bazel:release_cross",
|
||||
}
|
||||
|
||||
// bazelPRCoreExcludedTags are the tags whose targets never run in the PR-core
|
||||
|
|
|
|||
|
|
@ -9,155 +9,219 @@ import (
|
|||
"testing"
|
||||
)
|
||||
|
||||
// Review SF-6 (2026-10-03): check-release-cross-compile.sh (the F7a fold of
|
||||
// pr.yml's check-release-target-cross-compilation matrix into two per-group
|
||||
// legs) had no behavioural test. A planted "always exit 0" mutation passed
|
||||
// every other policy test, which would make the required
|
||||
// CHECK_RELEASE_TARGET_CROSS_COMPILATION token green even with every target
|
||||
// broken. These tests put a fake `go` on PATH that records every invocation
|
||||
// and can be told to fail for one target, then assert the script's actual
|
||||
// failure-propagation, env/flag, and group-coverage behaviour against it.
|
||||
// bazel-release-cross-compile.sh is the release-target cross-compilation
|
||||
// gate: bazel.yml's pure-Go lane runs it, and it builds
|
||||
// //tools/bazel:release_cross for every row of scripts/ci/release-targets.txt
|
||||
// in one invocation. These tests put a fake `bazel` on PATH that records
|
||||
// every invocation and can be told to fail for one platform, then assert the
|
||||
// script's platform list, coverage check, flags, failure propagation and
|
||||
// per-platform attribution against it, so an "always exit 0" mutation cannot
|
||||
// keep the gate green with every target broken.
|
||||
|
||||
// fakeGoRecordingScript is a stand-in for `go build` that appends one line
|
||||
// per invocation to $GO_CALL_LOG (goos, goarch, CGO_ENABLED, and the
|
||||
// remaining args) and exits non-zero only when GOOS/GOARCH matches
|
||||
// $GO_FAIL_TARGET.
|
||||
const fakeGoRecordingScript = `#!/usr/bin/env bash
|
||||
// fakeBazelRecordingScript stands in for bazel. `query` prints
|
||||
// $FAKE_BUILT_PKGS for the deps of //tools/bazel:release_cross and
|
||||
// $FAKE_WANT_PKGS for anything else; every command appends one line
|
||||
// ("<command>\t<args>") to $BAZEL_CALL_LOG; `build` exits non-zero when its
|
||||
// --//tools/bazel:release_platforms list names $BAZEL_FAIL_PLATFORM.
|
||||
const fakeBazelRecordingScript = `#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
printf '%s\t%s\t%s\t%s\n' "${GOOS:-}" "${GOARCH:-}" "${CGO_ENABLED:-}" "$*" >> "$GO_CALL_LOG"
|
||||
if [ "${GOOS:-}/${GOARCH:-}" = "${GO_FAIL_TARGET:-}" ]; then
|
||||
echo "simulated build failure for ${GOOS:-}/${GOARCH:-}" >&2
|
||||
exit 1
|
||||
cmd="$1"; shift
|
||||
printf '%s\t%s\n' "$cmd" "$*" >> "$BAZEL_CALL_LOG"
|
||||
if [ "$cmd" = query ]; then
|
||||
case "$*" in
|
||||
*"deps(//tools/bazel:release_cross)"*) printf '%s\n' $FAKE_BUILT_PKGS ;;
|
||||
*) printf '%s\n' $FAKE_WANT_PKGS ;;
|
||||
esac
|
||||
exit 0
|
||||
fi
|
||||
for a in "$@"; do
|
||||
case "$a" in
|
||||
--//tools/bazel:release_platforms=*)
|
||||
if [ -n "${BAZEL_FAIL_PLATFORM:-}" ] && [[ ",${a#*=}," == *",$BAZEL_FAIL_PLATFORM,"* ]]; then
|
||||
echo "simulated build failure for $a" >&2
|
||||
exit 1
|
||||
fi ;;
|
||||
esac
|
||||
done
|
||||
exit 0
|
||||
`
|
||||
|
||||
// runCheckReleaseCrossCompile copies the real check-release-cross-compile.sh
|
||||
// into a throwaway directory alongside a caller-supplied manifest (so the
|
||||
// test does not depend on the real scripts/ci/release-targets.txt growing or
|
||||
// shrinking), puts a fake `go` on PATH ahead of everything else, and runs the
|
||||
// script for the given group. failTarget (e.g. "linux/amd64") is the single
|
||||
// GOOS/GOARCH the fake go fails for; empty means every target succeeds.
|
||||
func runCheckReleaseCrossCompile(t *testing.T, group, manifest, failTarget string) (out string, err error, log string) {
|
||||
const testPkgs = "cmd/bd internal/types"
|
||||
|
||||
// runBazelReleaseCrossCompile copies the real script next to a
|
||||
// caller-supplied manifest (so the test does not track the real
|
||||
// release-targets.txt), puts the fake bazel first on PATH and runs it.
|
||||
func runBazelReleaseCrossCompile(t *testing.T, manifest, failPlatform, builtPkgs string, args ...string) (out string, err error, calls []string) {
|
||||
t.Helper()
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("script is a Bash boundary")
|
||||
}
|
||||
script, readErr := os.ReadFile(filepath.Join(sourceRepoRoot(t), "scripts", "ci", "check-release-cross-compile.sh"))
|
||||
script, readErr := os.ReadFile(filepath.Join(sourceRepoRoot(t), "scripts", "ci", "bazel-release-cross-compile.sh"))
|
||||
if readErr != nil {
|
||||
t.Fatalf("read check-release-cross-compile.sh: %v", readErr)
|
||||
t.Fatalf("read bazel-release-cross-compile.sh: %v", readErr)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
scriptsCI := filepath.Join(dir, "scripts", "ci")
|
||||
if mkErr := os.MkdirAll(scriptsCI, 0o700); mkErr != nil {
|
||||
t.Fatal(mkErr)
|
||||
}
|
||||
scriptPath := filepath.Join(scriptsCI, "check-release-cross-compile.sh")
|
||||
if writeErr := os.WriteFile(scriptPath, script, 0o700); writeErr != nil {
|
||||
t.Fatal(writeErr)
|
||||
}
|
||||
if writeErr := os.WriteFile(filepath.Join(scriptsCI, "release-targets.txt"), []byte(manifest), 0o600); writeErr != nil {
|
||||
t.Fatal(writeErr)
|
||||
}
|
||||
|
||||
fakeBin := filepath.Join(dir, "fakebin")
|
||||
if mkErr := os.MkdirAll(fakeBin, 0o700); mkErr != nil {
|
||||
t.Fatal(mkErr)
|
||||
for _, d := range []string{scriptsCI, fakeBin} {
|
||||
if mkErr := os.MkdirAll(d, 0o700); mkErr != nil {
|
||||
t.Fatal(mkErr)
|
||||
}
|
||||
}
|
||||
if writeErr := os.WriteFile(filepath.Join(fakeBin, "go"), []byte(fakeGoRecordingScript), 0o700); writeErr != nil {
|
||||
t.Fatal(writeErr)
|
||||
scriptPath := filepath.Join(scriptsCI, "bazel-release-cross-compile.sh")
|
||||
for path, content := range map[string]string{
|
||||
scriptPath: string(script),
|
||||
filepath.Join(scriptsCI, "release-targets.txt"): manifest,
|
||||
filepath.Join(fakeBin, "bazel"): fakeBazelRecordingScript,
|
||||
} {
|
||||
if writeErr := os.WriteFile(path, []byte(content), 0o700); writeErr != nil {
|
||||
t.Fatal(writeErr)
|
||||
}
|
||||
}
|
||||
logPath := filepath.Join(dir, "go-calls.log")
|
||||
logPath := filepath.Join(dir, "bazel-calls.log")
|
||||
if writeErr := os.WriteFile(logPath, nil, 0o600); writeErr != nil {
|
||||
t.Fatal(writeErr)
|
||||
}
|
||||
|
||||
cmd := exec.Command("bash", scriptPath, group)
|
||||
cmd := exec.Command("bash", append([]string{scriptPath}, args...)...)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(),
|
||||
"PATH="+fakeBin+":"+os.Getenv("PATH"),
|
||||
"GO_CALL_LOG="+logPath,
|
||||
"GO_FAIL_TARGET="+failTarget,
|
||||
"BAZEL_CALL_LOG="+logPath,
|
||||
"BAZEL_FAIL_PLATFORM="+failPlatform,
|
||||
"FAKE_WANT_PKGS="+testPkgs,
|
||||
"FAKE_BUILT_PKGS="+builtPkgs,
|
||||
)
|
||||
outBytes, runErr := cmd.CombinedOutput()
|
||||
logBytes, _ := os.ReadFile(logPath)
|
||||
return string(outBytes), runErr, string(logBytes)
|
||||
for _, line := range strings.Split(strings.TrimRight(string(logBytes), "\n"), "\n") {
|
||||
if line != "" {
|
||||
calls = append(calls, line)
|
||||
}
|
||||
}
|
||||
return string(outBytes), runErr, calls
|
||||
}
|
||||
|
||||
const testManifest = "linux amd64 unix\n" +
|
||||
"darwin arm64 unix\n" +
|
||||
"windows amd64 desktop\n"
|
||||
const testReleaseManifest = "# comment\n" +
|
||||
"linux amd64\n" +
|
||||
"\n" +
|
||||
"windows arm64\n" +
|
||||
"darwin arm64\n"
|
||||
|
||||
// A failure in one target must propagate as a non-zero exit (the gate's
|
||||
// whole point), must still attempt every other target in the group (so a PR
|
||||
// touching two platforms sees both failures in one run), must name the
|
||||
// failing target in an ::error:: line, and must pass CGO_ENABLED=0 and the
|
||||
// gms_pure_go tag through to every invocation, not just the failing one.
|
||||
func TestCheckReleaseCrossCompilePropagatesFailureAndCoversGroup(t *testing.T) {
|
||||
out, err, log := runCheckReleaseCrossCompile(t, "unix", testManifest, "linux/amd64")
|
||||
if err == nil {
|
||||
t.Fatalf("script exited 0 with a failing target; want non-zero\noutput:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "::error::cross-compilation failed for linux/amd64") {
|
||||
t.Errorf("output does not name the failing target:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "::error::cross-compilation failed for: linux/amd64") {
|
||||
t.Errorf("output does not contain the final failed-targets summary line:\n%s", out)
|
||||
}
|
||||
// Both unix-group targets must have been attempted, in spite of the first
|
||||
// one failing.
|
||||
for _, want := range []string{"linux\tamd64", "darwin\tarm64"} {
|
||||
if !strings.Contains(log, want) {
|
||||
t.Errorf("fake go call log is missing an attempt for %s; every group target must run even after an earlier one fails:\n%s", want, log)
|
||||
}
|
||||
}
|
||||
// The desktop-group target must not have been touched by a unix-group run.
|
||||
if strings.Contains(log, "windows\tamd64") {
|
||||
t.Errorf("fake go call log recorded a desktop-group target during a unix-group run:\n%s", log)
|
||||
}
|
||||
for _, line := range strings.Split(strings.TrimRight(log, "\n"), "\n") {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
fields := strings.Split(line, "\t")
|
||||
if len(fields) != 4 {
|
||||
t.Fatalf("malformed fake go call log line %q", line)
|
||||
}
|
||||
if fields[2] != "0" {
|
||||
t.Errorf("fake go call %q: CGO_ENABLED = %q, want \"0\"", line, fields[2])
|
||||
}
|
||||
if !strings.Contains(fields[3], "-tags gms_pure_go ./...") {
|
||||
t.Errorf("fake go call %q: args = %q, want them to contain \"-tags gms_pure_go ./...\"", line, fields[3])
|
||||
func buildCalls(calls []string) []string {
|
||||
var out []string
|
||||
for _, c := range calls {
|
||||
if strings.HasPrefix(c, "build\t") {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Every target in a group still passes without a failTarget.
|
||||
func TestCheckReleaseCrossCompileAllTargetsPass(t *testing.T) {
|
||||
out, err, log := runCheckReleaseCrossCompile(t, "unix", testManifest, "")
|
||||
const releaseFlag = "--//tools/bazel:release_platforms="
|
||||
|
||||
// One `bazel build --keep_going` of //tools/bazel:release_cross with every
|
||||
// manifest row as a rules_go platform name, the caller's extra flags passed
|
||||
// through, after a coverage check that queries the Go packages bazel knows
|
||||
// and the ones release_cross reaches.
|
||||
func TestBazelReleaseCrossCompileBuildsEveryManifestPlatform(t *testing.T) {
|
||||
out, err, calls := runBazelReleaseCrossCompile(t, testReleaseManifest, "", testPkgs, "--config=remote-exec")
|
||||
if err != nil {
|
||||
t.Fatalf("script failed with no simulated failures: %v\noutput:\n%s", err, out)
|
||||
}
|
||||
if !strings.Contains(out, "cross-compilation passed for group 'unix'") {
|
||||
t.Errorf("output does not report success for the unix group:\n%s", out)
|
||||
if len(calls) != 3 {
|
||||
t.Fatalf("bazel calls = %q, want two queries and one build", calls)
|
||||
}
|
||||
for _, want := range []string{"linux\tamd64", "darwin\tarm64"} {
|
||||
if !strings.Contains(log, want) {
|
||||
t.Errorf("fake go call log is missing an attempt for %s:\n%s", want, log)
|
||||
for i, want := range []string{
|
||||
`kind("go_library|go_binary", //...) except siblings(attr(tags, "\bcgo-only\b", `,
|
||||
"deps(//tools/bazel:release_cross)",
|
||||
} {
|
||||
if !strings.HasPrefix(calls[i], "query\t") || !strings.Contains(calls[i], want) || !strings.Contains(calls[i], "--output=package") {
|
||||
t.Errorf("bazel call %d = %q, want a package query containing %q", i, calls[i], want)
|
||||
}
|
||||
}
|
||||
build := calls[2]
|
||||
for _, required := range []string{
|
||||
"build\t--keep_going",
|
||||
releaseFlag + "linux_amd64,windows_arm64,darwin_arm64",
|
||||
"--config=remote-exec -- //tools/bazel:release_cross",
|
||||
} {
|
||||
if !strings.Contains(build, required) {
|
||||
t.Errorf("build call %q does not contain %q", build, required)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(out, "release cross-compilation passed: linux/amd64 windows/arm64 darwin/arm64") {
|
||||
t.Errorf("output does not report success for every target:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A Go package release_cross does not reach fails the script before any
|
||||
// build, naming the package.
|
||||
func TestBazelReleaseCrossCompileFailsOnUncoveredPackage(t *testing.T) {
|
||||
out, err, calls := runBazelReleaseCrossCompile(t, testReleaseManifest, "", "cmd/bd")
|
||||
if err == nil {
|
||||
t.Fatalf("script exited 0 with an uncovered package; want non-zero\noutput:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "::error::Go packages //tools/bazel:release_cross does not build") || !strings.Contains(out, "internal/types") {
|
||||
t.Errorf("output does not name the uncovered package:\n%s", out)
|
||||
}
|
||||
if b := buildCalls(calls); len(b) != 0 {
|
||||
t.Errorf("bazel build ran despite an uncovered package: %q", b)
|
||||
}
|
||||
}
|
||||
|
||||
// A failing platform fails the script; each platform is then rebuilt alone,
|
||||
// so the output names exactly the failing ones.
|
||||
func TestBazelReleaseCrossCompilePropagatesFailureAndNamesPlatform(t *testing.T) {
|
||||
out, err, calls := runBazelReleaseCrossCompile(t, testReleaseManifest, "windows_arm64", testPkgs)
|
||||
if err == nil {
|
||||
t.Fatalf("script exited 0 with a failing platform; want non-zero\noutput:\n%s", out)
|
||||
}
|
||||
builds := buildCalls(calls)
|
||||
if len(builds) != 4 {
|
||||
t.Fatalf("bazel build calls = %q, want the full build plus one per platform", builds)
|
||||
}
|
||||
for i, platform := range []string{"linux_amd64", "windows_arm64", "darwin_arm64"} {
|
||||
if !strings.Contains(builds[i+1], releaseFlag+platform+" ") {
|
||||
t.Errorf("attribution build %d = %q, want %s alone", i, builds[i+1], platform)
|
||||
}
|
||||
}
|
||||
for _, want := range []string{
|
||||
"::error::release cross-compilation failed for windows/arm64",
|
||||
"::error::release cross-compilation failed for: windows/arm64",
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Errorf("output does not contain %q:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
for _, unwanted := range []string{"failed for linux/amd64", "failed for darwin/arm64"} {
|
||||
if strings.Contains(out, unwanted) {
|
||||
t.Errorf("output blames a passing platform (%q):\n%s", unwanted, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A group name absent from the manifest must fail loudly, not silently
|
||||
// succeed with zero targets built.
|
||||
func TestCheckReleaseCrossCompileUnknownGroupFails(t *testing.T) {
|
||||
out, err, log := runCheckReleaseCrossCompile(t, "bogus-group", testManifest, "")
|
||||
// An empty manifest must fail loudly rather than pass with nothing built.
|
||||
func TestBazelReleaseCrossCompileEmptyManifestFails(t *testing.T) {
|
||||
out, err, calls := runBazelReleaseCrossCompile(t, "# nothing\n\n", "", testPkgs)
|
||||
if err == nil {
|
||||
t.Fatalf("script exited 0 for an unknown group; want non-zero\noutput:\n%s", out)
|
||||
t.Fatalf("script exited 0 for an empty manifest; want non-zero\noutput:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "::error::no release targets found for group 'bogus-group'") {
|
||||
t.Errorf("output does not report the unknown group:\n%s", out)
|
||||
if !strings.Contains(out, "::error::no release targets in") {
|
||||
t.Errorf("output does not report the empty manifest:\n%s", out)
|
||||
}
|
||||
if log != "" {
|
||||
t.Errorf("fake go was invoked for an unknown group; want zero invocations, got log:\n%s", log)
|
||||
if len(calls) != 0 {
|
||||
t.Errorf("bazel ran for an empty manifest: %q", calls)
|
||||
}
|
||||
}
|
||||
|
||||
// A malformed row (not exactly GOOS GOARCH) must fail, not be skipped.
|
||||
func TestBazelReleaseCrossCompileMalformedRowFails(t *testing.T) {
|
||||
out, err, _ := runBazelReleaseCrossCompile(t, "linux amd64 unix\n", "", testPkgs)
|
||||
if err == nil {
|
||||
t.Fatalf("script exited 0 for a malformed row; want non-zero\noutput:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "::error::malformed row") {
|
||||
t.Errorf("output does not report the malformed row:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
91
scripts/ci/bazel-release-cross-compile.sh
Executable file
91
scripts/ci/bazel-release-cross-compile.sh
Executable file
|
|
@ -0,0 +1,91 @@
|
|||
#!/usr/bin/env bash
|
||||
# Release-target cross-compilation gate (bazel.yml's pure-Go lane): build
|
||||
# every non-test Go target of the module for every GOOS/GOARCH row of
|
||||
# scripts/ci/release-targets.txt, the Bazel form of
|
||||
# `CGO_ENABLED=0 GOOS=<os> GOARCH=<arch> go build -tags gms_pure_go ./...`.
|
||||
#
|
||||
# One `bazel build` of //tools/bazel:release_cross with
|
||||
# --//tools/bazel:release_platforms set to every manifest row: its split
|
||||
# transition builds every listed target once per platform with cgo off (and
|
||||
# the gms_pure_go tag every build carries), all platforms in parallel on the
|
||||
# remote workers. --keep_going reports every broken target of every
|
||||
# platform in one run, and a failed run names each failing platform. The target list is every go_library and go_binary
|
||||
# (the packages `go build ./...` compiles; go_test targets are `go test`'s),
|
||||
# kept by tools/bazel/go_srcs.py (`make bazel-sync`) minus the ones tagged
|
||||
# cgo-only (incompatible with //tools/bazel:pure), which `go build ./...` skips
|
||||
# under CGO_ENABLED=0. Before building, the list is checked against `bazel
|
||||
# query`, so a target the generator missed fails here instead of going
|
||||
# uncompiled. The list also holds //tools/bazel:pure_bd_has_no_cgo_only_deps:
|
||||
# Bazel compiles gozstd's pure stubs (third_party/patches/
|
||||
# gozstd_nocgo.patch) where `go build` rejects a CGO_ENABLED=0 import of it,
|
||||
# so that check fails a platform whose pure bd links gozstd.
|
||||
#
|
||||
# Coverage boundary, as in the `go build` job this replaces: every target
|
||||
# builds with cgo off, but .goreleaser.yml builds linux/amd64, linux/arm64
|
||||
# and windows/amd64 with CGO_ENABLED=1, and release.yml builds the darwin
|
||||
# pair natively with CGO_ENABLED=1, so the cgo half of the tree (beads_cgo.go
|
||||
# -> internal/storage/embeddeddolt) is compiled for none of them here.
|
||||
#
|
||||
# Usage: bazel-release-cross-compile.sh [extra bazel build flags...]
|
||||
set -euo pipefail
|
||||
|
||||
manifest="$(dirname "${BASH_SOURCE[0]}")/release-targets.txt"
|
||||
|
||||
targets=()
|
||||
while read -r goos goarch extra; do
|
||||
if [[ -z "$goarch" || -n "$extra" ]]; then
|
||||
echo "::error::malformed row in $manifest: '$goos $goarch $extra', want 'GOOS GOARCH'"
|
||||
exit 1
|
||||
fi
|
||||
targets+=("$goos/$goarch")
|
||||
done < <(grep -v '^[[:space:]]*#' "$manifest" | grep -v '^[[:space:]]*$' || true)
|
||||
|
||||
if [[ "${#targets[@]}" -eq 0 ]]; then
|
||||
echo "::error::no release targets in $manifest"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
platforms=()
|
||||
for target in "${targets[@]}"; do
|
||||
platforms+=("${target%/*}_${target#*/}")
|
||||
done
|
||||
|
||||
# Every Bazel package holding a Go target (less testonly targets and the
|
||||
# packages of cgo-only ones) must be reached by //tools/bazel:release_cross;
|
||||
# a listed label that no longer exists fails the build itself.
|
||||
universe='kind("go_library|go_binary", //...)'
|
||||
wanted="$universe except siblings(attr(tags, \"\\bcgo-only\\b\", $universe)) except attr(testonly, 1, $universe)"
|
||||
# One query at a time: they share the Bazel server.
|
||||
want_pkgs="$(bazel query "$wanted" --output=package | LC_ALL=C sort -u)"
|
||||
built_pkgs="$(bazel query 'deps(//tools/bazel:release_cross)' --output=package | LC_ALL=C sort -u)"
|
||||
missing="$(LC_ALL=C comm -23 <(echo "$want_pkgs") <(echo "$built_pkgs"))"
|
||||
if [[ -n "$missing" ]]; then
|
||||
echo "::error::Go packages //tools/bazel:release_cross does not build (run \`make bazel-sync\`; a package whose Go targets are all invisible to //tools/bazel needs one visible to it):"
|
||||
echo "$missing"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
platform_list="$(IFS=,; echo "${platforms[*]}")"
|
||||
echo "::group::bazel build //tools/bazel:release_cross for ${targets[*]}"
|
||||
status=0
|
||||
bazel build --keep_going "--//tools/bazel:release_platforms=$platform_list" \
|
||||
"$@" -- //tools/bazel:release_cross || status=$?
|
||||
echo "::endgroup::"
|
||||
if [[ "$status" -ne 0 ]]; then
|
||||
# Bazel's errors above name the target, not the platform (each split
|
||||
# configuration's output directory is a hash). Rebuild one platform at a
|
||||
# time to name them: the same configurations, so everything that built
|
||||
# is a cache hit and only the failures run again.
|
||||
failed=()
|
||||
for i in "${!platforms[@]}"; do
|
||||
if ! bazel build --keep_going "--//tools/bazel:release_platforms=${platforms[$i]}" \
|
||||
"$@" -- //tools/bazel:release_cross >/dev/null 2>&1; then
|
||||
echo "::error::release cross-compilation failed for ${targets[$i]}"
|
||||
failed+=("${targets[$i]}")
|
||||
fi
|
||||
done
|
||||
echo "::error::release cross-compilation failed for: ${failed[*]:-(bazel exit $status; no single platform reproduced it, see the log above)}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "release cross-compilation passed: ${targets[*]}"
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
#!/usr/bin/env bash
|
||||
# Build the whole tree (not just ./cmd/bd) for every release target in the
|
||||
# given group of scripts/ci/release-targets.txt, with CGO disabled.
|
||||
#
|
||||
# This is the F7a fold of pr.yml's check-release-target-cross-compilation job:
|
||||
# one call per matrix leg (group), each building its targets sequentially in a
|
||||
# single job/runner instead of one runner per target. Every target in the
|
||||
# group runs even if an earlier one fails, so a PR touching two platforms at
|
||||
# once sees both failures in one log instead of needing a re-run per leg.
|
||||
#
|
||||
# Usage: check-release-cross-compile.sh <group>
|
||||
#
|
||||
# Review N-1 (2026-10-03): the spec sketched a 2-wide parallel build per leg
|
||||
# with a per-target GOCACHE. This builds sequentially with one shared GOCACHE
|
||||
# instead - simpler, and safe because the targets in a group never touch the
|
||||
# same GOCACHE entries destructively. Measured per-target time on an 8 vCPU
|
||||
# Blacksmith runner is a couple of minutes, off the gate's critical path; if a
|
||||
# future group's wall time regresses past ~4 minutes, revisit 2-wide.
|
||||
#
|
||||
# Coverage boundary, carried over from the pre-fold job: every target here
|
||||
# builds CGO_ENABLED=0, but .goreleaser.yml builds bd-linux-amd64,
|
||||
# bd-linux-arm64 and bd-windows-amd64 with CGO_ENABLED=1, and the darwin pair
|
||||
# is built natively at CGO_ENABLED=1 by release.yml's goreleaser-macos job.
|
||||
# Five of these eight targets are therefore a weaker proxy than the binary
|
||||
# they stand for: the cgo half of the tree (beads_cgo.go ->
|
||||
# internal/storage/embeddeddolt) is never compiled here, so a break confined
|
||||
# to it still passes this check. Only android/arm64, windows/arm64 and
|
||||
# freebsd/amd64 replicate their release configuration exactly.
|
||||
set -euo pipefail
|
||||
|
||||
group="${1:?usage: check-release-cross-compile.sh <group>}"
|
||||
manifest="$(dirname "${BASH_SOURCE[0]}")/release-targets.txt"
|
||||
|
||||
targets=()
|
||||
while read -r goos goarch row_group; do
|
||||
[[ "$row_group" == "$group" ]] || continue
|
||||
targets+=("$goos/$goarch")
|
||||
done < <(grep -v '^#' "$manifest" | grep -v '^[[:space:]]*$')
|
||||
|
||||
if [[ "${#targets[@]}" -eq 0 ]]; then
|
||||
echo "::error::no release targets found for group '$group' in $manifest"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
failed=()
|
||||
for target in "${targets[@]}"; do
|
||||
goos="${target%/*}"
|
||||
goarch="${target#*/}"
|
||||
echo "::group::Build $goos/$goarch"
|
||||
if CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" go build -tags gms_pure_go ./...; then
|
||||
echo "ok: $goos/$goarch"
|
||||
else
|
||||
echo "::error::cross-compilation failed for $goos/$goarch"
|
||||
failed+=("$goos/$goarch")
|
||||
fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
|
||||
if [[ "${#failed[@]}" -gt 0 ]]; then
|
||||
echo "::error::cross-compilation failed for: ${failed[*]}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "cross-compilation passed for group '$group': ${targets[*]}"
|
||||
|
|
@ -1,7 +1,6 @@
|
|||
# Release target manifest for pr.yml's check-release-target-cross-compilation
|
||||
# job (F7a). One "GOOS GOARCH GROUP" row per target; GROUP assigns the target
|
||||
# to one of the job's two matrix legs (see
|
||||
# scripts/ci/check-release-cross-compile.sh).
|
||||
# Release target manifest: one "GOOS GOARCH" row per shipped release target.
|
||||
# scripts/ci/bazel-release-cross-compile.sh (bazel.yml's pure-Go lane) builds
|
||||
# the whole tree for each row with Bazel (--platforms, cgo off).
|
||||
#
|
||||
# Kept in sync with .goreleaser.yml's builds: list, PLUS the darwin/amd64 and
|
||||
# darwin/arm64 pair that .goreleaser.yml deliberately omits (built instead by
|
||||
|
|
@ -11,15 +10,11 @@
|
|||
# TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser enforces this: it
|
||||
# fails if this file and .goreleaser.yml (plus the darwin exception) ever
|
||||
# disagree on the target set.
|
||||
#
|
||||
# Groups are an even split for balanced job runtime, not a semantic
|
||||
# boundary - cross-compilation never touches the target OS, so both groups
|
||||
# build and run entirely on the Linux runner.
|
||||
linux amd64 unix
|
||||
linux arm64 unix
|
||||
android arm64 unix
|
||||
freebsd amd64 unix
|
||||
darwin amd64 desktop
|
||||
darwin arm64 desktop
|
||||
windows amd64 desktop
|
||||
windows arm64 desktop
|
||||
linux amd64
|
||||
linux arm64
|
||||
android arm64
|
||||
freebsd amd64
|
||||
darwin amd64
|
||||
darwin arm64
|
||||
windows amd64
|
||||
windows arm64
|
||||
|
|
|
|||
|
|
@ -35,8 +35,7 @@ const sameRepoBlacksmith2vcpu = "${{ (github.event_name == 'merge_group' || (git
|
|||
|
||||
// F7a: the same same-repo expression at 4 vCPU and 8 vCPU, for jobs sized
|
||||
// larger than the 2 vCPU default (check-doc-flags,
|
||||
// pr-risk.yml's test-nix at 4 vCPU; check-release-target-cross-compilation at
|
||||
// 8 vCPU). F7c's advisory workflows also use the 4 vCPU size.
|
||||
// pr-risk.yml's test-nix at 4 vCPU; windows-test-binaries at 8 vCPU). F7c's advisory workflows also use the 4 vCPU size.
|
||||
const sameRepoBlacksmith4vcpu = "${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}"
|
||||
const sameRepoBlacksmith8vcpu = "${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}"
|
||||
|
||||
|
|
|
|||
|
|
@ -393,42 +393,63 @@ esac
|
|||
}
|
||||
}
|
||||
|
||||
// TestPRCIGateRequiresReleaseTargetCrossCompilation pins the cross-compilation
|
||||
// check into the gate. Wiring a job into ci-gate takes three separate edits --
|
||||
// needs:, the CI_GATE_REQUIRED token list, and the CHECK_* env mapping -- and
|
||||
// the gate silently ignores a token that is missing any one of them. Every
|
||||
// other load-bearing check in this file is pinned by name for that reason.
|
||||
func TestPRCIGateRequiresReleaseTargetCrossCompilation(t *testing.T) {
|
||||
// TestReleaseCrossCompileRunsInBazelPureLane pins the release-target
|
||||
// cross-compilation gate into bazel.yml's pure-Go lane, whose job.status
|
||||
// ci-gate requires (BAZEL_PURE), and keeps the retired pr.yml job from
|
||||
// coming back beside it: a second, `go build` copy of the gate would run
|
||||
// every target twice.
|
||||
func TestReleaseCrossCompileRunsInBazelPureLane(t *testing.T) {
|
||||
const (
|
||||
jobName = "check-release-target-cross-compilation"
|
||||
token = "CHECK_RELEASE_TARGET_CROSS_COMPILATION"
|
||||
retiredJob = "check-release-target-cross-compilation"
|
||||
retiredToken = "CHECK_RELEASE_TARGET_CROSS_COMPILATION"
|
||||
stepName = "Cross-compile every release target (--config=release-cross)"
|
||||
)
|
||||
|
||||
gate := readCIWorkflow(t, "pr.yml").job(t, "ci-gate")
|
||||
pr := readCIWorkflow(t, "pr.yml")
|
||||
if _, ok := pr.Jobs[retiredJob]; ok {
|
||||
t.Errorf("pr.yml still defines %s; bazel.yml's %s lane runs the release cross-compilation", retiredJob, bazelPureJobName)
|
||||
}
|
||||
gate := pr.job(t, "ci-gate")
|
||||
gateEnv := gate.step(t, "Evaluate CI gate").Env
|
||||
if contains(gate.Needs, retiredJob) || gateEnv[retiredToken] != "" || contains(strings.Fields(gateEnv["CI_GATE_REQUIRED"]), retiredToken) {
|
||||
t.Errorf("ci-gate still wires the retired %s job", retiredJob)
|
||||
}
|
||||
if !contains(strings.Fields(gateEnv["CI_GATE_REQUIRED"]), "BAZEL_PURE") {
|
||||
t.Errorf("ci-gate CI_GATE_REQUIRED does not include BAZEL_PURE, the lane that cross-compiles the release targets")
|
||||
}
|
||||
|
||||
if !contains(gate.Needs, jobName) {
|
||||
t.Errorf("ci-gate needs %q: %v", jobName, gate.Needs)
|
||||
run := readCIWorkflow(t, bazelWorkflowName).job(t, bazelPureJobName).step(t, stepName).Run
|
||||
for _, required := range []string{
|
||||
"set -euo pipefail",
|
||||
"./scripts/ci/bazel-release-cross-compile.sh",
|
||||
} {
|
||||
if !strings.Contains(run, required) {
|
||||
t.Errorf("%s step %q does not contain %q:\n%s", bazelPureJobName, stepName, required, run)
|
||||
}
|
||||
}
|
||||
if got, want := gateEnv[token], "${{ needs."+jobName+".result }}"; got != want {
|
||||
t.Errorf("ci-gate env %s = %q, want %q", token, got, want)
|
||||
}
|
||||
if !contains(strings.Fields(gateEnv["CI_GATE_REQUIRED"]), token) {
|
||||
t.Errorf("ci-gate CI_GATE_REQUIRED does not include %q", token)
|
||||
|
||||
// The transition is what makes every platform's build pure Go (cgo off,
|
||||
// as CGO_ENABLED=0 go build) for the platform the script names.
|
||||
bzl := readPolicyFile(t, bazelPolicyRoot(t), "tools/bazel/release_cross.bzl")
|
||||
for _, required := range []string{
|
||||
`"//command_line_option:platforms": ["@rules_go//go/toolchain:" + platform],`,
|
||||
`"@rules_go//go/config:pure": True,`,
|
||||
`inputs = ["//tools/bazel:release_platforms"],`,
|
||||
} {
|
||||
if !strings.Contains(bzl, required) {
|
||||
t.Errorf("tools/bazel/release_cross.bzl does not contain %q", required)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser keeps
|
||||
// scripts/ci/release-targets.txt (F7a: the single source of truth for the
|
||||
// pr.yml cross-compilation job's two matrix legs, read by
|
||||
// scripts/ci/check-release-cross-compile.sh) and the set of shipped release
|
||||
// targets in lockstep. Without a guard a newly added release target -- the
|
||||
// way freebsd/amd64 once was -- is silently uncovered while a green "release
|
||||
// target cross-compilation" check still stands. That is worse than having no
|
||||
// check at all, because the check's existence implies the coverage it has
|
||||
// quietly lost.
|
||||
// scripts/ci/release-targets.txt (the single source of truth for the
|
||||
// platforms scripts/ci/bazel-release-cross-compile.sh builds in bazel.yml's
|
||||
// pure-Go lane) and the set of shipped release targets in lockstep. Without
|
||||
// a guard a newly added release target -- the way freebsd/amd64 once was --
|
||||
// is silently uncovered while a green cross-compilation check still stands.
|
||||
// That is worse than having no check at all, because the check's existence
|
||||
// implies the coverage it has quietly lost.
|
||||
func TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser(t *testing.T) {
|
||||
const jobName = "check-release-target-cross-compilation"
|
||||
const manifestPath = "scripts/ci/release-targets.txt"
|
||||
|
||||
// darwin/amd64 and darwin/arm64 are shipped release targets that are
|
||||
|
|
@ -447,51 +468,21 @@ func TestReleaseTargetCrossCompilationMatrixMatchesGoreleaser(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// The job itself must still read the manifest, so a future rewrite of the
|
||||
// job that stops threading matrix.group through to the script cannot pass
|
||||
// silently.
|
||||
job := readCIWorkflow(t, "pr.yml").job(t, jobName)
|
||||
groups := append([]string(nil), job.Strategy.Matrix.Group...)
|
||||
sort.Strings(groups)
|
||||
if !reflect.DeepEqual(groups, []string{"desktop", "unix"}) {
|
||||
t.Errorf("%s matrix groups = %v, want [desktop unix]", jobName, groups)
|
||||
}
|
||||
var ranScript bool
|
||||
for _, step := range job.Steps {
|
||||
if strings.Contains(step.Run, "check-release-cross-compile.sh ${{ matrix.group }}") {
|
||||
ranScript = true
|
||||
}
|
||||
}
|
||||
if !ranScript {
|
||||
t.Errorf("%s does not run check-release-cross-compile.sh with matrix.group", jobName)
|
||||
}
|
||||
|
||||
raw, err := os.ReadFile(filepath.Join(sourceRepoRoot(t), manifestPath))
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", manifestPath, err)
|
||||
}
|
||||
got := make(map[string]bool)
|
||||
seenGroups := make(map[string]bool)
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) != 3 {
|
||||
t.Fatalf("%s: malformed row %q, want 'GOOS GOARCH GROUP'", manifestPath, line)
|
||||
}
|
||||
goos, goarch, group := fields[0], fields[1], fields[2]
|
||||
if group != "unix" && group != "desktop" {
|
||||
t.Fatalf("%s: row %q has unknown group %q", manifestPath, line, group)
|
||||
}
|
||||
seenGroups[group] = true
|
||||
got[goos+"/"+goarch] = true
|
||||
}
|
||||
for _, g := range []string{"unix", "desktop"} {
|
||||
if !seenGroups[g] {
|
||||
t.Errorf("%s: no targets in group %q", manifestPath, g)
|
||||
if len(fields) != 2 {
|
||||
t.Fatalf("%s: malformed row %q, want 'GOOS GOARCH'", manifestPath, line)
|
||||
}
|
||||
got[fields[0]+"/"+fields[1]] = true
|
||||
}
|
||||
|
||||
for target, source := range want {
|
||||
|
|
@ -1421,11 +1412,7 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "test-windows-liveness"), []goCacheStep{
|
||||
restoreModuleCache(), restoreBuildCache("non-race"),
|
||||
})
|
||||
for _, jobName := range []string{
|
||||
"advisory-reports", "check-release-target-cross-compilation",
|
||||
} {
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, jobName), []goCacheStep{restoreModuleCache()})
|
||||
}
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "advisory-reports"), []goCacheStep{restoreModuleCache()})
|
||||
// F7b review (S2): check-doc-freshness-platforms additionally restores a
|
||||
// restore-only non-race Go build cache, seeded by the same push-to-main
|
||||
// savers pr-preflight-platforms' leg restores below (see that job's own
|
||||
|
|
@ -1462,7 +1449,7 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
"pr.yml": {
|
||||
"build-artifacts": true, "pr-core-wrapper": true, "scripts-go-checks": true, "worktree-remove-windows": true,
|
||||
"check-doc-freshness-platforms": true, "pr-preflight-platforms": true, "advisory-reports": true,
|
||||
"check-release-target-cross-compilation": true, "pr-lint-wrapper": true, "windows-test-binaries": true,
|
||||
"pr-lint-wrapper": true, "windows-test-binaries": true,
|
||||
"test-windows-liveness": true,
|
||||
},
|
||||
"pr-risk.yml": {"build-embedded": true},
|
||||
|
|
@ -2987,15 +2974,13 @@ func TestSameRepoBlacksmithRunners(t *testing.T) {
|
|||
// size (module-cache misses on the Blacksmith pool are acceptable
|
||||
// per spec; jobs that need a Blacksmith-side build-cache saver are
|
||||
// F7b's scope, not this one). windows-test-binaries is F4's Linux
|
||||
// mingw cross-compile job, sized at 8vcpu like the cross-compilation
|
||||
// matrix fold below.
|
||||
// mingw cross-compile job, sized at 8vcpu.
|
||||
"pr.yml": {
|
||||
"bazel-coverage": sameRepoBlacksmith2vcpu, "ci-gate": sameRepoBlacksmith2vcpu,
|
||||
"fast-checks": sameRepoBlacksmith2vcpu,
|
||||
"advisory-reports": sameRepoBlacksmith2vcpu,
|
||||
"check-doc-flags": sameRepoBlacksmith4vcpu,
|
||||
"check-release-target-cross-compilation": sameRepoBlacksmith8vcpu,
|
||||
"windows-test-binaries": sameRepoBlacksmith8vcpu,
|
||||
"fast-checks": sameRepoBlacksmith2vcpu,
|
||||
"advisory-reports": sameRepoBlacksmith2vcpu,
|
||||
"check-doc-flags": sameRepoBlacksmith4vcpu,
|
||||
"windows-test-binaries": sameRepoBlacksmith8vcpu,
|
||||
// F7b: scripts-go-checks/pr-lint-wrapper moved to same-repo
|
||||
// Blacksmith now that main.yml's blacksmith-go-build-cache and
|
||||
// pr-lint-wrapper `venue` matrix seed their Blacksmith-selection
|
||||
|
|
|
|||
|
|
@ -8,7 +8,12 @@ go_library(
|
|||
"resolver_windows.go",
|
||||
],
|
||||
importpath = "github.com/steveyegge/beads/scripts/internal/testbash",
|
||||
visibility = ["//scripts:__subpackages__"],
|
||||
# //tools/bazel:release_cross cross-compiles it like every package
|
||||
# `go build ./...` compiles.
|
||||
visibility = [
|
||||
"//scripts:__subpackages__",
|
||||
"//tools/bazel:__pkg__",
|
||||
],
|
||||
)
|
||||
|
||||
go_test(
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
load("@rules_shell//shell:sh_binary.bzl", "sh_binary")
|
||||
load("@rules_shell//shell:sh_test.bzl", "sh_test")
|
||||
load(":dolt.bzl", "hermetic_bin")
|
||||
load(":release_cross.bzl", "release_cross_build", "string_list_flag")
|
||||
|
||||
# Platforms with a pinned dolt release; elsewhere (e.g. windows) the hermetic
|
||||
# test wrapper is skipped by `bazel build //...` instead of failing analysis.
|
||||
|
|
@ -99,6 +100,207 @@ filegroup(
|
|||
for cpu in ("amd64", "arm64")
|
||||
]
|
||||
|
||||
# On when rules_go builds pure Go (--@rules_go//go/config:pure, as
|
||||
# --config=pure and :release_cross's transition set it): cgo is off.
|
||||
config_setting(
|
||||
name = "pure",
|
||||
flag_values = {"@rules_go//go/config:pure": "true"},
|
||||
visibility = ["//:__subpackages__"],
|
||||
)
|
||||
|
||||
# Bazel compiles gozstd's pure stubs (third_party/patches/gozstd_nocgo.patch)
|
||||
# where `go build` with CGO_ENABLED=0 rejects any import of it, so a pure bd
|
||||
# that links gozstd still builds. Go binaries keep their function names
|
||||
# (pclntab) even when stripped, so this fails the build of a pure bd that
|
||||
# links gozstd for whatever platform it targets, without running it:
|
||||
# :release_cross builds it for every release platform. "main.main" proves the name table
|
||||
# was readable. Pure builds only: a cgo bd links gozstd by design.
|
||||
genrule(
|
||||
name = "pure_bd_has_no_cgo_only_deps",
|
||||
srcs = ["//cmd/bd"],
|
||||
outs = ["pure_bd_has_no_cgo_only_deps.txt"],
|
||||
cmd = """
|
||||
if grep -qaF 'github.com/dolthub/gozstd.' $<; then
|
||||
echo "$< links github.com/dolthub/gozstd (cgo-only) in a pure build; CGO_ENABLED=0 go build would reject it" >&2
|
||||
exit 1
|
||||
fi
|
||||
grep -qaF 'main.main' $< || { echo "$<: no Go function names found" >&2; exit 1; }
|
||||
echo "no cgo-only deps: $<" > $@
|
||||
""",
|
||||
target_compatible_with = select({
|
||||
":pure": [],
|
||||
"//conditions:default": ["@platforms//:incompatible"],
|
||||
}),
|
||||
)
|
||||
|
||||
# The rules_go platforms (<os>_<arch>) :release_cross builds for.
|
||||
# scripts/ci/bazel-release-cross-compile.sh sets it from
|
||||
# scripts/ci/release-targets.txt, the single source of truth; empty fails
|
||||
# :release_cross's analysis.
|
||||
string_list_flag(
|
||||
name = "release_platforms",
|
||||
build_setting_default = [],
|
||||
)
|
||||
|
||||
# :release_cross (the managed block below) is the release-target
|
||||
# cross-compilation: every go_library and go_binary, cgo off, once per
|
||||
# platform in :release_platforms, all in one invocation. It is the Bazel
|
||||
# form of `CGO_ENABLED=0 GOOS=<os> GOARCH=<arch> go build -tags gms_pure_go
|
||||
# ./...` per release target, plus :pure_bd_has_no_cgo_only_deps. Manual: it
|
||||
# needs :release_platforms, which `bazel build //...` does not set.
|
||||
|
||||
# --- begin release_cross (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
|
||||
|
||||
# manual: needs --//tools/bazel:release_platforms (see above).
|
||||
release_cross_build(
|
||||
name = "release_cross",
|
||||
tags = ["manual"],
|
||||
targets = [
|
||||
":pure_bd_has_no_cgo_only_deps",
|
||||
"//:beads",
|
||||
"//backend/conformance:conformance",
|
||||
"//backend:backend",
|
||||
"//beadserrors:beadserrors",
|
||||
"//cmd/bd/doctor/fix:fix",
|
||||
"//cmd/bd/doctor:doctor",
|
||||
"//cmd/bd/protocol:protocol",
|
||||
"//cmd/bd/setup:setup",
|
||||
"//cmd/bd:bd",
|
||||
"//format:format",
|
||||
"//internal/ado:ado",
|
||||
"//internal/atomicfile:atomicfile",
|
||||
"//internal/audit:audit",
|
||||
"//internal/beads:beads",
|
||||
"//internal/ceiling:ceiling",
|
||||
"//internal/compact:compact",
|
||||
"//internal/config:config",
|
||||
"//internal/configfile:configfile",
|
||||
"//internal/creds:creds",
|
||||
"//internal/debug:debug",
|
||||
"//internal/doltremote:doltremote",
|
||||
"//internal/doltserver:doltserver",
|
||||
"//internal/doltversion:doltversion",
|
||||
"//internal/eventsjournal:eventsjournal",
|
||||
"//internal/execenv:execenv",
|
||||
"//internal/fdhygiene:fdhygiene",
|
||||
"//internal/formula/cmd/schemagen:schemagen",
|
||||
"//internal/formula/schemagen:schemagen",
|
||||
"//internal/formula:formula",
|
||||
"//internal/git:git",
|
||||
"//internal/gitenv:gitenv",
|
||||
"//internal/githooksenv:githooksenv",
|
||||
"//internal/github:github",
|
||||
"//internal/gitignore:gitignore",
|
||||
"//internal/gitlab:gitlab",
|
||||
"//internal/gittraceenv:gittraceenv",
|
||||
"//internal/hooks:hooks",
|
||||
"//internal/httpapi/apigen:apigen",
|
||||
"//internal/httpapi/spec:spec",
|
||||
"//internal/httpapi/wireshape/cmd/gendigest:gendigest",
|
||||
"//internal/httpapi/wireshape:wireshape",
|
||||
"//internal/httpapi:httpapi",
|
||||
"//internal/httpclient/encode:encode",
|
||||
"//internal/httpclient/wire:wire",
|
||||
"//internal/httpclient:httpclient",
|
||||
"//internal/idgen:idgen",
|
||||
"//internal/jira:jira",
|
||||
"//internal/linear:linear",
|
||||
"//internal/lockfile:lockfile",
|
||||
"//internal/memoryapi:memoryapi",
|
||||
"//internal/metrics:metrics",
|
||||
"//internal/migration/legacysqlite:legacysqlite",
|
||||
"//internal/migration:migration",
|
||||
"//internal/molecules:molecules",
|
||||
"//internal/notion:notion",
|
||||
"//internal/procid:procid",
|
||||
"//internal/query:query",
|
||||
"//internal/recipes:recipes",
|
||||
"//internal/remotecache:remotecache",
|
||||
"//internal/routing:routing",
|
||||
"//internal/storage/backendnames:backendnames",
|
||||
"//internal/storage/backends:backends",
|
||||
"//internal/storage/batchbench:batchbench",
|
||||
"//internal/storage/batchfixtures:batchfixtures",
|
||||
"//internal/storage/contextinfo:contextinfo",
|
||||
"//internal/storage/createbatchequiv:createbatchequiv",
|
||||
"//internal/storage/dberrors:dberrors",
|
||||
"//internal/storage/dbproxy/identity:identity",
|
||||
"//internal/storage/dbproxy/pidfile:pidfile",
|
||||
"//internal/storage/dbproxy/proxy:proxy",
|
||||
"//internal/storage/dbproxy/server:server",
|
||||
"//internal/storage/dbproxy/util:util",
|
||||
"//internal/storage/depid:depid",
|
||||
"//internal/storage/dolt:dolt",
|
||||
"//internal/storage/doltutil:doltutil",
|
||||
"//internal/storage/domain/db:db",
|
||||
"//internal/storage/domain/fs:fs",
|
||||
"//internal/storage/domain/git:git",
|
||||
"//internal/storage/domain:domain",
|
||||
"//internal/storage/embeddeddolt:embeddeddolt",
|
||||
"//internal/storage/externaldeps:externaldeps",
|
||||
"//internal/storage/fs:fs",
|
||||
"//internal/storage/git:git",
|
||||
"//internal/storage/issueops:issueops",
|
||||
"//internal/storage/journalscan:journalscan",
|
||||
"//internal/storage/kvkeys:kvkeys",
|
||||
"//internal/storage/memoryops:memoryops",
|
||||
"//internal/storage/rowid:rowid",
|
||||
"//internal/storage/schema:schema",
|
||||
"//internal/storage/sqlbuild:sqlbuild",
|
||||
"//internal/storage/sqlclass:sqlclass",
|
||||
"//internal/storage/sqlcount:sqlcount",
|
||||
"//internal/storage/storagecontract:storagecontract",
|
||||
"//internal/storage/uow:uow",
|
||||
"//internal/storage/versioncontrolops:versioncontrolops",
|
||||
"//internal/storage:storage",
|
||||
"//internal/telemetry:telemetry",
|
||||
"//internal/templates/agents:agents",
|
||||
"//internal/templates:templates",
|
||||
"//internal/testutil/bazeltest:bazeltest",
|
||||
"//internal/testutil/credentialcmd:credentialcmd",
|
||||
"//internal/testutil/fixtures:fixtures",
|
||||
"//internal/testutil/integration:integration",
|
||||
"//internal/testutil:testutil",
|
||||
"//internal/timeparsing:timeparsing",
|
||||
"//internal/tracker/conformance:conformance",
|
||||
"//internal/tracker:tracker",
|
||||
"//internal/types:types",
|
||||
"//internal/ui:ui",
|
||||
"//internal/uimd:uimd",
|
||||
"//internal/utils:utils",
|
||||
"//internal/validation:validation",
|
||||
"//internal/versioncheck:versioncheck",
|
||||
"//internal/workapi/storecounter:storecounter",
|
||||
"//internal/workapi/storequerier:storequerier",
|
||||
"//internal/workapi/storereader:storereader",
|
||||
"//internal/workapi/storereadycounter:storereadycounter",
|
||||
"//internal/workapi/storestats:storestats",
|
||||
"//internal/workapi/storeversionreconciler:storeversionreconciler",
|
||||
"//internal/workapi/storeworkspaceconfig:storeworkspaceconfig",
|
||||
"//internal/workapi:workapi",
|
||||
"//internal/workspacegate:workspacegate",
|
||||
"//internal/worktreeremove:worktreeremove",
|
||||
"//issueops:issueops",
|
||||
"//journalops:journalops",
|
||||
"//memoryops:memoryops",
|
||||
"//plugins/beads:beads",
|
||||
"//schema:schema",
|
||||
"//scripts/bench-ready-indexes:bench-ready-indexes",
|
||||
"//scripts/check-versions:check-versions",
|
||||
"//scripts/checkworkflowtags:checkworkflowtags",
|
||||
"//scripts/internal/testbash:testbash",
|
||||
"//scripts/migration-test/catalog:catalog",
|
||||
"//scripts/pr-lint:pr-lint",
|
||||
"//scripts/repro-dolt-hang:repro-dolt-hang",
|
||||
"//scripts/repro-dolt-prod-timeouts:repro-dolt-prod-timeouts",
|
||||
"//scripts/workflowtags:workflowtags",
|
||||
"//test/conformance:conformance",
|
||||
"//tools/docsmint:docsmint",
|
||||
],
|
||||
)
|
||||
|
||||
# --- end release_cross ---
|
||||
|
||||
# --- begin repo_files (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---
|
||||
|
||||
filegroup(
|
||||
|
|
|
|||
|
|
@ -32,6 +32,17 @@ sync step, on every PR) fails on a missing or stale block. Trees in
|
|||
.bazelignore (.beads, website, the nested example modules, agent worktrees,
|
||||
node_modules) are outside Bazel and so outside //:repo_files.
|
||||
|
||||
tools/bazel/BUILD.bazel also gets a managed `release_cross` block: the
|
||||
release_cross_build (tools/bazel/release_cross.bzl) that
|
||||
scripts/ci/bazel-release-cross-compile.sh builds for every release platform,
|
||||
listing every go_library and go_binary it can see (the packages `go build
|
||||
./...` compiles; a private library is compiled by the go_binary that embeds
|
||||
it) except the cgo-only ones a pure build cannot link (tagged "cgo-only" and
|
||||
incompatible with //tools/bazel:pure) and testonly fixtures. The script checks
|
||||
against `bazel query` before it builds that every Bazel package holding Go
|
||||
targets is reached, so a package this parser misses fails CI instead of
|
||||
going uncompiled.
|
||||
|
||||
With --check nothing is written: stale blocks are printed as a diff and the
|
||||
exit status is 1 (see `make bazel-sync-check`).
|
||||
"""
|
||||
|
|
@ -65,11 +76,24 @@ TREES = (
|
|||
|
||||
BEGIN = "# --- begin go_srcs (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---"
|
||||
END = "# --- end go_srcs ---"
|
||||
RELEASE_BEGIN = "# --- begin release_cross (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---"
|
||||
RELEASE_END = "# --- end release_cross ---"
|
||||
RELEASE_PKG = "tools/bazel"
|
||||
# Built in every release configuration besides the Go targets.
|
||||
RELEASE_EXTRA_TARGETS = (":pure_bd_has_no_cgo_only_deps",)
|
||||
# Visibilities that let //tools/bazel:release_cross depend on a target.
|
||||
RELEASE_VISIBLE = ('"//visibility:public"', '"//:__subpackages__"', '"//tools/bazel:__pkg__"')
|
||||
# A buildifier-formatted go_library/go_binary call and its name.
|
||||
GO_RULE_RE = re.compile(r'^(go_library|go_binary)\(\n name = "([^"]+)",\n(.*?)^\)', re.DOTALL | re.MULTILINE)
|
||||
# How a cgo-only target opts out of pure builds (see
|
||||
# internal/storage/embeddeddolt/cmd/BUILD.bazel and the "cgo-only" tag in
|
||||
# scripts/bazel_policy_test.go).
|
||||
CGO_ONLY_TAG = '"cgo-only"'
|
||||
REPO_BEGIN = "# --- begin repo_files (managed by tools/bazel/go_srcs.py; run `make bazel-sync`) ---"
|
||||
REPO_END = "# --- end repo_files ---"
|
||||
BLOCK_RES = tuple(
|
||||
re.compile(r"\n*" + re.escape(begin) + r".*?" + re.escape(end) + r"\n*", re.DOTALL)
|
||||
for begin, end in ((BEGIN, END), (REPO_BEGIN, REPO_END))
|
||||
for begin, end in ((BEGIN, END), (RELEASE_BEGIN, RELEASE_END), (REPO_BEGIN, REPO_END))
|
||||
)
|
||||
SKIP_DIRS = {"testdata", "node_modules"}
|
||||
|
||||
|
|
@ -206,6 +230,43 @@ def repo_files_block(pkg: str, packages: list[str]) -> str:
|
|||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def go_targets(packages: list[str]) -> list[str]:
|
||||
"""Every go_library and go_binary that a pure build can compile."""
|
||||
labels = []
|
||||
for pkg in packages:
|
||||
with open(os.path.join(pkg, "BUILD.bazel")) as f:
|
||||
src = f.read()
|
||||
for kind, name, body in GO_RULE_RE.findall(src):
|
||||
# cgo-only: `go build ./...` skips it with CGO_ENABLED=0.
|
||||
# testonly: a fixture under testdata/, which `./...` excludes.
|
||||
# Not visible here: a main package's private embedded library
|
||||
# or a package-restricted helper; its package is covered by a
|
||||
# visible target (or the script's package check fails).
|
||||
if CGO_ONLY_TAG in body or "testonly = True" in body:
|
||||
continue
|
||||
vis = re.search(r"^ visibility = \[(.*?)\]", body, re.DOTALL | re.MULTILINE)
|
||||
if not vis or not any(v in vis.group(1) for v in RELEASE_VISIBLE):
|
||||
continue
|
||||
path = "" if pkg == "." else pkg
|
||||
labels.append(f"//{path}:{name}")
|
||||
return sorted(labels)
|
||||
|
||||
|
||||
def release_cross_block(targets: list[str]) -> str:
|
||||
lines = [
|
||||
RELEASE_BEGIN,
|
||||
"",
|
||||
"# manual: needs --//tools/bazel:release_platforms (see above).",
|
||||
"release_cross_build(",
|
||||
' name = "release_cross",',
|
||||
' tags = ["manual"],',
|
||||
" targets = [",
|
||||
]
|
||||
lines += [f' "{t}",' for t in sorted(RELEASE_EXTRA_TARGETS) + targets]
|
||||
lines += [" ],", ")", "", RELEASE_END]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def rewrite(path: str, new_blocks: list[str], check: bool) -> bool:
|
||||
"""Bring path's managed blocks up to date; return True if any was stale.
|
||||
|
||||
|
|
@ -266,6 +327,8 @@ def main(argv: list[str]) -> int:
|
|||
for pkg in packages:
|
||||
path = os.path.join(pkg, "BUILD.bazel")
|
||||
blocks = [block(pkg, wanted[pkg])] if pkg in wanted else []
|
||||
if pkg == RELEASE_PKG:
|
||||
blocks.append(release_cross_block(go_targets(packages)))
|
||||
blocks.append(repo_files_block(pkg, packages))
|
||||
changed = rewrite(path, blocks, check)
|
||||
if changed:
|
||||
|
|
|
|||
55
tools/bazel/release_cross.bzl
Normal file
55
tools/bazel/release_cross.bzl
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
"""Build Go targets for every release platform in one Bazel invocation.
|
||||
|
||||
scripts/ci/bazel-release-cross-compile.sh passes the platforms of
|
||||
scripts/ci/release-targets.txt as --//tools/bazel:release_platforms; the
|
||||
split transition below then configures every target once per platform, with
|
||||
pure Go (cgo off), so all platforms build in parallel on the remote workers
|
||||
instead of one `bazel build --platforms=...` after another.
|
||||
"""
|
||||
|
||||
def _release_platforms_transition_impl(settings, _attr):
|
||||
platforms = settings["//tools/bazel:release_platforms"]
|
||||
if not platforms:
|
||||
fail("--//tools/bazel:release_platforms is empty; scripts/ci/bazel-release-cross-compile.sh sets it from scripts/ci/release-targets.txt")
|
||||
return {
|
||||
platform: {
|
||||
"//command_line_option:platforms": ["@rules_go//go/toolchain:" + platform],
|
||||
"@rules_go//go/config:pure": True,
|
||||
}
|
||||
for platform in platforms
|
||||
}
|
||||
|
||||
_release_platforms_transition = transition(
|
||||
implementation = _release_platforms_transition_impl,
|
||||
inputs = ["//tools/bazel:release_platforms"],
|
||||
outputs = [
|
||||
"//command_line_option:platforms",
|
||||
"@rules_go//go/config:pure",
|
||||
],
|
||||
)
|
||||
|
||||
def _release_cross_build_impl(ctx):
|
||||
return [DefaultInfo(files = depset(transitive = [
|
||||
dep[DefaultInfo].files
|
||||
for deps in ctx.split_attr.targets.values()
|
||||
for dep in deps
|
||||
]))]
|
||||
|
||||
release_cross_build = rule(
|
||||
implementation = _release_cross_build_impl,
|
||||
doc = "Builds `targets` for every rules_go platform (<os>_<arch>) in --//tools/bazel:release_platforms, with cgo off.",
|
||||
attrs = {
|
||||
"targets": attr.label_list(
|
||||
cfg = _release_platforms_transition,
|
||||
mandatory = True,
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
def _string_list_flag_impl(_ctx):
|
||||
return []
|
||||
|
||||
string_list_flag = rule(
|
||||
implementation = _string_list_flag_impl,
|
||||
build_setting = config.string_list(flag = True),
|
||||
)
|
||||
Loading…
Add table
Add a link
Reference in a new issue