fix(bazel): one nogo-validating lane per race configuration (F5 S1) (#7479)
* fix(bazel): one nogo-validating lane per race configuration (F5 S1) The race configuration (--@rules_go//go/config:race) is built by five lanes that duplicate the same RunNogo/ValidateNogo work: bazel-test (--config=ci), bazel-embedded, bazel-doltserver, bazel-proxied, and the advisory bazel-dolt-race shadow, plus the package-mcp/package-npm gates' explicit race build. --norun_validations is a build-request option, not a configuration flag (proven here by aquery: GoCompilePkg/ GoLink/RunNogo/ValidateNogo action keys are byte-identical with and without it), so passing it on every lane but bazel-test removes the duplicate work without touching a single action key or dropping any lint coverage: bazel-test's --config=ci already validates every file any of the others compiles. scripts/nogo_lint_policy_test.go replaces the old blanket "--norun_validations never appears in .bazelrc" check with TestNogoConfigurationsHaveOneValidatingLane (T1: one owner per configuration fingerprint), TestNogoOwnersAreRequired (T2: the owner is a required CI Gate check whose `if:` covers every non-owner's), TestNogoRaceOwnerBuildsEverything (T3: the owner builds //... whole, no exclusions) and TestNoUnconditionalValidationOff (T5: nothing else may disable validation). tools/bazel/nogo_ownership.py (make nogo-ownership) is a dynamic, advisory aquery cross-check of the same property, run on demand rather than from inside a Bazel action. Verified locally: aquery shows identical action keys for the affected lanes with/without --norun_validations; a planted printf-format vet finding in internal/idgen/hash.go failed the owner build and passed the non-owner build (then reverted, never committed); scripts/bazel_dolt_race_shadow_test.go and the full //scripts suite stay green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(bazel): close S1 nogo-ownership review gaps (allowlist, owner pin, real aquery) Addresses maintainer-city-c3's review of #7479: 1. T6 TestRunValidationsOnlyOnAllowlistedLines replaces the coverage gap left when main's blanket run_validations ban was dropped: only the four race-group non-owners' "test:<cfg> --norun_validations" lines in .bazelrc, and the two package-gate bazel-build steps in bazel.yml, are permitted. A later test:integration/doltserver-cmd --norun_validations (out of T1's table, S2) now fails here instead of landing silently. Verified by mutation: adding test:integration --norun_validations fails the new test; reverting passes. 3. TestNogoRaceOwnerBuildsEverything (T3) now also pins that the race owner's "bazel test //... --config=ci" step has no step-level if: or continue-on-error, and that its job has none either. Verified by mutation: adding if: ${{ always() }} to that step fails; reverting passes. 2. tools/bazel/nogo_ownership.py's aquery comparison was querying the identical mnemonic("RunNogo", //...) for the owner and every non-owner, so `missing` could never be non-empty. It now derives each non-owner's actual built target set the way --build_tests_only + --test_tag_filters does: the deps closure of tests(//...) filtered to targets carrying one of the lane's tags, read exactly from `query --output=xml` (not attr()'s substring regex match, which would wrongly fold "dolt-server-proxied" and "dolt-server-cmd" into a "dolt-server" filter). Confirmed against a real (offline, --nohome_rc) bazel run: reports OK at head, and reports 100+ missing targets when the owner's query is artificially narrowed to a subpackage -- it can now actually fail. Also: refreshed a stale .bazelrc comment claiming nogo runs beside every lane's compile, which S1 made untrue for race-group non-owners. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
c1ccb728ec
commit
b8e6ba1791
10 changed files with 753 additions and 33 deletions
34
.bazelrc
34
.bazelrc
|
|
@ -104,6 +104,16 @@ test:embedded --remote_download_regex=.*/test\.xml$
|
|||
# jobs, which never retried. Evicted inputs fail the step instead; a
|
||||
# visible "Re-run failed jobs" retries it.
|
||||
test:embedded --experimental_remote_cache_eviction_retries=0
|
||||
# The race configuration's nogo validations run once, in bazel.yml's `test`
|
||||
# lane (`bazel test //... --config=ci`), which is a strict superset of every
|
||||
# race-group lane's RunNogo/ValidateNogo keys. --norun_validations is a
|
||||
# build-request option, not a configuration flag: it changes no action key
|
||||
# (compiles, links and test PASSes are shared with `test` through the
|
||||
# action cache) and discards no analysis; it only skips re-running nogo's
|
||||
# validation actions that the owner lane already runs on this configuration.
|
||||
# scripts/nogo_lint_policy_test.go's TestNogoConfigurationsHaveOneValidatingLane
|
||||
# pins `test` as this configuration's sole validating lane.
|
||||
test:embedded --norun_validations
|
||||
|
||||
# Dolt-server lane: what pr.yml's retired container-backed jobs ("Test
|
||||
# (storage domain + uow)" and "Contract corpus") ran, but every dolt
|
||||
|
|
@ -127,6 +137,9 @@ test:doltserver --test_arg=-test.parallel=4
|
|||
# runner shape; remote workers are not limited by it.
|
||||
test:doltserver --local_test_jobs=4
|
||||
test:doltserver --remote_download_regex=.*/test\.(log|xml)$
|
||||
# Race-group nogo is owned by the `test` lane; see test:embedded's comment
|
||||
# above.
|
||||
test:doltserver --norun_validations
|
||||
|
||||
# Proxied-server tier: PR Risk's "Test (Proxied Dolt Cmd N/15)" jobs, as
|
||||
# //cmd/bd:bd_proxied_test and //cmd/bd:bd_managed_local_test (the
|
||||
|
|
@ -160,6 +173,9 @@ test:doltserver-proxied --remote_download_regex=.*/test\.(log|xml)$
|
|||
# jobs, which never retried. Evicted inputs fail the step instead; a
|
||||
# visible "Re-run failed jobs" retries it.
|
||||
test:doltserver-proxied --experimental_remote_cache_eviction_retries=0
|
||||
# Race-group nogo is owned by the `test` lane; see test:embedded's comment
|
||||
# above.
|
||||
test:doltserver-proxied --norun_validations
|
||||
|
||||
# dolt-server + proxied union (item 5, Phase 1 shadow): bazel.yml's advisory
|
||||
# bazel-dolt-race job runs both tiers in one invocation, beside their own
|
||||
|
|
@ -186,6 +202,10 @@ test:dolt-race --test_arg=-test.parallel=4
|
|||
test:dolt-race --local_test_jobs=4
|
||||
test:dolt-race --remote_download_regex=.*/test\.(log|xml)$
|
||||
test:dolt-race --experimental_remote_cache_eviction_retries=0
|
||||
# The union carries its members' --norun_validations line
|
||||
# (TestBazelDoltRaceConfigIsMemberUnion needs no change: both members and the
|
||||
# union carry the same line).
|
||||
test:dolt-race --norun_validations
|
||||
|
||||
# Server-Dolt storage tier: PR Risk's "Test (Server Dolt Conformance)" and
|
||||
# "Test (Server Dolt Full Suite N/16)" jobs, as
|
||||
|
|
@ -320,11 +340,15 @@ build:js-wasm --use_target_platform_for_tests
|
|||
build:js-wasm --remote_download_outputs=toplevel
|
||||
|
||||
# nogo (//tools/nogo: go test's vet checks plus the golangci-lint linters
|
||||
# .golangci.yml enables) runs beside every Go compile of every lane, so any
|
||||
# `bazel build`/`bazel test` fails on a finding. --config=nogo asks for the
|
||||
# analysis alone (the nogo_fix output group; nothing is linked), in the race
|
||||
# configuration bazel.yml's unit lane (test:ci) builds, so a local run reuses
|
||||
# CI's remote cache. make lint, scripts/pr-lint and the pre-commit hook use it.
|
||||
# .golangci.yml enables) runs beside every Go compile, so `bazel build`/
|
||||
# `bazel test` fails on a finding -- in bazel.yml's one validating (owner)
|
||||
# lane per configuration (since F5 S1: a race-group non-owner lane reuses
|
||||
# its owner's nogo results instead of re-running them; see
|
||||
# scripts/nogo_lint_policy_test.go's nogoConfigurations table).
|
||||
# --config=nogo asks for the analysis alone (the nogo_fix output group;
|
||||
# nothing is linked), in the race configuration bazel.yml's unit lane
|
||||
# (test:ci) builds, so a local run reuses CI's remote cache. make lint,
|
||||
# scripts/pr-lint and the pre-commit hook use it.
|
||||
build:nogo --@rules_go//go/config:race
|
||||
build:nogo --keep_going
|
||||
build:nogo --output_groups=nogo_fix
|
||||
|
|
|
|||
4
.github/workflows/bazel.yml
vendored
4
.github/workflows/bazel.yml
vendored
|
|
@ -2543,7 +2543,7 @@ jobs:
|
|||
# bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path).
|
||||
- name: bazel build //cmd/bd:bd_for_tests
|
||||
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
||||
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$'
|
||||
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$' --norun_validations
|
||||
|
||||
- name: Package the Bazel-built bd
|
||||
if: ${{ steps.detect.outputs.mcp_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
||||
|
|
@ -2665,7 +2665,7 @@ jobs:
|
|||
# bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path).
|
||||
- name: bazel build //cmd/bd:bd_for_tests
|
||||
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
||||
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$'
|
||||
run: bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$' --norun_validations
|
||||
|
||||
- name: Package the Bazel-built bd
|
||||
if: ${{ steps.detect.outputs.npm_package == 'true' && needs.rbe.outputs.mode == 'remote' }}
|
||||
|
|
|
|||
12
Makefile
12
Makefile
|
|
@ -46,7 +46,7 @@ endif
|
|||
.PHONY: all build doctor-build test test-icu-path test-full-cgo test-regression test-upgrade test-cross-version test-migration corpus-regen githooks-regen bench bench-quick clean clean-test-tmp install install-force help check-up-to-date fmt fmt-check check-testing-short
|
||||
.PHONY: lint lint-changed vet
|
||||
.PHONY: check check-go test-go check-docs-go
|
||||
.PHONY: ci-pr-core ci-pr-lint ci-complexity ci-complexity-diff ci-complexity-check ci-package-mcp ci-package-npm
|
||||
.PHONY: ci-pr-core ci-pr-lint ci-complexity ci-complexity-diff ci-complexity-check ci-package-mcp ci-package-npm nogo-ownership
|
||||
.PHONY: api-gen api-check
|
||||
.PHONY: bazel-sync bazel-sync-check
|
||||
|
||||
|
|
@ -237,6 +237,16 @@ ci-pr-core:
|
|||
ci-pr-lint:
|
||||
@./scripts/ci/pr-lint.sh
|
||||
|
||||
# nogo-ownership is a dynamic, advisory cross-check (not part of the required
|
||||
# //scripts static suite, which cannot shell out to Bazel): it asks real
|
||||
# `bazel aquery` whether the nogo owner lane for each configuration group
|
||||
# (today: race, bazel-test/--config=ci) really does validate every file a
|
||||
# non-owner lane (embedded, doltserver, doltserver-proxied, dolt-race, which
|
||||
# pass --norun_validations) compiles. See tools/bazel/nogo_ownership.py and
|
||||
# scripts/nogo_lint_policy_test.go's nogoConfigurations table.
|
||||
nogo-ownership:
|
||||
@python3 tools/bazel/nogo_ownership.py
|
||||
|
||||
# The CI lint gate: native, plus //tools/bazel:release_cross for windows/amd64
|
||||
# and darwin/arm64 (BD_LINT_TARGETS selects; scripts/pr-lint).
|
||||
lint: ci-pr-lint
|
||||
|
|
|
|||
|
|
@ -1025,10 +1025,15 @@ Required` requires them to have run remotely and passed.
|
|||
the job it replaces.
|
||||
- `go test`'s own vet checks (cmd/go's `defaultVetFlags`, policy-tested
|
||||
equal to the toolchain's) and the golangci-lint linters `.golangci.yml`
|
||||
enables run as nogo (`//tools/nogo`) beside every compile of every Bazel
|
||||
lane: natively in `bazel test //... --config=ci`, and for every release
|
||||
platform in the `bazel-release-cross` lane
|
||||
(engdocs/LINTING.md). The former
|
||||
enables run as nogo (`//tools/nogo`): natively in `bazel test //...
|
||||
--config=ci`, and for every release platform in the
|
||||
`bazel-release-cross` lane (engdocs/LINTING.md). Each Bazel configuration
|
||||
validates in exactly one required lane, not beside every compile of
|
||||
every lane: the race configuration's nogo is owned by `test` above, and
|
||||
`embedded`, `doltserver`, `doltserver-proxied`, `dolt-race` (shadow) and
|
||||
the package gates pass `--norun_validations`, since they compile a
|
||||
strict subset of what `test` already validates on that configuration
|
||||
(`scripts/nogo_lint_policy_test.go`, engdocs/LINTING.md). The former
|
||||
`scripts-go-checks` (`Go checks (vet)`) and `pr-lint-wrapper`
|
||||
(`PR Lint (native|windows|darwin)`) jobs are retired.
|
||||
- The repository policy tests (`./scripts/...`, including the D2 guards)
|
||||
|
|
|
|||
|
|
@ -49,8 +49,14 @@ Where it gates:
|
|||
validates each of those compiles. So `//go:build windows`, `darwin`,
|
||||
`!linux` (and freebsd, android, arm64) files are analyzed from Linux: a
|
||||
superset of golangci-lint's former `GOOS=windows`/`GOOS=darwin` legs.
|
||||
- **Every other Bazel lane** (integration, embedded, dolt-server) validates
|
||||
what it compiles, including files only its build tags select.
|
||||
- The race configuration's nogo validations run once, owned by the `test`
|
||||
lane above: `embedded`, `doltserver`, `doltserver-proxied`, `dolt-race`
|
||||
(shadow) and the package gates (`package-mcp`, `package-npm`) all pass
|
||||
`--norun_validations`, since each compiles a strict subset of what `test`
|
||||
already validates on that configuration (`scripts/nogo_lint_policy_test.go`'s
|
||||
`TestNogoConfigurationsHaveOneValidatingLane` pins this).
|
||||
- **Every other Bazel lane** (integration) still validates what it compiles,
|
||||
including files only its build tags select.
|
||||
|
||||
Every PR is checked against the whole tree, not only its diff: the tree has
|
||||
no findings, so there is no baseline to scope against.
|
||||
|
|
|
|||
|
|
@ -4310,6 +4310,8 @@ var bazelEmbeddedRCLines = []string{
|
|||
"test:embedded --test_env=GO_TEST_WRAP_TESTV=1",
|
||||
"test:embedded --remote_download_regex=.*/test\\.xml$",
|
||||
"test:embedded --experimental_remote_cache_eviction_retries=0",
|
||||
// F5 S1: the race configuration's nogo is owned by the test lane.
|
||||
"test:embedded --norun_validations",
|
||||
}
|
||||
|
||||
func TestBazelEmbeddedJobRunsEmbeddedTier(t *testing.T) {
|
||||
|
|
|
|||
|
|
@ -4,6 +4,8 @@ import (
|
|||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
|
@ -15,6 +17,18 @@ import (
|
|||
// platforms compile inside the pure lane's release cross-compile
|
||||
// (//tools/bazel:release_cross, every release platform). These pin that
|
||||
// wiring and that golangci-lint is gone from CI, hooks and make.
|
||||
//
|
||||
// nogo validation is duplicated across lanes that compile the same Bazel
|
||||
// configuration (race: test, embedded, doltserver, doltserver-proxied,
|
||||
// dolt-race and the package gates). TestLintAndVetRunAsNogo used to forbid
|
||||
// `run_validations` anywhere in .bazelrc ("every lane validates"); that
|
||||
// blanket ban is replaced by TestNogoConfigurationsHaveOneValidatingLane
|
||||
// (T1), TestNogoOwnersAreRequired (T2), TestNogoRaceOwnerBuildsEverything
|
||||
// (T3), TestNoUnconditionalValidationOff (T5) and
|
||||
// TestRunValidationsOnlyOnAllowlistedLines (T6), which together pin "every
|
||||
// configuration's nogo runs in exactly one required lane" instead.
|
||||
// --norun_validations is a build-request option, not a configuration flag:
|
||||
// it changes no action key and discards no analysis.
|
||||
|
||||
// nogoBazelrcLines are .bazelrc's nogo configs, exactly.
|
||||
var nogoBazelrcLines = []string{
|
||||
|
|
@ -38,24 +52,6 @@ func TestLintAndVetRunAsNogo(t *testing.T) {
|
|||
if !equalStrings(got, nogoBazelrcLines) {
|
||||
t.Errorf(".bazelrc nogo configs = %q, want %q", got, nogoBazelrcLines)
|
||||
}
|
||||
// Every lane validates: a lane that skipped nogo would let findings in
|
||||
// the files only it compiles (integration-tagged, pure) through.
|
||||
if strings.Contains(rc, "run_validations") {
|
||||
t.Error(".bazelrc turns nogo validation off for some lane")
|
||||
}
|
||||
|
||||
// The cross-platform half: the pure lane's release cross-compile builds
|
||||
// every library and binary for every release platform, and nogo
|
||||
// validates each compile (TestReleaseCrossCompileRunsInBazelPureLane
|
||||
// pins the step). Nothing on that path may opt out of validation.
|
||||
if script := readPolicyFile(t, root, "scripts/ci/bazel-release-cross-compile.sh"); strings.Contains(script, "run_validations") {
|
||||
t.Error("scripts/ci/bazel-release-cross-compile.sh turns nogo validation off")
|
||||
}
|
||||
for _, line := range strings.Split(rc, "\n") {
|
||||
if strings.HasPrefix(strings.TrimSpace(line), "build:release-cross") && strings.Contains(line, "validation") {
|
||||
t.Errorf(".bazelrc %q changes validation for the release cross-compile", line)
|
||||
}
|
||||
}
|
||||
|
||||
// No workflow installs or runs golangci-lint.
|
||||
entries, err := os.ReadDir(filepath.Join(root, ".github", "workflows"))
|
||||
|
|
@ -107,3 +103,399 @@ func TestLintAndVetRunAsNogo(t *testing.T) {
|
|||
t.Error("scripts/ci/pr-lint.sh runs gofmt; //scripts/repochecks:fmt_test gates formatting")
|
||||
}
|
||||
}
|
||||
|
||||
// --- F5 S1: one validating lane per nogo configuration -----------------
|
||||
|
||||
// nogoKeyFlagPattern matches the build-affecting flags f5-spec.md §6 T1
|
||||
// calls out as fingerprint-relevant: they select a distinct Bazel
|
||||
// configuration (and so a distinct set of nogo action keys). Flags like
|
||||
// --test_tag_filters, --keep_going or --test_arg narrow which tests run but
|
||||
// do not change what is compiled, so they are not in the fingerprint.
|
||||
var nogoKeyFlagPattern = regexp.MustCompile(`--@rules_go//go/config:(race|pure|tags=\S+)|--platforms=\S+|//tools/bazel:release_platforms\S*`)
|
||||
|
||||
// expandBazelrcConfig returns every "<cmd> <opt>" line (as bazelRCConfigLines
|
||||
// formats them) that --config=name pulls in, recursively following any
|
||||
// nested --config=Y reference within those lines (as .bazelrc's own comments
|
||||
// describe, e.g. test:ci --config=prcore). seen prevents infinite recursion
|
||||
// on a cycle.
|
||||
func expandBazelrcConfig(rc, name string, seen map[string]bool) []string {
|
||||
if seen[name] {
|
||||
return nil
|
||||
}
|
||||
seen[name] = true
|
||||
var out []string
|
||||
for _, l := range bazelRCConfigLines(rc, name) {
|
||||
if v, ok := strings.CutPrefix(l, "build --config="); ok {
|
||||
out = append(out, expandBazelrcConfig(rc, v, seen)...)
|
||||
continue
|
||||
}
|
||||
if v, ok := strings.CutPrefix(l, "test --config="); ok {
|
||||
out = append(out, expandBazelrcConfig(rc, v, seen)...)
|
||||
continue
|
||||
}
|
||||
out = append(out, l)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// nogoFingerprint reduces a config's expanded lines to its key-affecting
|
||||
// flags, sorted and deduplicated, so two configs that build the same thing
|
||||
// compare equal regardless of line order or non-key-affecting flags.
|
||||
func nogoFingerprint(lines []string) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, l := range lines {
|
||||
for _, m := range nogoKeyFlagPattern.FindAllString(l, -1) {
|
||||
if !seen[m] {
|
||||
seen[m] = true
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// bazelrcConfigSkipsValidations reports whether --config=name's own
|
||||
// (non-recursive) .bazelrc lines pass --norun_validations. Non-recursive
|
||||
// because each non-owner config sets the flag itself (D1); inheriting it
|
||||
// through a nested --config would make the policy table here miss a
|
||||
// config that silently stopped validating by referencing one that did.
|
||||
func bazelrcConfigSkipsValidations(rc, name string) bool {
|
||||
for _, l := range bazelRCConfigLines(rc, name) {
|
||||
if strings.Contains(l, "--norun_validations") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// nogoConfigurations: f5-spec.md §6 T1's table. Each row is one Bazel
|
||||
// configuration (one nogo fingerprint): the .bazelrc --config names (or, for
|
||||
// the package gates, a pseudo-config checked against their pinned command
|
||||
// line) that build it, the bazel.yml job that owns validating it, and the
|
||||
// jobs that must pass --norun_validations instead.
|
||||
//
|
||||
// Only the race group is classified here (F5 S1). The race+integration
|
||||
// group (integration, doltserver-integration, doltserver-cmd) and `pure`'s
|
||||
// optional D4a are later slices (f5-spec.md §5 S2, S3): every lane in them
|
||||
// still validates today, so they are intentionally left out of this table
|
||||
// rather than added as a row that TestNogoConfigurationsHaveOneValidatingLane
|
||||
// could never satisfy. `pure`, js/wasm and release-cross are unaffected by
|
||||
// any slice (single-lane configurations: there is no other lane to
|
||||
// deduplicate against), so they are included as trivial one-member rows for
|
||||
// completeness and as a guard against a future lane quietly joining them.
|
||||
var nogoConfigurations = []struct {
|
||||
name string
|
||||
configs []string // .bazelrc --config names sharing this fingerprint, owner first
|
||||
owner string // bazel.yml job name required to validate
|
||||
nonOwners []string // bazel.yml job names required to pass --norun_validations
|
||||
}{
|
||||
{
|
||||
name: "race",
|
||||
configs: []string{"ci", "embedded", "doltserver", "doltserver-proxied", "dolt-race"},
|
||||
owner: bazelJobName,
|
||||
nonOwners: []string{bazelEmbedJobName, bazelDoltJobName, bazelProxiedJobName, bazelDoltRaceJobName},
|
||||
},
|
||||
{
|
||||
name: "pure",
|
||||
configs: []string{"pure"},
|
||||
owner: bazelPureJobName,
|
||||
},
|
||||
{
|
||||
name: "js-wasm",
|
||||
configs: []string{"js-wasm"},
|
||||
owner: bazelPureJobName,
|
||||
},
|
||||
}
|
||||
|
||||
// nogoPackageGateJobs: the package gates do not select a named .bazelrc
|
||||
// --config; their `bazel build` command line spells the race flag out
|
||||
// (pr_lanes_bazel_coverage_test.go pins the exact command). They are race
|
||||
// non-owners, checked against the "race" row's fingerprint and
|
||||
// --norun_validations requirement directly from that pinned command.
|
||||
var nogoPackageGateJobs = []string{bazelPackageMCPJobName, bazelPackageNPMJobName}
|
||||
|
||||
// T1 TestNogoConfigurationsHaveOneValidatingLane: every classified Bazel
|
||||
// configuration has exactly one validating (required) lane; every lane that
|
||||
// builds the same configuration passes --norun_validations instead; and
|
||||
// every config within a row shares that row's fingerprint, so a key-affecting
|
||||
// flag added to a non-owner (which would split it into a different, now
|
||||
// unvalidated, configuration) fails here instead of silently losing
|
||||
// coverage.
|
||||
func TestNogoConfigurationsHaveOneValidatingLane(t *testing.T) {
|
||||
rc := readPolicyFile(t, bazelPolicyRoot(t), ".bazelrc")
|
||||
workflow := readCIWorkflow(t, bazelWorkflowName)
|
||||
|
||||
seenConfig := map[string]string{} // config -> row name, to catch a config in two rows
|
||||
for _, row := range nogoConfigurations {
|
||||
if row.owner == "" {
|
||||
t.Fatalf("nogoConfigurations row %q has no owner", row.name)
|
||||
}
|
||||
if len(row.configs) == 0 {
|
||||
t.Fatalf("nogoConfigurations row %q has no configs", row.name)
|
||||
}
|
||||
ownerFP := nogoFingerprint(expandBazelrcConfig(rc, row.configs[0], map[string]bool{}))
|
||||
for _, c := range row.configs {
|
||||
if prev, ok := seenConfig[c]; ok {
|
||||
t.Errorf("--config=%s appears in both row %q and row %q", c, prev, row.name)
|
||||
}
|
||||
seenConfig[c] = row.name
|
||||
fp := nogoFingerprint(expandBazelrcConfig(rc, c, map[string]bool{}))
|
||||
if !sameStringSet(fp, ownerFP) {
|
||||
t.Errorf("row %q: --config=%s fingerprint %v != owner %s's %v (a key-affecting flag split this config out of the row)",
|
||||
row.name, c, fp, row.configs[0], ownerFP)
|
||||
}
|
||||
}
|
||||
|
||||
ownerJob := workflow.job(t, row.owner)
|
||||
if ownerJob.ContinueOnError {
|
||||
t.Errorf("row %q owner %s has continue-on-error; a skipped finding there would never fail anything", row.name, row.owner)
|
||||
}
|
||||
if bazelrcConfigSkipsValidations(rc, row.configs[0]) {
|
||||
t.Errorf("row %q owner config --config=%s passes --norun_validations; it is the row's only validating lane", row.name, row.configs[0])
|
||||
}
|
||||
|
||||
nonOwners := append([]string{}, row.nonOwners...)
|
||||
if row.name == "race" {
|
||||
nonOwners = append(nonOwners, nogoPackageGateJobs...)
|
||||
}
|
||||
for i, job := range nonOwners {
|
||||
if job == row.owner {
|
||||
t.Errorf("row %q lists its own owner %s as a non-owner", row.name, job)
|
||||
}
|
||||
isPackageGate := slices.Contains(nogoPackageGateJobs, job)
|
||||
if isPackageGate {
|
||||
// The package gates spell their flags out on the command
|
||||
// line rather than through a named --config
|
||||
// (pr_lanes_bazel_coverage_test.go pins the exact string);
|
||||
// confirmed here against their job's steps instead of
|
||||
// .bazelrc.
|
||||
step := workflow.job(t, job).step(t, "bazel build //cmd/bd:bd_for_tests")
|
||||
if !strings.Contains(step.Run, "--@rules_go//go/config:race") {
|
||||
t.Errorf("row %q non-owner %s's build step does not build the race configuration: %q", row.name, job, step.Run)
|
||||
}
|
||||
if !strings.Contains(step.Run, "--norun_validations") {
|
||||
t.Errorf("row %q non-owner %s's build step does not pass --norun_validations", row.name, job)
|
||||
}
|
||||
continue
|
||||
}
|
||||
// Map job -> its own --config, which is row.configs[1+i'] in
|
||||
// declaration order (owner's config is row.configs[0]).
|
||||
idx := i + 1
|
||||
if idx >= len(row.configs) {
|
||||
t.Fatalf("row %q has more non-owner jobs than configs (idx %d, configs %v)", row.name, idx, row.configs)
|
||||
}
|
||||
cfg := row.configs[idx]
|
||||
if !bazelrcConfigSkipsValidations(rc, cfg) {
|
||||
t.Errorf("row %q non-owner %s (--config=%s) does not pass --norun_validations; it duplicates %s's nogo", row.name, job, cfg, row.owner)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// T2 TestNogoOwnersAreRequired: each owner job is required by pr.yml's CI
|
||||
// Gate (or, for a flag-gated lane, the flag that is pinned true) with no
|
||||
// job-level continue-on-error, and its `if:` runs at least everywhere its
|
||||
// non-owners' do — otherwise a mode could run a non-owner's compile with no
|
||||
// owner validating it at all (f5-spec.md §9).
|
||||
func TestNogoOwnersAreRequired(t *testing.T) {
|
||||
workflow := readCIWorkflow(t, bazelWorkflowName)
|
||||
gate := readCIWorkflow(t, "pr.yml").job(t, "ci-gate")
|
||||
gateEnv := gate.step(t, "Evaluate CI gate").Env
|
||||
required := strings.Fields(gateEnv["CI_GATE_REQUIRED"])
|
||||
|
||||
ownerGateToken := map[string]string{
|
||||
bazelJobName: "BAZEL_TEST",
|
||||
bazelPureJobName: "BAZEL_PURE",
|
||||
}
|
||||
|
||||
events := []string{"pull_request", "merge_group", "pull_request_target", "push", "schedule", "workflow_dispatch"}
|
||||
|
||||
for _, row := range nogoConfigurations {
|
||||
ownerJob := workflow.job(t, row.owner)
|
||||
if token, ok := ownerGateToken[row.owner]; ok && !contains(required, token) {
|
||||
t.Errorf("row %q owner %s: CI_GATE_REQUIRED %v lacks %s", row.name, row.owner, required, token)
|
||||
}
|
||||
if ownerJob.ContinueOnError {
|
||||
t.Errorf("row %q owner %s has continue-on-error", row.name, row.owner)
|
||||
}
|
||||
|
||||
nonOwners := append([]string{}, row.nonOwners...)
|
||||
if row.name == "race" {
|
||||
nonOwners = append(nonOwners, nogoPackageGateJobs...)
|
||||
}
|
||||
for _, job := range nonOwners {
|
||||
if slices.Contains(nogoPackageGateJobs, job) {
|
||||
// The package gates' bazel build only ever runs in mode
|
||||
// remote (TestPackageGateJobs pins the surrounding `if:`),
|
||||
// and the owner's `if:` (mode != 'skip') is true whenever
|
||||
// mode == 'remote': "remote" is never "skip".
|
||||
step := workflow.job(t, job).step(t, "bazel build //cmd/bd:bd_for_tests")
|
||||
if !strings.Contains(step.If, "mode == 'remote'") {
|
||||
t.Errorf("%s's bazel-build step if %q no longer implies the owner's mode != 'skip'", job, step.If)
|
||||
}
|
||||
continue
|
||||
}
|
||||
nonOwnerJob := workflow.job(t, job)
|
||||
for _, mode := range bazelRBEModes {
|
||||
for _, event := range events {
|
||||
ctx := map[string]string{
|
||||
"needs.rbe.outputs.mode": mode,
|
||||
"needs.rbe.outputs.enabled": bazelModeEnabled(mode),
|
||||
"github.event_name": event,
|
||||
}
|
||||
if evalRRCIf(t, nonOwnerJob.If, ctx) && !evalRRCIf(t, ownerJob.If, ctx) {
|
||||
t.Errorf("row %q: %s runs in mode=%s event=%s but owner %s does not (owner if %q, non-owner if %q)",
|
||||
row.name, job, mode, event, row.owner, ownerJob.If, nonOwnerJob.If)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// T3 TestNogoRaceOwnerBuildsEverything: the race owner's target set covers
|
||||
// every file any race-group lane compiles (f5-spec.md §2.1): its test step
|
||||
// runs over the whole tree with no "-//" exclusion, and test:ci's expansion
|
||||
// narrows no target selection (--build_tests_only, --build_tag_filters)
|
||||
// — only test execution (--test_tag_filters etc., which T1's fingerprint
|
||||
// deliberately ignores).
|
||||
func TestNogoRaceOwnerBuildsEverything(t *testing.T) {
|
||||
workflow := readCIWorkflow(t, bazelWorkflowName)
|
||||
ownerJob := workflow.job(t, bazelJobName)
|
||||
test := ownerJob.step(t, "bazel test //... --config=ci")
|
||||
if !strings.Contains(test.Run, "bazel test //... --config=ci") {
|
||||
t.Fatalf("%s step does not run bazel test //... --config=ci:\n%s", bazelJobName, test.Run)
|
||||
}
|
||||
if regexp.MustCompile(`bazel test[^\n]*\s-//`).MatchString(test.Run) {
|
||||
t.Errorf("%s excludes a target (-//...) from //...: it would stop validating what it excludes", bazelJobName)
|
||||
}
|
||||
// The race owner's validating step must be unconditional: a step-level
|
||||
// `if:` or continue-on-error here would let the suite report green while
|
||||
// race-group nogo silently did not run, with nothing else in this file
|
||||
// (or T1/T2, which only check job-level fields) noticing.
|
||||
if test.If != "" {
|
||||
t.Errorf("%s step %q has if %q; the race owner's validating step must be unconditional", bazelJobName, test.Name, test.If)
|
||||
}
|
||||
if test.ContinueOnError != nil && test.ContinueOnError != false {
|
||||
t.Errorf("%s step %q has continue-on-error %v; a failed race-group nogo finding there would never fail the job", bazelJobName, test.Name, test.ContinueOnError)
|
||||
}
|
||||
if ownerJob.ContinueOnError {
|
||||
t.Errorf("%s job has continue-on-error; the race owner's job must be able to fail", bazelJobName)
|
||||
}
|
||||
|
||||
rc := readPolicyFile(t, bazelPolicyRoot(t), ".bazelrc")
|
||||
for _, l := range expandBazelrcConfig(rc, "ci", map[string]bool{}) {
|
||||
if strings.Contains(l, "build_tests_only") || strings.Contains(l, "build_tag_filters") {
|
||||
t.Errorf("test:ci (or a config it includes) narrows the target set: %q; the race owner must build //... whole", l)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// T5 TestNoUnconditionalValidationOff: no common/build/test line without a
|
||||
// :config suffix touches run_validations (that would turn validation off
|
||||
// everywhere, defeating every row's owner), and neither does build:nogo*,
|
||||
// build:release-cross, build:js-wasm, scripts/ci/bazel-release-cross-compile.sh,
|
||||
// or any row's own owner config (an owner opting itself out would leave its
|
||||
// configuration with no validating lane at all).
|
||||
func TestNoUnconditionalValidationOff(t *testing.T) {
|
||||
root := sourceRepoRoot(t)
|
||||
rc := readPolicyFile(t, root, ".bazelrc")
|
||||
|
||||
for _, raw := range strings.Split(rc, "\n") {
|
||||
line := strings.TrimSpace(raw)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
head, _, _ := strings.Cut(line, " ")
|
||||
cmd, _, hasCfg := strings.Cut(head, ":")
|
||||
if cmd != "common" && cmd != "build" && cmd != "test" {
|
||||
continue
|
||||
}
|
||||
if !hasCfg && strings.Contains(line, "run_validations") {
|
||||
t.Errorf(".bazelrc %q: an unconfigured line must not touch run_validations; it would apply to every lane, including every row's owner", line)
|
||||
}
|
||||
}
|
||||
|
||||
for _, prefix := range []string{"build:nogo", "build:release-cross", "build:js-wasm"} {
|
||||
for _, raw := range strings.Split(rc, "\n") {
|
||||
line := strings.TrimSpace(raw)
|
||||
if strings.HasPrefix(line, prefix) && strings.Contains(line, "validation") {
|
||||
t.Errorf(".bazelrc %q changes validation for %s, which has no non-owner duplicate to remove", line, prefix)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if script := readPolicyFile(t, root, "scripts/ci/bazel-release-cross-compile.sh"); strings.Contains(script, "run_validations") {
|
||||
t.Error("scripts/ci/bazel-release-cross-compile.sh turns nogo validation off; release-cross has no non-owner duplicate to remove")
|
||||
}
|
||||
|
||||
for _, row := range nogoConfigurations {
|
||||
if bazelrcConfigSkipsValidations(rc, row.configs[0]) {
|
||||
t.Errorf("row %q owner config --config=%s passes --norun_validations; it is the row's only validating lane", row.name, row.configs[0])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// T6 TestRunValidationsOnlyOnAllowlistedLines: the only
|
||||
// run_validations/norun_validations lines anywhere in .bazelrc or bazel.yml
|
||||
// are exactly the ones T1's race row already requires: one
|
||||
// "test:<non-owner config> --norun_validations" line per non-owner in
|
||||
// .bazelrc, and the two package-gate jobs' bazel-build steps. Main's
|
||||
// TestLintAndVetRunAsNogo used to ban run_validations outright, which caught
|
||||
// a configuration outside T1's table (e.g. test:integration,
|
||||
// test:doltserver-cmd -- the race+integration group is a later slice,
|
||||
// f5-spec.md §5 S2) quietly adding --norun_validations with no owner to
|
||||
// notice. T5 does not re-check that case: it only forbids unconfigured
|
||||
// lines and a short prefix list. This allowlist restores the ban for every
|
||||
// other line without re-banning the four the race row requires.
|
||||
func TestRunValidationsOnlyOnAllowlistedLines(t *testing.T) {
|
||||
root := sourceRepoRoot(t)
|
||||
|
||||
var raceNonOwners []string
|
||||
for _, row := range nogoConfigurations {
|
||||
if row.name == "race" {
|
||||
raceNonOwners = row.configs[1:]
|
||||
}
|
||||
}
|
||||
if len(raceNonOwners) == 0 {
|
||||
t.Fatal("race row has no non-owner configs to allowlist")
|
||||
}
|
||||
allowed := map[string]bool{}
|
||||
for _, cfg := range raceNonOwners {
|
||||
allowed["test:"+cfg+" --norun_validations"] = true
|
||||
}
|
||||
|
||||
rc := readPolicyFile(t, root, ".bazelrc")
|
||||
for _, raw := range strings.Split(rc, "\n") {
|
||||
line := strings.TrimSpace(raw)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(line, "run_validations") && !allowed[line] {
|
||||
t.Errorf(".bazelrc %q: run_validations is only allowed as one of %v; any other line (an unlisted config, an --integration config, or a bare --run_validations toggle) would turn validation off with no owner lane catching it", line, sortedKeys(allowed))
|
||||
}
|
||||
}
|
||||
|
||||
// bazel.yml: only the two package-gate jobs' bazel-build steps may pass
|
||||
// --norun_validations (they spell the race build flags out on the
|
||||
// command line rather than through a .bazelrc --config; T1 pins their
|
||||
// content). Anywhere else -- a new job, a new step, a non-package-gate
|
||||
// job -- would turn a lane's validation off unnoticed.
|
||||
workflow := readCIWorkflow(t, bazelWorkflowName)
|
||||
for name, j := range workflow.Jobs {
|
||||
for _, st := range j.Steps {
|
||||
if !strings.Contains(st.Run, "run_validations") {
|
||||
continue
|
||||
}
|
||||
if !slices.Contains(nogoPackageGateJobs, name) {
|
||||
stepLabel := st.Name
|
||||
if stepLabel == "" {
|
||||
stepLabel = st.Uses
|
||||
}
|
||||
t.Errorf("%s job %q step %q: run_validations outside the package-gate build steps", bazelWorkflowName, name, stepLabel)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -64,6 +64,8 @@ var bazelPRLaneRCLines = map[string][]string{
|
|||
"test:doltserver --test_arg=-test.parallel=4",
|
||||
"test:doltserver --local_test_jobs=4",
|
||||
"test:doltserver --remote_download_regex=.*/test\\.(log|xml)$",
|
||||
// F5 S1: the race configuration's nogo is owned by the test lane.
|
||||
"test:doltserver --norun_validations",
|
||||
},
|
||||
"pure": {
|
||||
"build:pure --@rules_go//go/config:pure",
|
||||
|
|
@ -318,7 +320,11 @@ func testPackageGateJobs(t *testing.T, prGateRequired []string) {
|
|||
// e2b78f7d7a: both configs cquery to the identical
|
||||
// bazel-out/k8-fastbuild-ST-.../bin/cmd/bd/bd_for_tests/bd path).
|
||||
buildStep := job.step(t, "bazel build //cmd/bd:bd_for_tests")
|
||||
if buildStep.If != bazelPathIf || buildStep.Run != `bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$'` {
|
||||
// F5 S1: the package gates' build is a race-configuration non-owner
|
||||
// (bd_for_tests is also built, with its nogo already validated, by
|
||||
// bazel-test's `bazel test //... --config=ci`), so it passes
|
||||
// --norun_validations too.
|
||||
if buildStep.If != bazelPathIf || buildStep.Run != `bazel build --@rules_go//go/config:race //cmd/bd:bd_for_tests --remote_download_regex='.*/bin/cmd/bd/bd_for_tests/bd$' --norun_validations` {
|
||||
t.Errorf("%s bazel build step: if %q, run %q; want if %q", lane.job, buildStep.If, buildStep.Run, bazelPathIf)
|
||||
}
|
||||
pkgBD := job.step(t, "Package the Bazel-built bd")
|
||||
|
|
|
|||
|
|
@ -1092,6 +1092,8 @@ var bazelDoltServerRCLines = map[string][]string{
|
|||
"test:doltserver-proxied --local_test_jobs=4",
|
||||
"test:doltserver-proxied --remote_download_regex=.*/test\\.(log|xml)$",
|
||||
"test:doltserver-proxied --experimental_remote_cache_eviction_retries=0",
|
||||
// F5 S1: the race configuration's nogo is owned by the test lane.
|
||||
"test:doltserver-proxied --norun_validations",
|
||||
},
|
||||
"doltserver-integration": {
|
||||
"build:doltserver-integration --@rules_go//go/config:tags=gms_pure_go,integration",
|
||||
|
|
|
|||
273
tools/bazel/nogo_ownership.py
Normal file
273
tools/bazel/nogo_ownership.py
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Dynamic cross-check for the nogo-ownership policy (scripts/nogo_lint_policy_test.go).
|
||||
|
||||
The static policy tests (T1-T3, T5-T6) pin .bazelrc/bazel.yml *text*: a
|
||||
config's flags, and which lane owns its validations. This script instead
|
||||
asks Bazel itself whether the owner's RunNogo action set for a configuration
|
||||
really is a superset of every non-owner's RunNogo action set for that same
|
||||
configuration -- i.e. the owner genuinely analyzes every first-party file a
|
||||
non-owner compiles, so skipping validation on the non-owner loses no lint
|
||||
coverage.
|
||||
|
||||
Scope (S1): the race configuration group only -- bazel-test (owner, race)
|
||||
vs. embedded/doltserver/doltserver-proxied/dolt-race (non-owners, which pass
|
||||
--norun_validations). The integration group (S2) is out of scope until its
|
||||
lanes gain the same split; see scripts/nogo_lint_policy_test.go's
|
||||
nogoConfigurations table, which this script's RACE_GROUP mirrors.
|
||||
|
||||
Each config's key-affecting flags (the same fingerprint
|
||||
scripts/nogo_lint_policy_test.go's nogoKeyFlagPattern extracts) are read
|
||||
straight out of .bazelrc and passed to `aquery`/`query` directly: those are
|
||||
"build"-family commands and do not see flags defined under a bare `test:X`
|
||||
.bazelrc stanza, so `--config=X` itself cannot be used here.
|
||||
|
||||
The owner (`test:ci`) builds //... whole (T3 pins that: no
|
||||
--build_tests_only, no --build_tag_filters), so its RunNogo set is queried
|
||||
directly over //.... Every non-owner here instead passes --build_tests_only
|
||||
with a --test_tag_filters=<tag[,tag...]> value, which narrows what `bazel
|
||||
test` actually builds to deps(the tagged tests) -- querying RunNogo over
|
||||
//... whole for a non-owner would silently compare the *owner's* universe
|
||||
against itself and always report full coverage (ga-f5s1-nogo-ownership-1:
|
||||
this was exactly that bug). So each non-owner's target set here is derived
|
||||
the same way `bazel test --build_tests_only --test_tag_filters=...` derives
|
||||
it: the deps closure of `tests(//...)` filtered down to targets carrying one
|
||||
of the lane's tags, read as an *exact* tag (not a substring -- "dolt-server"
|
||||
is a prefix of both "dolt-server-proxied" and "dolt-server-cmd", so a naive
|
||||
substring match over `attr(tags, ...)` would wrongly fold those in; tags are
|
||||
read from `--output=xml` instead, which lists each tag as its own exact
|
||||
string, to sidestep that).
|
||||
|
||||
This is advisory and dynamic (runs real `bazel query`/`aquery`, a few
|
||||
minutes locally): it is intentionally not part of the required `//scripts`
|
||||
static test suite, which cannot shell out to Bazel from inside a Bazel
|
||||
action. Run it with `make nogo-ownership`, locally or on demand in a
|
||||
slice's own PR.
|
||||
|
||||
Exit status: 0 if every non-owner's RunNogo target set is covered by the
|
||||
owner's, 2 otherwise. Prints the offending targets.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
|
||||
# (name, owner config, [non-owner config, ...]) for the race group.
|
||||
# Keep in sync with scripts/nogo_lint_policy_test.go's nogoConfigurations.
|
||||
RACE_GROUP = (
|
||||
"race",
|
||||
"ci",
|
||||
["embedded", "doltserver", "doltserver-proxied", "dolt-race"],
|
||||
)
|
||||
|
||||
# Mirrors scripts/nogo_lint_policy_test.go's nogoKeyFlagPattern.
|
||||
KEY_FLAG_RE = re.compile(
|
||||
r"--@rules_go//go/config:(?:race|pure|tags=\S+)"
|
||||
r"|--platforms=\S+"
|
||||
r"|//tools/bazel:release_platforms\S*"
|
||||
)
|
||||
|
||||
CONFIG_LINE_RE = re.compile(r"^(build|test):([A-Za-z0-9_-]+)\s+(.*)$")
|
||||
TEST_TAG_FILTERS_RE = re.compile(r"^(?:build|test) --test_tag_filters=(\S+)$")
|
||||
BUILD_TESTS_ONLY_RE = re.compile(r"^(?:build|test) --build_tests_only$")
|
||||
|
||||
|
||||
def bazelrc_lines(rc_text: str, name: str) -> list[str]:
|
||||
"""Every raw ("build"|"test", opt) line for --config=name, in file order."""
|
||||
out = []
|
||||
for line in rc_text.splitlines():
|
||||
line = line.split(" #", 1)[0].rstrip()
|
||||
m = CONFIG_LINE_RE.match(line)
|
||||
if m and m.group(2) == name:
|
||||
out.append(f"{m.group(1)} {m.group(3)}")
|
||||
return out
|
||||
|
||||
|
||||
def expand_config(rc_text: str, name: str, seen: set[str]) -> list[str]:
|
||||
"""Recursively follows nested --config=Y references, as the Go test does."""
|
||||
if name in seen:
|
||||
return []
|
||||
seen.add(name)
|
||||
out = []
|
||||
for line in bazelrc_lines(rc_text, name):
|
||||
for prefix in ("build --config=", "test --config="):
|
||||
if line.startswith(prefix):
|
||||
out.extend(expand_config(rc_text, line[len(prefix):], seen))
|
||||
break
|
||||
else:
|
||||
out.append(line)
|
||||
return out
|
||||
|
||||
|
||||
def fingerprint(rc_text: str, name: str) -> list[str]:
|
||||
lines = expand_config(rc_text, name, set())
|
||||
flags = set()
|
||||
for line in lines:
|
||||
flags.update(m.group(0) for m in KEY_FLAG_RE.finditer(line))
|
||||
return sorted(flags)
|
||||
|
||||
|
||||
def test_tag_filters(rc_text: str, name: str) -> list[str] | None:
|
||||
"""The lane's --test_tag_filters value, as a list of exact tags, if any.
|
||||
|
||||
None means the lane sets no --test_tag_filters (the owner: it builds
|
||||
//... whole, per T3). A --test_tag_filters value with a leading '-'
|
||||
(tag exclusion) is not used by any race-group lane today; this raises
|
||||
rather than silently mis-scoping a future lane that added one.
|
||||
"""
|
||||
for line in expand_config(rc_text, name, set()):
|
||||
m = TEST_TAG_FILTERS_RE.match(line)
|
||||
if not m:
|
||||
continue
|
||||
tags = m.group(1).split(",")
|
||||
for tag in tags:
|
||||
if tag.startswith("-"):
|
||||
raise NotImplementedError(
|
||||
f"--config={name}: --test_tag_filters={m.group(1)} has an "
|
||||
"exclusion tag; this script only understands inclusion filters"
|
||||
)
|
||||
return tags
|
||||
return None
|
||||
|
||||
|
||||
def lane_builds_tests_only(rc_text: str, name: str) -> bool:
|
||||
return any(BUILD_TESTS_ONLY_RE.match(line) for line in expand_config(rc_text, name, set()))
|
||||
|
||||
|
||||
def run_query_xml(bazel: str, query: str) -> ET.Element:
|
||||
cmd = [bazel, "--nohome_rc", "query", query, "--output=xml", "--noimplicit_deps"]
|
||||
out = subprocess.run(cmd, capture_output=True, text=True, check=True).stdout
|
||||
return ET.fromstring(out)
|
||||
|
||||
|
||||
def tagged_test_targets(bazel: str, tags: list[str]) -> set[str]:
|
||||
"""Labels of tests(//...) carrying at least one of tags, read exactly.
|
||||
|
||||
Tags are read from --output=xml's own <list name="tags"><string .../></list>
|
||||
elements rather than matched with attr()'s regex against a stringified
|
||||
list: "dolt-server" is a prefix of both "dolt-server-proxied" and
|
||||
"dolt-server-cmd" (and attr()'s per-element regex match is a substring
|
||||
match, not an exact one), so attr(tags, "dolt-server", //...) also
|
||||
returns proxied- and cmd-tagged targets -- the exact bug this script
|
||||
exists to avoid reintroducing one level up.
|
||||
"""
|
||||
root = run_query_xml(bazel, "tests(//...)")
|
||||
wanted = set(tags)
|
||||
labels = set()
|
||||
for rule in root.findall("rule"):
|
||||
rule_tags = {
|
||||
s.get("value")
|
||||
for list_el in rule.findall("list[@name='tags']")
|
||||
for s in list_el.findall("string")
|
||||
}
|
||||
if rule_tags & wanted:
|
||||
labels.add(rule.get("name"))
|
||||
return labels
|
||||
|
||||
|
||||
def lane_universe(bazel: str, targets: set[str] | None) -> str:
|
||||
"""The query-language target expression this lane builds.
|
||||
|
||||
None (the owner, or any lane with no --build_tests_only): //... whole.
|
||||
Otherwise (a lane that builds only its tagged tests and their deps, as
|
||||
--build_tests_only + --test_tag_filters does): the deps closure of just
|
||||
those targets.
|
||||
"""
|
||||
if targets is None:
|
||||
return "//..."
|
||||
if not targets:
|
||||
raise ValueError("lane_universe: empty tagged target set")
|
||||
return "deps(set(" + " ".join(sorted(targets)) + "))"
|
||||
|
||||
|
||||
def run_nogo_targets(bazel: str, flags: list[str], universe: str) -> set[str]:
|
||||
"""Returns the set of target labels with a RunNogo action under flags."""
|
||||
cmd = [bazel, "--nohome_rc", "aquery", f'mnemonic("RunNogo", {universe})', *flags,
|
||||
"--output=jsonproto"]
|
||||
out = subprocess.run(cmd, capture_output=True, text=True, check=True).stdout
|
||||
data = json.loads(out)
|
||||
targets_by_id = {t["id"]: t["label"] for t in data.get("targets", [])}
|
||||
labels = set()
|
||||
for action in data.get("actions", []):
|
||||
if action.get("mnemonic") != "RunNogo":
|
||||
continue
|
||||
label = targets_by_id.get(action.get("targetId"))
|
||||
if label and not label.startswith("@"):
|
||||
labels.add(label)
|
||||
return labels
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--bazel", default="bazel", help="bazel/bazelisk binary to invoke")
|
||||
parser.add_argument("--bazelrc", default=".bazelrc")
|
||||
args = parser.parse_args()
|
||||
|
||||
rc_text = Path(args.bazelrc).read_text()
|
||||
name, owner, non_owners = RACE_GROUP
|
||||
|
||||
owner_fp = fingerprint(rc_text, owner)
|
||||
if lane_builds_tests_only(rc_text, owner):
|
||||
raise NotImplementedError(
|
||||
f"--config={owner} now passes --build_tests_only, narrowing its target "
|
||||
"set; this script assumes the race owner builds //... whole "
|
||||
"(scripts/nogo_lint_policy_test.go's T3). --test_tag_filters alone "
|
||||
"(exclusion-only, as today's owner has) only narrows test execution, "
|
||||
"not the build, so it is not checked here."
|
||||
)
|
||||
owner_targets = run_nogo_targets(args.bazel, owner_fp, "//...")
|
||||
|
||||
problems = []
|
||||
for non_owner in non_owners:
|
||||
non_owner_fp = fingerprint(rc_text, non_owner)
|
||||
if non_owner_fp != owner_fp:
|
||||
problems.append(
|
||||
f"{name}: --config={non_owner}'s fingerprint {non_owner_fp} "
|
||||
f"differs from owner --config={owner}'s {owner_fp}; it is no "
|
||||
"longer safe to skip its validations"
|
||||
)
|
||||
continue
|
||||
|
||||
tags = test_tag_filters(rc_text, non_owner)
|
||||
if tags is None or not lane_builds_tests_only(rc_text, non_owner):
|
||||
# No narrowing this script knows how to replicate: fall back to
|
||||
# the owner's own universe, which is always a safe (if coarser)
|
||||
# comparison.
|
||||
non_owner_universe = "//..."
|
||||
else:
|
||||
tagged = tagged_test_targets(args.bazel, tags)
|
||||
if not tagged:
|
||||
problems.append(
|
||||
f"{name}: --config={non_owner}'s --test_tag_filters={','.join(tags)} "
|
||||
"matches no test in //...; its coverage could not be checked"
|
||||
)
|
||||
continue
|
||||
non_owner_universe = lane_universe(args.bazel, tagged)
|
||||
|
||||
non_owner_targets = run_nogo_targets(args.bazel, non_owner_fp, non_owner_universe)
|
||||
missing = non_owner_targets - owner_targets
|
||||
if missing:
|
||||
problems.append(
|
||||
f"{name}: --config={non_owner} compiles {len(missing)} target(s) "
|
||||
f"with no RunNogo action under the owner --config={owner}: "
|
||||
+ ", ".join(sorted(missing)[:10])
|
||||
+ (" ..." if len(missing) > 10 else "")
|
||||
)
|
||||
|
||||
if problems:
|
||||
print("nogo_ownership: FAIL", file=sys.stderr)
|
||||
for p in problems:
|
||||
print(f" {p}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
print(f"nogo_ownership: OK ({name}: --config={owner} covers "
|
||||
f"{', '.join(non_owners)})")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Loading…
Add table
Add a link
Reference in a new issue