ci: parallel PR Lint matrix with pinned golangci-lint, fold duplicate Lint, split scripts-go-checks (F5) (#7127)
* ci(lint,scripts): fold duplicate Lint job; matrix PR Lint and scripts-go-checks (F5) Lint (golangci-lint-action, only-new-issues) was functionally identical to pr-lint-wrapper's native leg (H7): same pinned binary, config, flags, and diff scope. Fold it into the wrapper in both pr.yml and main.yml, and drop LINT from pr.yml's ci-gate. pr-lint-wrapper becomes a 3-leg matrix (native/windows/darwin, fail-fast: false) under the same PR_LINT_WRAPPER gate id, installing golangci-lint v2.10.1 from a pinned, sha256-verified release binary (scripts/ci/install-golangci-lint.sh) instead of go install/the Action, with a per-leg golangci-lint cache (pr.yml restores only; main.yml's matrix is the sole saver). scripts-go-checks becomes a 3-leg matrix (scripts-test/vet/allowlisted, fail-fast: false) under the same SCRIPTS_GO_CHECKS gate id, giving go vet its own build cache instead of sharing the race build cache. main.yml gains a go-vet-cache job to warm that cache on push; build-artifacts remains the Go module cache's sole saver. Splitting each job's previously-sequential passes into parallel matrix legs, combined with removing the duplicate Lint job outright, takes wall-clock time off the PR gate's critical path. Updates scripts/ci_workflow_test.go and scripts/pr_lanes_bazel_coverage_test.go for the new matrices, cache topology, and the removed LINT id; adds TestGolangciLintInstallScriptPinned. Updates engdocs/LINTING.md and the live prose in engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md (its disclaimed "historical snapshot" gate-wiring block is left as-is). Verified: go test ./scripts/..., bazel --nohome_rc test //scripts:scripts_test (after adding engdocs/LINTING.md and .githooks/pre-commit as scripts_test data), actionlint, shellcheck, make ci-pr-policy, make bazel-sync-check. Confirmed lint-leg parity against current main with the pinned binary (0 issues PR-scoped; the same two pre-existing unparam findings full-tree, unchanged) and scripts-go-checks leg parity standalone. Mutation-tested the new/updated policy assertions with six deliberate breaks (dropped matrix leg, reintroduced LINT, corrupted the pinned sha256, removed a cache-save step, dropped the vet leg, reintroduced the lint job); each was caught with an on-target failure. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(ci): review-fix pass on F5's lint/scripts-go-checks matrix (SF-1..SF-5, N-1..N-8) Addresses all SHOULD-FIX and NIT findings from the review of 7e858e2714 (no BLOCKERs): - SF-1: lint/vet cache saves now run on !cancelled() instead of bare cache-miss, so a red lint/vet run (main is currently lint-red on two pre-existing, unrelated unparam findings) still warms the cache for PR legs. - SF-2: ci-measurements.yml now exports golangci-lint's install dir onto PATH in the same step, since $GITHUB_PATH only takes effect in later steps. - SF-3: adds a behavioral test (fake curl/tar) proving the install script's sha256 verification fails closed. - SF-4: drops the commit sha from the golangci-lint cache key (saves only on config/dependency change, not every push) and trims the windows/darwin legs to cache only the small golangci-lint result cache, not their large GOCACHE. - SF-5: removes the vet cache's dead non-race-build-cache fallback restore key (different path list, can never hit) in both pr.yml and main.yml. - N-1..N-8: doc/comment accuracy fixes, restores the "moved code reads as new" caveat, adds a module-cache restore step and timeout-minutes to pr-lint-wrapper/go-vet-cache, pins fetch-depth/GOCACHE via new policy-test assertions, and flags a superseded doc snapshot. New commit on top of 7e858e2714; existing commit trailers are left untouched per explicit instruction. Not pushed, no PR opened. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * build(bazel): auto-sync BUILD files Applied from the bazel-sync-patch artifact of https://github.com/gastownhall/beads/actions/runs/37105481015 (`make bazel-sync`: gazelle, tools/bazel/go_srcs.py, bazel mod tidy). Only BUILD.bazel, MODULE.bazel and MODULE.bazel.lock are ever pushed; see scripts/bazel-autofix-push.sh. --------- Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
parent
6e9a15507e
commit
bc19f6e478
14 changed files with 945 additions and 113 deletions
7
.github/workflows/ci-measurements.yml
vendored
7
.github/workflows/ci-measurements.yml
vendored
|
|
@ -83,7 +83,12 @@ jobs:
|
|||
source ./.buildflags
|
||||
|
||||
ci_time "install gotestsum" -- go install gotest.tools/gotestsum@v1.13.0
|
||||
ci_time "install golangci-lint" -- go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.10.1
|
||||
ci_time "install golangci-lint" -- ./scripts/ci/install-golangci-lint.sh
|
||||
# install-golangci-lint.sh appends to $GITHUB_PATH, which GitHub
|
||||
# Actions only applies to *later steps*, not later commands in this
|
||||
# same run block. Export it here too so the "pr-lint wrapper" call
|
||||
# below finds golangci-lint on PATH.
|
||||
export PATH="$RUNNER_TEMP/golangci-lint:$PATH"
|
||||
status=0
|
||||
ci_time "pr-policy wrapper" -- make ci-pr-policy || status=$?
|
||||
# -timeout=30m keeps this in step with the lane it measures
|
||||
|
|
|
|||
103
.github/workflows/main.yml
vendored
103
.github/workflows/main.yml
vendored
|
|
@ -471,26 +471,72 @@ jobs:
|
|||
run: make ci-pr-core
|
||||
|
||||
pr-lint-wrapper:
|
||||
name: PR Lint (wrapper timing)
|
||||
name: PR Lint (${{ matrix.target }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
target: [native, windows, darwin]
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Go
|
||||
id: setup-go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: false
|
||||
|
||||
- name: Restore Go module cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ~/go/pkg/mod
|
||||
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
||||
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Install golangci-lint
|
||||
run: go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.10.1
|
||||
run: ./scripts/ci/install-golangci-lint.sh
|
||||
|
||||
# This push lane is the only saver (§0, F5.2): keyed on file content and
|
||||
# the leg; a miss on the exact content key falls back to the leg's most
|
||||
# recent entry, which golangci-lint revalidates against the current
|
||||
# config and inputs. pr.yml only ever restores. The key omits the sha
|
||||
# (SF-4) so a save only happens when .golangci.yml or go.sum change,
|
||||
# not on every push to main.
|
||||
- name: Restore golangci-lint cache
|
||||
id: restore-golangci-lint-cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
~/.cache/golangci-lint
|
||||
${{ matrix.target == 'native' && format('{0}/go-cache/lint-{1}', runner.temp, matrix.target) || '' }}
|
||||
key: beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-${{ hashFiles('.golangci.yml', 'go.sum') }}
|
||||
restore-keys: beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-
|
||||
|
||||
# No BD_LINT_NEW_FROM_MERGE_BASE here: the push lane sweeps the whole
|
||||
# tree, including the GOOS=windows and GOOS=darwin cross-lints. pr.yml
|
||||
# scopes the same wrapper to the PR's own diff.
|
||||
- name: Run PR lint wrapper
|
||||
env:
|
||||
BD_LINT_TARGETS: ${{ matrix.target }}
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/lint-${{ matrix.target }}
|
||||
run: make ci-pr-lint
|
||||
|
||||
# !cancelled() (not just a cache-hit check): golangci-lint's own cache is
|
||||
# content-addressed and GOCACHE is unaffected by lint findings, so it is
|
||||
# safe -- and worth keeping -- to save even when the lint step itself
|
||||
# fails (SF-1). Without this, a red lint run on main never warms the
|
||||
# cache and every PR leg stays cold.
|
||||
- name: Save golangci-lint cache
|
||||
if: ${{ !cancelled() && steps.restore-golangci-lint-cache.outputs.cache-hit != 'true' }}
|
||||
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
~/.cache/golangci-lint
|
||||
${{ matrix.target == 'native' && format('{0}/go-cache/lint-{1}', runner.temp, matrix.target) || '' }}
|
||||
key: beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-${{ hashFiles('.golangci.yml', 'go.sum') }}
|
||||
|
||||
# Cross-platform test matrix (Linux/macOS only - Windows uses smoke tests)
|
||||
#
|
||||
# Both legs run every package in one `./...` invocation, so go test's DEFAULT
|
||||
|
|
@ -1253,28 +1299,59 @@ jobs:
|
|||
- name: Check gofmt
|
||||
run: make fmt-check
|
||||
|
||||
# The whole-tree lint sweep. pr.yml's job of the same name reports only what
|
||||
# a PR introduces, so this push lane is where a pre-existing or newly landed
|
||||
# issue anywhere in the tree surfaces.
|
||||
lint:
|
||||
name: Lint
|
||||
# Warms the vet cache pr.yml's scripts-go-checks vet leg restores (F5.3).
|
||||
# `go vet` caches successful vet actions, so a warm cache re-vets only
|
||||
# changed packages and their dependents. Restore-only in pr.yml (§0); this
|
||||
# push job is the only saver, on an exact-key miss.
|
||||
go-vet-cache:
|
||||
name: Warm go vet cache
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
- name: Set up Go
|
||||
id: setup-go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: false
|
||||
|
||||
- name: golangci-lint
|
||||
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9
|
||||
env:
|
||||
GOWORK: "off"
|
||||
- name: Restore Go module cache
|
||||
id: restore-go-module-cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
version: v2.10.1
|
||||
args: --config=.golangci.yml --modules-download-mode=readonly --timeout=5m --build-tags=gms_pure_go
|
||||
path: ~/go/pkg/mod
|
||||
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
||||
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Restore vet Go build cache
|
||||
id: restore-vet-go-build-cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ runner.temp }}/go-cache/vet
|
||||
key: beads-go-vet-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-vet-${{ github.sha }}
|
||||
# The non-race build cache fallback this used to list can never hit:
|
||||
# actions/cache versions an entry by its path list, and that cache
|
||||
# saves to a different path (.../go-cache/non-race), so the prefix
|
||||
# match is always filtered out (SF-5).
|
||||
restore-keys: beads-go-vet-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-vet-
|
||||
|
||||
- name: Go vet with go test's checks
|
||||
env:
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/vet
|
||||
run: bash scripts/ci/go-test-vet.sh
|
||||
|
||||
# build-artifacts is the module cache's sole saver (same key); this job
|
||||
# only restores it to warm the vet run, and only saves its own cache.
|
||||
# !cancelled(): go vet's cache is content-addressed, so it is safe to
|
||||
# save even when the vet step itself fails (SF-1).
|
||||
- name: Save vet Go build cache
|
||||
if: ${{ !cancelled() && steps.restore-vet-go-build-cache.outputs.cache-hit != 'true' }}
|
||||
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ runner.temp }}/go-cache/vet
|
||||
key: beads-go-vet-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-vet-${{ github.sha }}
|
||||
|
||||
test-nix:
|
||||
name: Test Nix Flake
|
||||
|
|
|
|||
124
.github/workflows/pr.yml
vendored
124
.github/workflows/pr.yml
vendored
|
|
@ -865,9 +865,13 @@ jobs:
|
|||
# - the Go tests `bazel test --config=ci` does not run or skips
|
||||
# (tools/bazel/equivalence_allowlist.txt), each required to pass.
|
||||
scripts-go-checks:
|
||||
name: Go test (scripts), go vet and Bazel-skipped tests
|
||||
name: Go checks (${{ matrix.check }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
check: [scripts-test, vet, allowlisted]
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
|
||||
|
|
@ -886,16 +890,39 @@ jobs:
|
|||
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Restore race Go build cache
|
||||
if: matrix.check == 'scripts-test'
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ runner.temp }}/go-cache/race
|
||||
key: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-race-${{ github.sha }}
|
||||
restore-keys: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-race-
|
||||
|
||||
- name: Restore vet Go build cache
|
||||
if: matrix.check == 'vet'
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ runner.temp }}/go-cache/vet
|
||||
key: beads-go-vet-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-vet-${{ github.sha }}
|
||||
# The non-race build cache fallback this used to list can never hit:
|
||||
# actions/cache versions an entry by its path list, and that cache
|
||||
# saves to a different path (.../go-cache/non-race), so the prefix
|
||||
# match is always filtered out (SF-5).
|
||||
restore-keys: beads-go-vet-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-vet-
|
||||
|
||||
- name: Restore non-race Go build cache
|
||||
if: matrix.check == 'allowlisted'
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ${{ runner.temp }}/go-cache/non-race
|
||||
key: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-${{ github.sha }}
|
||||
restore-keys: beads-go-build-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-gms_pure_go-non-race-
|
||||
|
||||
- name: Install Dolt
|
||||
if: matrix.check == 'scripts-test' || matrix.check == 'allowlisted'
|
||||
run: ./scripts/ci/install-dolt.sh
|
||||
|
||||
- name: Configure Git and Dolt identity
|
||||
if: matrix.check == 'scripts-test' || matrix.check == 'allowlisted'
|
||||
run: |
|
||||
git config --global user.name "CI Bot"
|
||||
git config --global user.email "ci@beads.test"
|
||||
|
|
@ -903,22 +930,34 @@ jobs:
|
|||
dolt config --global --add user.email "ci@beads.test"
|
||||
|
||||
- name: Go test the scripts packages
|
||||
if: matrix.check == 'scripts-test'
|
||||
env:
|
||||
BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION: '1'
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/race
|
||||
run: bash scripts/ci/scripts-go-test.sh
|
||||
|
||||
- name: Go vet with go test's checks
|
||||
if: ${{ !cancelled() && steps.setup-go.outcome == 'success' }}
|
||||
if: ${{ !cancelled() && steps.setup-go.outcome == 'success' && matrix.check == 'vet' }}
|
||||
env:
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/vet
|
||||
run: bash scripts/ci/go-test-vet.sh
|
||||
|
||||
- name: Run the Go tests the Bazel lane skips
|
||||
if: ${{ !cancelled() && steps.setup-go.outcome == 'success' }}
|
||||
if: ${{ !cancelled() && steps.setup-go.outcome == 'success' && matrix.check == 'allowlisted' }}
|
||||
env:
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/non-race
|
||||
run: bash scripts/ci/allowlisted-go-tests.sh
|
||||
|
||||
pr-lint-wrapper:
|
||||
name: PR Lint (wrapper timing)
|
||||
name: PR Lint (${{ matrix.target }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
target: [native, windows, darwin]
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
with:
|
||||
|
|
@ -926,21 +965,47 @@ jobs:
|
|||
fetch-depth: 0
|
||||
|
||||
- name: Set up Go
|
||||
id: setup-go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: false
|
||||
|
||||
- name: Restore Go module cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: ~/go/pkg/mod
|
||||
key: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.mod', 'go.sum') }}
|
||||
restore-keys: beads-go-mod-v2-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
||||
|
||||
- name: Install golangci-lint
|
||||
run: go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.10.1
|
||||
run: ./scripts/ci/install-golangci-lint.sh
|
||||
|
||||
# Keyed on file content and the leg; a miss on the exact content key
|
||||
# falls back to the leg's most recent entry, which golangci-lint
|
||||
# revalidates against the current config and inputs. pr.yml never saves
|
||||
# this cache (§0); main.yml's pr-lint-wrapper matrix is the only saver,
|
||||
# on an exact-key miss. The key omits the sha (SF-4) so main.yml only
|
||||
# saves when .golangci.yml or go.sum change.
|
||||
- name: Restore golangci-lint cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
~/.cache/golangci-lint
|
||||
${{ matrix.target == 'native' && format('{0}/go-cache/lint-{1}', runner.temp, matrix.target) || '' }}
|
||||
key: beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-${{ hashFiles('.golangci.yml', 'go.sum') }}
|
||||
restore-keys: beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-
|
||||
|
||||
# Scope the wrapper's lint to what this PR introduces, matching the
|
||||
# `lint` job's only-new-issues below. main.yml runs the same wrapper with
|
||||
# this unset, so the whole tree is still swept on every push to main.
|
||||
# Both PR triggers (pull_request, merge_group) target main.
|
||||
# (now-deleted) `lint` job's only-new-issues scope. main.yml runs the
|
||||
# same wrapper with this unset, so the whole tree is still swept on
|
||||
# every push to main. Both PR triggers (pull_request, merge_group)
|
||||
# target main.
|
||||
- name: Run PR lint wrapper
|
||||
env:
|
||||
BD_LINT_NEW_FROM_MERGE_BASE: origin/main
|
||||
BD_LINT_TARGETS: ${{ matrix.target }}
|
||||
GOCACHE: ${{ runner.temp }}/go-cache/lint-${{ matrix.target }}
|
||||
run: make ci-pr-lint
|
||||
|
||||
# Advisory architecture signal. This deliberately does not participate in
|
||||
|
|
@ -1289,46 +1354,6 @@ jobs:
|
|||
- name: Check gofmt
|
||||
run: make fmt-check
|
||||
|
||||
# Lint lanes are split deliberately: the PR lane reports only the issues a PR
|
||||
# introduces, and main.yml's identical job sweeps the whole tree on every
|
||||
# push. The tradeoff is that a pre-existing issue in untouched code no longer
|
||||
# blocks a PR — but a lint word that lands on main now reds main's own run,
|
||||
# where it belongs, instead of reddening every open PR for a change it did
|
||||
# not make.
|
||||
#
|
||||
# "New" is measured off the diff, so MOVED CODE READS AS NEW: a pre-existing
|
||||
# violation carried into a PR by a move or a rename is reported against that
|
||||
# PR and blocks it. Fix it or //nolint it there — that is the accepted cost of
|
||||
# the trade above.
|
||||
lint:
|
||||
name: Lint
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
# only-new-issues reads the PR patch from the API.
|
||||
pull-requests: read
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
||||
with:
|
||||
# On merge_group there is no PR patch, so the action falls back to
|
||||
# --new-from-rev and needs real history.
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
cache: false
|
||||
|
||||
- name: golangci-lint
|
||||
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9
|
||||
env:
|
||||
GOWORK: "off"
|
||||
with:
|
||||
version: v2.10.1
|
||||
only-new-issues: true
|
||||
args: --config=.golangci.yml --modules-download-mode=readonly --timeout=5m --build-tags=gms_pure_go
|
||||
|
||||
windows-make-shell:
|
||||
name: Windows Make shell (${{ matrix.host }})
|
||||
runs-on: windows-latest
|
||||
|
|
@ -1950,7 +1975,6 @@ jobs:
|
|||
- test-dolt-server-fingerprint
|
||||
- contract-corpus
|
||||
- fmt-check
|
||||
- lint
|
||||
- windows-make-shell
|
||||
- bazel
|
||||
- bazel-coverage
|
||||
|
|
@ -1984,7 +2008,6 @@ jobs:
|
|||
TEST_DOLT_SERVER_FINGERPRINT
|
||||
CONTRACT_CORPUS
|
||||
FMT_CHECK
|
||||
LINT
|
||||
WINDOWS_MAKE_SHELL
|
||||
BAZEL
|
||||
BAZEL_TEST
|
||||
|
|
@ -2024,7 +2047,6 @@ jobs:
|
|||
TEST_DOLT_SERVER_FINGERPRINT: ${{ needs.test-dolt-server-fingerprint.result }}
|
||||
CONTRACT_CORPUS: ${{ needs.contract-corpus.result }}
|
||||
FMT_CHECK: ${{ needs.fmt-check.result }}
|
||||
LINT: ${{ needs.lint.result }}
|
||||
WINDOWS_MAKE_SHELL: ${{ needs.windows-make-shell.result }}
|
||||
# bazel.yml's lanes report their job.status as outputs; a lane that
|
||||
# did not report (skipped, or lost before its last step) reads as
|
||||
|
|
|
|||
11
BUILD.bazel
11
BUILD.bazel
|
|
@ -32,6 +32,17 @@ exports_files(
|
|||
visibility = ["//scripts/gitattributespolicy:__pkg__"],
|
||||
)
|
||||
|
||||
# //scripts:scripts_test checks the pinned golangci-lint version/sha256 in
|
||||
# engdocs/LINTING.md and the pre-commit hook's separate `go run` pin against
|
||||
# scripts/ci/install-golangci-lint.sh, so both must be readable as data.
|
||||
exports_files(
|
||||
[
|
||||
"engdocs/LINTING.md",
|
||||
".githooks/pre-commit",
|
||||
],
|
||||
visibility = ["//scripts:__pkg__"],
|
||||
)
|
||||
|
||||
# //scripts:scripts_test keeps pr.yml's release-target cross-compilation matrix
|
||||
# in lockstep with the release targets .goreleaser.yml declares.
|
||||
exports_files(
|
||||
|
|
|
|||
|
|
@ -486,7 +486,7 @@ Do not require these existing check names directly:
|
|||
- `Test (macos-latest)`
|
||||
- `Test (storage domain + uow)`
|
||||
- `Test (Dolt server fingerprint)`
|
||||
- `Go test (scripts), go vet and Bazel-skipped tests`
|
||||
- `Go checks (scripts-test)`, `Go checks (vet)` and `Go checks (allowlisted)`
|
||||
- `Contract corpus (golden + determinism + conformance)`
|
||||
- `PR Core (wrapper timing)`
|
||||
- `Build Artifacts`
|
||||
|
|
@ -500,7 +500,7 @@ Do not require these existing check names directly:
|
|||
- `Test (Server Dolt Full Suite 1/16)` through `Test (Server Dolt Full Suite 16/16)`
|
||||
- `Test (Windows - smoke)`
|
||||
- `Check formatting`
|
||||
- `Lint`
|
||||
- `PR Lint (native)`, `PR Lint (windows)` and `PR Lint (darwin)`
|
||||
- `Test Nix Flake`
|
||||
- `Differential Regression (v0.49.6 baseline)`
|
||||
- `Upgrade smoke (<version> -> candidate)`
|
||||
|
|
|
|||
|
|
@ -181,7 +181,8 @@ Key jobs preserved by display name:
|
|||
- `Check version consistency`, `Check no duplicate migration versions`,
|
||||
`Check doc flags freshness`, and PR-only `Check for .beads changes`.
|
||||
- `PR Policy (wrapper timing)`, `PR Core (wrapper timing)`, and
|
||||
`PR Lint (wrapper timing)`.
|
||||
`PR Lint (wrapper timing)` (superseded by F5's 3-leg `PR Lint (native|windows|darwin)`
|
||||
matrix; this is a dated snapshot).
|
||||
- `Package Gate (MCP)`, `Package Gate (npm)`, and `Package Gate (website)`.
|
||||
- `Test (storage domain + uow)`.
|
||||
- `Build (Embedded Dolt)`, `Test (Embedded Dolt Storage N/5)`, and
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Linting Policy
|
||||
|
||||
Last reviewed: 2026-09-13
|
||||
Last reviewed: 2026-10-03
|
||||
|
||||
Freshness source: `.golangci.yml`, `scripts/ci/pr-lint.sh`, `scripts/pr-lint/`,
|
||||
`Makefile`, `.github/workflows/pr.yml`, and `.github/workflows/main.yml`.
|
||||
|
|
@ -9,18 +9,36 @@ This document explains the required Go lint gate for this codebase.
|
|||
|
||||
## Current Status
|
||||
|
||||
Lint is a required CI gate, and it runs in TWO LANES over the same pinned
|
||||
golangci-lint v2.10.1 release and the same `.golangci.yml`. Each must pass with
|
||||
zero issues in its own scope.
|
||||
Lint is a required CI gate: ONE job per lane (`pr-lint-wrapper`), over the same
|
||||
pinned golangci-lint v2.10.1 release and the same `.golangci.yml`. Each lane
|
||||
must pass with zero issues in its own scope.
|
||||
|
||||
- **The PR lane reports only what the PR introduces.** Both of its jobs are
|
||||
scoped to the diff against the merge base with `main`: the `lint` job passes
|
||||
`only-new-issues`, and `pr-lint-wrapper` runs the repository-owned
|
||||
`ci-pr-lint` wrapper with `BD_LINT_NEW_FROM_MERGE_BASE`. A finding in code the
|
||||
PR did not touch does not block it.
|
||||
- **The main lane sweeps the whole tree.** Both jobs run unscoped on every push
|
||||
to `main`, so a finding that lands there reds main's own run rather than every
|
||||
open PR.
|
||||
- **The PR lane reports only what the PR introduces.** `pr-lint-wrapper` runs
|
||||
the repository-owned `ci-pr-lint` wrapper with `BD_LINT_NEW_FROM_MERGE_BASE`
|
||||
set to the diff against the merge base with `main`. A finding in code the PR
|
||||
did not touch does not block it. "New" is measured off the diff, so MOVED
|
||||
CODE READS AS NEW: a pre-existing violation carried into a PR by a move or a
|
||||
rename is reported against that PR and blocks it. Fix it or `//nolint` it
|
||||
there — that is the accepted cost of the diff-scoped trade.
|
||||
- **The main lane sweeps the whole tree.** The same job runs on every push to
|
||||
`main` with `BD_LINT_NEW_FROM_MERGE_BASE` unset, so a finding that lands
|
||||
there reds main's own run rather than every open PR.
|
||||
|
||||
`pr-lint-wrapper` is a 3-leg matrix, `matrix.target: [native, windows, darwin]`
|
||||
(`fail-fast: false`), one leg per `scripts/pr-lint` pass (see below); the gate
|
||||
requires the matrix's aggregate result under the same `PR_LINT_WRAPPER` id as
|
||||
before. A former separate `Lint` job (the `golangci-lint-action` with
|
||||
`only-new-issues`) was functionally identical to this matrix's native leg
|
||||
(same binary, config, flags and diff scope) and was folded into it; see #5629.
|
||||
|
||||
Each leg installs golangci-lint from the pinned release binary
|
||||
(`scripts/ci/install-golangci-lint.sh`, sha256-verified) instead of
|
||||
`go install`, and restores a per-leg cache
|
||||
(`~/.cache/golangci-lint` plus a leg-specific `GOCACHE`) keyed on
|
||||
`.golangci.yml`, `go.sum` and the leg. A miss on the exact content key falls
|
||||
back to the leg's most recent entry; golangci-lint revalidates it against the
|
||||
current config and inputs, so a stale restore cannot mask an issue. pr.yml
|
||||
only ever restores; main.yml's matrix is the only saver, on an exact-key miss.
|
||||
|
||||
Run the wrapper locally with:
|
||||
|
||||
|
|
@ -52,9 +70,12 @@ failures remain attributable without duplicating target-selection policy in
|
|||
Bash.
|
||||
|
||||
The shared driver owns the lint passes used by `scripts/ci/pr-lint.sh` and
|
||||
Beads-source `bd preflight`. The standalone workflow lint action and pre-commit
|
||||
hook do not call it. The hook uses changed-lines scope, adds `--fix`, and omits
|
||||
the Windows cross-lint pass. Workflow consolidation remains tracked in #5629.
|
||||
Beads-source `bd preflight`. The pre-commit hook does not call it: the hook
|
||||
uses changed-lines scope, adds `--fix`, and omits the Windows cross-lint pass.
|
||||
The driver honors `BD_LINT_TARGETS` (a comma list of `native`, `windows` and
|
||||
`darwin`; default all three) so each CI matrix leg can run just its own pass;
|
||||
`make ci-pr-lint` and `bd preflight` both leave it unset and keep running all
|
||||
three.
|
||||
|
||||
The driver matches `.buildflags` when preparing `GOFLAGS`: appending
|
||||
`-tags=gms_pure_go` can override an inherited bare-Go tags value; it does not
|
||||
|
|
|
|||
|
|
@ -44,8 +44,10 @@ go_test(
|
|||
],
|
||||
data = [
|
||||
":scripts_files",
|
||||
"//:.githooks/pre-commit",
|
||||
"//:.goreleaser.yml",
|
||||
"//:bazel_policy_files",
|
||||
"//:engdocs/LINTING.md",
|
||||
"//:mcp_package_policy_files",
|
||||
"//:test_sh_files",
|
||||
"//:workflow_files",
|
||||
|
|
|
|||
90
scripts/ci/install-golangci-lint.sh
Executable file
90
scripts/ci/install-golangci-lint.sh
Executable file
|
|
@ -0,0 +1,90 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Install the pinned golangci-lint release binary instead of `go install`
|
||||
# (F5.2): the release tarball drops the install step from about 58s to about
|
||||
# 3s, because it is a straight download-and-verify rather than a compile.
|
||||
# CI-only: this script requires $RUNNER_TEMP and $GITHUB_PATH, so the local
|
||||
# `.githooks/pre-commit` hook keeps using
|
||||
# `go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.10.1`,
|
||||
# which needs no change here. `make ci-pr-lint` (scripts/pr-lint) looks up
|
||||
# whatever `golangci-lint` binary is on PATH; it does not use `go run`.
|
||||
#
|
||||
# Keep this version and the sha256 values in sync with CONTRIBUTING.md,
|
||||
# engdocs/LINTING.md, .github/workflows/pr.yml, .github/workflows/main.yml,
|
||||
# .github/workflows/ci-measurements.yml and .githooks/pre-commit. Verified
|
||||
# locally against the release's own golangci-lint-2.10.1-checksums.txt.
|
||||
readonly version="2.10.1"
|
||||
readonly max_attempts=3
|
||||
readonly retry_delay_seconds=5
|
||||
|
||||
declare -A sha256_by_arch=(
|
||||
[amd64]="dfa775874cf0561b404a02a8f4481fc69b28091da95aa697259820d429b09c99"
|
||||
[arm64]="6652b42ae02915eb2f9cb2a2e0cac99514c8eded8388d88ae3e06e1a52c00de8"
|
||||
)
|
||||
|
||||
: "${RUNNER_TEMP:?RUNNER_TEMP is required; this script is CI-only}"
|
||||
: "${GITHUB_PATH:?GITHUB_PATH is required; this script is CI-only}"
|
||||
|
||||
os="$(uname -s | tr '[:upper:]' '[:lower:]')"
|
||||
if [[ "$os" != "linux" ]]; then
|
||||
printf 'install-golangci-lint.sh only supports Linux runners, got: %s\n' "$os" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
arch="$(uname -m)"
|
||||
case "$arch" in
|
||||
x86_64 | amd64) arch="amd64" ;;
|
||||
aarch64 | arm64) arch="arm64" ;;
|
||||
*)
|
||||
printf 'Unsupported architecture for pinned golangci-lint install: %s\n' "$arch" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
expected_sha256="${sha256_by_arch[$arch]:?no pinned sha256 for arch: $arch}"
|
||||
readonly asset="golangci-lint-${version}-${os}-${arch}.tar.gz"
|
||||
readonly url="https://github.com/golangci/golangci-lint/releases/download/v${version}/${asset}"
|
||||
|
||||
workdir="$(mktemp -d)"
|
||||
trap 'rm -rf "$workdir"' EXIT
|
||||
|
||||
for ((attempt = 1; attempt <= max_attempts; attempt++)); do
|
||||
# See install-dolt.sh for why the status capture can't live in the `if`.
|
||||
status=0
|
||||
curl -fsSL -o "$workdir/$asset" "$url" || status=$?
|
||||
if ((status == 0)); then
|
||||
break
|
||||
fi
|
||||
if ((attempt == max_attempts)); then
|
||||
printf 'Failed to download %s after %d attempts (curl exit %d).\n' "$url" "$max_attempts" "$status" >&2
|
||||
exit "$status"
|
||||
fi
|
||||
printf 'Failed to download %s (attempt %d/%d); retrying in %d seconds.\n' \
|
||||
"$url" "$attempt" "$max_attempts" "$retry_delay_seconds" >&2
|
||||
sleep "$retry_delay_seconds"
|
||||
done
|
||||
|
||||
actual_sha256="$(sha256sum "$workdir/$asset" | awk '{print $1}')"
|
||||
if [[ "$actual_sha256" != "$expected_sha256" ]]; then
|
||||
printf '%s sha256 mismatch: got %s, want %s\n' "$asset" "$actual_sha256" "$expected_sha256" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
tar -xzf "$workdir/$asset" -C "$workdir"
|
||||
|
||||
install_dir="$RUNNER_TEMP/golangci-lint"
|
||||
mkdir -p "$install_dir"
|
||||
install -m 0755 "$workdir/golangci-lint-${version}-${os}-${arch}/golangci-lint" "$install_dir/golangci-lint"
|
||||
|
||||
echo "$install_dir" >> "$GITHUB_PATH"
|
||||
|
||||
# Fail loudly if the extracted binary does not report the pinned version: a
|
||||
# bad tarball layout would otherwise silently put a stale or wrong binary on
|
||||
# PATH. Compare the version token exactly, as install-dolt.sh does.
|
||||
installed="$("$install_dir/golangci-lint" version)"
|
||||
if [[ "$installed" != *"version $version"* ]]; then
|
||||
printf 'golangci-lint reports "%s", want version %s\n' "$installed" "$version" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'Installed pinned golangci-lint %s to %s (sha256 %s)\n' "$version" "$install_dir" "$actual_sha256"
|
||||
|
|
@ -13,7 +13,26 @@ source "$REPO_ROOT/scripts/ci/lib/timing.sh"
|
|||
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
ci_time "gofmt check" -- ./scripts/ci/fmt-check.sh
|
||||
# gofmt only has one pass (there is no windows- or darwin-only gofmt target),
|
||||
# so it belongs to the native leg of the pr.yml PR Lint matrix. Skip it when
|
||||
# BD_LINT_TARGETS names only windows and/or darwin; run it when the variable
|
||||
# is unset, empty, or names "native" (matching scripts/pr-lint's own default
|
||||
# of all three targets).
|
||||
should_run_fmt_check() {
|
||||
local IFS=','
|
||||
local target saw_target=0
|
||||
for target in ${BD_LINT_TARGETS:-}; do
|
||||
target="${target//[[:space:]]/}"
|
||||
[[ -z "$target" ]] && continue
|
||||
saw_target=1
|
||||
[[ "$target" == "native" ]] && return 0
|
||||
done
|
||||
((saw_target == 0))
|
||||
}
|
||||
|
||||
if should_run_fmt_check; then
|
||||
ci_time "gofmt check" -- ./scripts/ci/fmt-check.sh
|
||||
fi
|
||||
|
||||
# The checkout-owned Go driver is the single authority for the native and
|
||||
# cross-target lint arguments. Files guarded by //go:build windows && !cgo or
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import (
|
|||
"path/filepath"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
|
|
@ -37,8 +38,58 @@ func TestCIWorkflowArtifactOwnership(t *testing.T) {
|
|||
|
||||
assertJobRunsExactly(t, workflow.job(t, "pr-policy-wrapper"), "make ci-pr-policy")
|
||||
assertJobRunsExactly(t, workflow.job(t, "pr-lint-wrapper"), "make ci-pr-lint")
|
||||
|
||||
// F5.1: the duplicate whole-tree/diff-scoped `Lint` job (and its
|
||||
// LINT gate id, pr.yml only) is folded into pr-lint-wrapper.
|
||||
if _, found := workflow.Jobs["lint"]; found {
|
||||
t.Errorf("%s has a %q job; its only-new-issues pass duplicates pr-lint-wrapper's native leg (H7) and was folded into it", workflowName, "lint")
|
||||
}
|
||||
|
||||
// F5.2: pr-lint-wrapper is a 3-leg matrix keyed on matrix.target,
|
||||
// with BD_LINT_TARGETS telling scripts/pr-lint which legs to run.
|
||||
lintWrapper := workflow.job(t, "pr-lint-wrapper")
|
||||
if lintWrapper.Strategy.FailFast {
|
||||
t.Errorf("%s pr-lint-wrapper strategy.fail-fast = true, want false", workflowName)
|
||||
}
|
||||
if want := []string{"native", "windows", "darwin"}; !equalStrings(lintWrapper.Strategy.Matrix.Target, want) {
|
||||
t.Errorf("%s pr-lint-wrapper matrix.target = %v, want %v", workflowName, lintWrapper.Strategy.Matrix.Target, want)
|
||||
}
|
||||
if got := lintWrapper.step(t, "Run PR lint wrapper").Env["BD_LINT_TARGETS"]; got != "${{ matrix.target }}" {
|
||||
t.Errorf("%s pr-lint-wrapper BD_LINT_TARGETS = %q, want %q", workflowName, got, "${{ matrix.target }}")
|
||||
}
|
||||
// N-5: a dropped GOCACHE env here silently stops the lint cache
|
||||
// from doing anything (the step still passes; it just never
|
||||
// reuses or populates the restored/saved directory).
|
||||
if got, want := lintWrapper.step(t, "Run PR lint wrapper").Env["GOCACHE"], "${{ runner.temp }}/go-cache/lint-${{ matrix.target }}"; got != want {
|
||||
t.Errorf("%s pr-lint-wrapper GOCACHE = %q, want %q", workflowName, got, want)
|
||||
}
|
||||
if got := lintWrapper.step(t, "Install golangci-lint").Run; got != "./scripts/ci/install-golangci-lint.sh" {
|
||||
t.Errorf("%s pr-lint-wrapper installs golangci-lint with %q, want the pinned install script", workflowName, got)
|
||||
}
|
||||
// N-7.
|
||||
if lintWrapper.TimeoutMinutes != 20 {
|
||||
t.Errorf("%s pr-lint-wrapper timeout-minutes = %d, want 20 (matches scripts-go-checks)", workflowName, lintWrapper.TimeoutMinutes)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
if got := readCIWorkflow(t, "pr.yml").job(t, "pr-lint-wrapper").step(t, "Run PR lint wrapper").Env["BD_LINT_NEW_FROM_MERGE_BASE"]; got != "origin/main" {
|
||||
t.Errorf("pr.yml pr-lint-wrapper BD_LINT_NEW_FROM_MERGE_BASE = %q, want origin/main (PR lane stays diff-scoped)", got)
|
||||
}
|
||||
if got := readCIWorkflow(t, "main.yml").job(t, "pr-lint-wrapper").step(t, "Run PR lint wrapper").Env["BD_LINT_NEW_FROM_MERGE_BASE"]; got != "" {
|
||||
t.Errorf("main.yml pr-lint-wrapper BD_LINT_NEW_FROM_MERGE_BASE = %q, want unset", got)
|
||||
}
|
||||
// N-5: --new-from-merge-base in pr.yml's pr-lint-wrapper needs real
|
||||
// history; a dropped fetch-depth: 0 would fail loudly (M3, a NIT-grade
|
||||
// mutation per the review), but pin it anyway since it is part of the
|
||||
// same wrapper contract as the GOCACHE pin above.
|
||||
if got := readCIWorkflow(t, "pr.yml").job(t, "pr-lint-wrapper").Steps[0].With["fetch-depth"]; got != "0" {
|
||||
t.Errorf("pr.yml pr-lint-wrapper checkout fetch-depth = %q, want %q", got, "0")
|
||||
}
|
||||
// go-vet-cache (N-7).
|
||||
if job := readCIWorkflow(t, "main.yml").job(t, "go-vet-cache"); job.TimeoutMinutes != 20 {
|
||||
t.Errorf("main.yml go-vet-cache timeout-minutes = %d, want 20 (matches scripts-go-checks)", job.TimeoutMinutes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPRCIGateRequiresPolicyAndLintWrappers(t *testing.T) {
|
||||
|
|
@ -65,6 +116,223 @@ func TestPRCIGateRequiresPolicyAndLintWrappers(t *testing.T) {
|
|||
t.Errorf("ci-gate CI_GATE_REQUIRED does not include %q", required)
|
||||
}
|
||||
}
|
||||
|
||||
// F5.1/F5.4: the deleted `lint` job's LINT id must not linger in the gate.
|
||||
if contains(gate.Needs, "lint") {
|
||||
t.Errorf("ci-gate needs %q, which no longer exists: %v", "lint", gate.Needs)
|
||||
}
|
||||
if contains(strings.Fields(gateEnv["CI_GATE_REQUIRED"]), "LINT") {
|
||||
t.Error("ci-gate CI_GATE_REQUIRED still includes LINT")
|
||||
}
|
||||
if _, found := gateEnv["LINT"]; found {
|
||||
t.Error("ci-gate env still sets LINT")
|
||||
}
|
||||
}
|
||||
|
||||
// F5.2/F5.7: the pinned golangci-lint install script's version and sha256
|
||||
// values must stay in sync with LINTING.md's documented version and with
|
||||
// .githooks/pre-commit's `go run ...@<version>`, and the three workflows that
|
||||
// use the script (pr.yml, main.yml, ci-measurements.yml) must all call it
|
||||
// instead of `go install`.
|
||||
func TestGolangciLintInstallScriptPinned(t *testing.T) {
|
||||
root := sourceRepoRoot(t)
|
||||
script := readPolicyFile(t, root, "scripts/ci/install-golangci-lint.sh")
|
||||
|
||||
const version = "2.10.1"
|
||||
wantSHA256 := map[string]string{
|
||||
"amd64": "dfa775874cf0561b404a02a8f4481fc69b28091da95aa697259820d429b09c99",
|
||||
"arm64": "6652b42ae02915eb2f9cb2a2e0cac99514c8eded8388d88ae3e06e1a52c00de8",
|
||||
}
|
||||
|
||||
if !strings.Contains(script, `readonly version="`+version+`"`) {
|
||||
t.Errorf("install-golangci-lint.sh does not pin version %q", version)
|
||||
}
|
||||
for arch, sha := range wantSHA256 {
|
||||
if !strings.Contains(script, "["+arch+"]=\""+sha+"\"") {
|
||||
t.Errorf("install-golangci-lint.sh does not pin %s sha256 %q", arch, sha)
|
||||
}
|
||||
}
|
||||
|
||||
linting := readPolicyFile(t, root, "engdocs/LINTING.md")
|
||||
if !strings.Contains(linting, "v"+version) {
|
||||
t.Errorf("engdocs/LINTING.md does not mention v%s", version)
|
||||
}
|
||||
|
||||
preCommit := readPolicyFile(t, root, ".githooks/pre-commit")
|
||||
if !strings.Contains(preCommit, "golangci-lint/v2/cmd/golangci-lint@v"+version) {
|
||||
t.Errorf(".githooks/pre-commit does not pin golangci-lint @v%s", version)
|
||||
}
|
||||
// The hook stays on `go run` deliberately (CI-only RUNNER_TEMP/GITHUB_PATH
|
||||
// requirement); it must not switch to the install script.
|
||||
if strings.Contains(preCommit, "install-golangci-lint.sh") {
|
||||
t.Error(".githooks/pre-commit must keep using `go run`, not the CI-only install script")
|
||||
}
|
||||
|
||||
for _, workflow := range []string{"pr.yml", "main.yml"} {
|
||||
job := readCIWorkflow(t, workflow).job(t, "pr-lint-wrapper")
|
||||
if got := job.step(t, "Install golangci-lint").Run; got != "./scripts/ci/install-golangci-lint.sh" {
|
||||
t.Errorf("%s pr-lint-wrapper installs golangci-lint with %q, want the pinned install script", workflow, got)
|
||||
}
|
||||
}
|
||||
measurements := readPolicyFile(t, root, ".github/workflows/ci-measurements.yml")
|
||||
if strings.Contains(measurements, "go install github.com/golangci/golangci-lint") {
|
||||
t.Error("ci-measurements.yml still uses go install for golangci-lint; use scripts/ci/install-golangci-lint.sh")
|
||||
}
|
||||
if !strings.Contains(measurements, "install-golangci-lint.sh") {
|
||||
t.Error("ci-measurements.yml does not call scripts/ci/install-golangci-lint.sh")
|
||||
}
|
||||
|
||||
// SF-2: install-golangci-lint.sh and the "pr-lint wrapper" measurement run
|
||||
// in the same shell block (not separate steps), so appending to
|
||||
// $GITHUB_PATH (which only takes effect in later *steps*) is not enough;
|
||||
// the script's install_dir must also be exported into this step's own
|
||||
// PATH, after the install call and before the wrapper that needs it.
|
||||
const (
|
||||
installMarker = "install-golangci-lint.sh"
|
||||
pathExport = `export PATH="$RUNNER_TEMP/golangci-lint:$PATH"`
|
||||
wrapperMarker = `ci_time "pr-lint wrapper"`
|
||||
)
|
||||
installIdx := strings.Index(measurements, installMarker)
|
||||
pathIdx := strings.Index(measurements, pathExport)
|
||||
wrapperIdx := strings.Index(measurements, wrapperMarker)
|
||||
if pathIdx < 0 {
|
||||
t.Errorf("ci-measurements.yml does not export golangci-lint's install dir onto PATH in the same step (want %q)", pathExport)
|
||||
}
|
||||
if installIdx < 0 || wrapperIdx < 0 {
|
||||
t.Fatalf("ci-measurements.yml missing expected markers: install=%d wrapper=%d", installIdx, wrapperIdx)
|
||||
}
|
||||
if !(installIdx < pathIdx && pathIdx < wrapperIdx) {
|
||||
t.Errorf("ci-measurements.yml PATH export must sit between the golangci-lint install (%d) and the pr-lint wrapper call (%d); got PATH export at %d", installIdx, wrapperIdx, pathIdx)
|
||||
}
|
||||
}
|
||||
|
||||
// TestInstallGolangciLintFailsClosedOnChecksumMismatch is the regression net
|
||||
// for install-golangci-lint.sh's supply-chain gate (SF-3). Without it, nothing
|
||||
// exercises the sha256 comparison's behavior -- TestGolangciLintInstallScriptPinned
|
||||
// above only pins the hash strings as text, so e.g. replacing the comparison
|
||||
// with an always-false condition passes every other test in this package. A
|
||||
// fake curl hands the script a tarball whose content never matches either
|
||||
// pinned hash; a fake tar makes extraction succeed anyway (so a bypassed check
|
||||
// is exposed cleanly as "installed the wrong thing", not masked by tar
|
||||
// choking on bad bytes), which is exactly the gap a real supply-chain attack
|
||||
// (a compromised release asset, a MITM'd download) would exploit if the
|
||||
// comparison were missing or broken.
|
||||
func TestInstallGolangciLintFailsClosedOnChecksumMismatch(t *testing.T) {
|
||||
bash := requireHostTool(t, "bash")
|
||||
|
||||
bin := t.TempDir()
|
||||
stateDir := t.TempDir()
|
||||
runnerTemp := t.TempDir()
|
||||
curlLog := filepath.Join(stateDir, "curl-calls")
|
||||
tarLog := filepath.Join(stateDir, "tar-calls")
|
||||
githubPath := filepath.Join(stateDir, "github-path")
|
||||
for _, path := range []string{curlLog, tarLog, githubPath} {
|
||||
if err := os.WriteFile(path, nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// Writes fixed, wrong-checksum bytes to curl's -o target. Any download
|
||||
// (first attempt or a retry) "succeeds" the same way, so the script
|
||||
// reaches the sha256 comparison deterministically.
|
||||
writeExecutable(t, filepath.Join(bin, "curl"), `#!/bin/sh
|
||||
set -eu
|
||||
printf '%s\n' "$*" >>"$GOLANGCI_INSTALL_CURL_LOG"
|
||||
out=""
|
||||
prev=""
|
||||
for arg in "$@"; do
|
||||
if [ "$prev" = "-o" ]; then
|
||||
out="$arg"
|
||||
fi
|
||||
prev="$arg"
|
||||
done
|
||||
if [ -n "$out" ]; then
|
||||
printf 'not the real golangci-lint release asset\n' >"$out"
|
||||
fi
|
||||
`)
|
||||
// Materializes a plausible-looking extracted binary unconditionally, the
|
||||
// way a real tarball's contents would after a successful `tar -xzf`. The
|
||||
// script's sha256 check -- not tar's own format validation -- must be
|
||||
// what stands between this fake binary and $RUNNER_TEMP/golangci-lint.
|
||||
writeExecutable(t, filepath.Join(bin, "tar"), `#!/bin/sh
|
||||
set -eu
|
||||
printf '%s\n' "$*" >>"$GOLANGCI_INSTALL_TAR_LOG"
|
||||
outdir=""
|
||||
prev=""
|
||||
for arg in "$@"; do
|
||||
if [ "$prev" = "-C" ]; then
|
||||
outdir="$arg"
|
||||
fi
|
||||
prev="$arg"
|
||||
done
|
||||
mkdir -p "$outdir/golangci-lint-2.10.1-linux-amd64"
|
||||
cat >"$outdir/golangci-lint-2.10.1-linux-amd64/golangci-lint" <<'SCRIPT'
|
||||
#!/bin/sh
|
||||
printf 'golangci-lint has version 2.10.1 built from unknown\n'
|
||||
SCRIPT
|
||||
chmod +x "$outdir/golangci-lint-2.10.1-linux-amd64/golangci-lint"
|
||||
`)
|
||||
// Pins the platform so the asset name, arch lookup and extracted layout
|
||||
// are fixed regardless of the machine running the test.
|
||||
writeExecutable(t, filepath.Join(bin, "uname"), `#!/bin/sh
|
||||
set -eu
|
||||
case "${1:-}" in
|
||||
-m) printf 'x86_64\n' ;;
|
||||
*) printf 'Linux\n' ;;
|
||||
esac
|
||||
`)
|
||||
|
||||
pathEnv := shellPathEnv()
|
||||
binPath := shellPathUnderEnv(t, bash, bin, pathEnv)
|
||||
statePath := shellPathUnderEnv(t, bash, stateDir, pathEnv)
|
||||
commandPath := binPath + ":" + os.Getenv("PATH") + ":/usr/bin:/bin"
|
||||
if runtime.GOOS == "windows" {
|
||||
commandPath = binPath + ":/usr/bin:/bin"
|
||||
}
|
||||
root := sourceRepoRoot(t)
|
||||
env := []string{
|
||||
"PATH=" + os.Getenv("PATH"),
|
||||
"BEADS_TEST_COMMAND_PATH=" + commandPath,
|
||||
"LC_ALL=C",
|
||||
"LANG=C",
|
||||
"BASH_ENV=",
|
||||
"ENV=",
|
||||
"RUNNER_TEMP=" + shellPathUnderEnv(t, bash, runnerTemp, pathEnv),
|
||||
"GITHUB_PATH=" + statePath + "/github-path",
|
||||
"GOLANGCI_INSTALL_CURL_LOG=" + statePath + "/curl-calls",
|
||||
"GOLANGCI_INSTALL_TAR_LOG=" + statePath + "/tar-calls",
|
||||
}
|
||||
for _, name := range []string{"curl", "tar", "uname"} {
|
||||
requireShellCommandPath(t, bash, root, env, name, binPath+"/"+name)
|
||||
}
|
||||
|
||||
cmd := bashScriptCommand(bash, "scripts/ci/install-golangci-lint.sh")
|
||||
cmd.Dir = root
|
||||
cmd.Env = env
|
||||
output, runErr := cmd.CombinedOutput()
|
||||
|
||||
if code := pullDoltExitCode(runErr); code == 0 {
|
||||
t.Fatalf("exit = 0, want non-zero; output:\n%s", output)
|
||||
}
|
||||
if !strings.Contains(string(output), "sha256 mismatch") {
|
||||
t.Errorf("output does not contain %q:\n%s", "sha256 mismatch", output)
|
||||
}
|
||||
if calls := readCallLines(t, tarLog); len(calls) != 0 {
|
||||
t.Errorf("tar was invoked %d time(s), want 0 (the sha256 check must reject the download before extraction): %q", len(calls), calls)
|
||||
}
|
||||
if calls := readCallLines(t, curlLog); len(calls) == 0 {
|
||||
t.Fatalf("curl was never invoked; test did not exercise the script's download path")
|
||||
}
|
||||
if entries, err := os.ReadDir(runnerTemp); err != nil {
|
||||
t.Fatal(err)
|
||||
} else if len(entries) != 0 {
|
||||
t.Errorf("RUNNER_TEMP = %v, want empty (nothing installed on a checksum mismatch)", entries)
|
||||
}
|
||||
if data, err := os.ReadFile(githubPath); err != nil {
|
||||
t.Fatal(err)
|
||||
} else if len(data) != 0 {
|
||||
t.Errorf("GITHUB_PATH = %q, want empty (nothing appended on a checksum mismatch)", data)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPRCIGateRequiresReleaseTargetCrossCompilation pins the cross-compilation
|
||||
|
|
@ -898,6 +1166,12 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
assertGoCacheInventory(t, workflows["main.yml"].job(t, "test-windows"), []goCacheStep{
|
||||
mainRestoreModuleCache(), mainRestoreBuildCache("non-race"), saveModuleCache(), saveBuildCache("non-race"),
|
||||
})
|
||||
assertGoCacheInventory(t, workflows["main.yml"].job(t, "pr-lint-wrapper"), []goCacheStep{
|
||||
restoreModuleCache(), mainRestoreGolangciCache(), saveGolangciCache(),
|
||||
})
|
||||
assertGoCacheInventory(t, workflows["main.yml"].job(t, "go-vet-cache"), []goCacheStep{
|
||||
mainRestoreModuleCache(), mainRestoreVetCache(), saveVetCache(),
|
||||
})
|
||||
assertConditionalCacheWritersHaveMatrixMember(t, workflows["main.yml"].job(t, "test"), macOSRunner)
|
||||
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "build-artifacts"), []goCacheStep{
|
||||
|
|
@ -909,8 +1183,9 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
// D2 step 3: PR Core's environment for ./scripts/..., go vet and the
|
||||
// Bazel-skipped tests; read-only like PR Core.
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "scripts-go-checks"), []goCacheStep{
|
||||
restoreModuleCache(), restoreBuildCache("race"),
|
||||
restoreModuleCache(), restoreBuildCacheIf("race", "matrix.check == 'scripts-test'"), restoreVetCache(), restoreBuildCacheIf("non-race", "matrix.check == 'allowlisted'"),
|
||||
})
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "pr-lint-wrapper"), []goCacheStep{restoreModuleCache(), restoreGolangciCache()})
|
||||
assertGoCacheInventory(t, workflows["pr.yml"].job(t, "test-macos"), []goCacheStep{
|
||||
restoreModuleCache(), restoreBuildCache("non-race"), restoreBuildCache("race"),
|
||||
})
|
||||
|
|
@ -929,11 +1204,12 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
assertNoUnmanagedGoCacheSteps(t, workflows, map[string]map[string]bool{
|
||||
"main.yml": {
|
||||
"build-artifacts": true, "build-embedded": true, "pr-core-wrapper": true, "test": true, "test-windows": true,
|
||||
"pr-lint-wrapper": true, "go-vet-cache": true,
|
||||
},
|
||||
"pr.yml": {
|
||||
"build-artifacts": true, "pr-core-wrapper": true, "scripts-go-checks": true, "test-macos": true, "worktree-remove-windows": true,
|
||||
"check-doc-freshness-platforms": true, "pr-preflight-platforms": true, "build-examples": true,
|
||||
"check-release-target-cross-compilation": true,
|
||||
"check-release-target-cross-compilation": true, "pr-lint-wrapper": true,
|
||||
},
|
||||
"pr-risk.yml": {"build-embedded": true},
|
||||
})
|
||||
|
|
@ -1022,6 +1298,8 @@ func TestGoCacheOwnershipTopology(t *testing.T) {
|
|||
assertGoCacheWriter(t, workflows["main.yml"], "test", "${{ matrix.os }}", "Save race Go build cache", failureSurvivingCacheSaveCondition(macOSMatrixCondition, cacheMissCondition(goBuildCacheRestoreID("race"))))
|
||||
assertGoCacheWriter(t, workflows["main.yml"], "test-windows", "windows-latest", "Save Go module cache", cacheMissCondition(goModuleCacheRestoreID))
|
||||
assertGoCacheWriter(t, workflows["main.yml"], "test-windows", "windows-latest", "Save non-race Go build cache", cacheMissCondition(goBuildCacheRestoreID("non-race")))
|
||||
assertGoCacheWriter(t, workflows["main.yml"], "pr-lint-wrapper", "ubuntu-latest", "Save golangci-lint cache", failureSurvivingCacheSaveCondition(cacheMissCondition(golangciCacheRestoreID())))
|
||||
assertGoCacheWriter(t, workflows["main.yml"], "go-vet-cache", "ubuntu-latest", "Save vet Go build cache", failureSurvivingCacheSaveCondition(cacheMissCondition(goVetCacheRestoreID())))
|
||||
for _, target := range []struct{ workflow, job string }{
|
||||
{"main.yml", "build-artifacts"},
|
||||
{"main.yml", "build-embedded"},
|
||||
|
|
@ -1346,6 +1624,83 @@ func saveBuildCacheAfterFailureOnMacOS(profile string) goCacheStep {
|
|||
return step
|
||||
}
|
||||
|
||||
// The go-vet-cache family (F5.3): a dedicated, go-vet-only GOCACHE restored
|
||||
// by pr.yml's scripts-go-checks vet leg and warmed by main.yml's go-vet-cache
|
||||
// job on push. There is deliberately no non-race-build-cache fallback restore
|
||||
// key (SF-5): actions/cache versions each entry by its path list, and that
|
||||
// cache saves to a different path, so a prefix fallback naming it could never
|
||||
// hit.
|
||||
const goVetCacheSchema = "v1"
|
||||
|
||||
func goVetCachePrefix() string {
|
||||
return "beads-go-vet-" + goVetCacheSchema + "-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-base-" + goBaseTag + "-vet-"
|
||||
}
|
||||
|
||||
func goVetCacheKey() string { return goVetCachePrefix() + "${{ github.sha }}" }
|
||||
|
||||
func goVetCacheRestoreKeys() string { return goVetCachePrefix() }
|
||||
|
||||
func goVetCachePath() string { return "${{ runner.temp }}/go-cache/vet" }
|
||||
|
||||
func goVetCacheRestoreID() string { return "restore-vet-go-build-cache" }
|
||||
|
||||
func restoreVetCache() goCacheStep {
|
||||
return goCacheStep{name: "Restore vet Go build cache", family: cacheRestoreActionFamily, key: goVetCacheKey(), restoreKeys: goVetCacheRestoreKeys(), path: goVetCachePath(), ifCondition: "matrix.check == 'vet'"}
|
||||
}
|
||||
|
||||
func mainRestoreVetCache() goCacheStep {
|
||||
step := restoreVetCache()
|
||||
step.id = goVetCacheRestoreID()
|
||||
step.ifCondition = ""
|
||||
return step
|
||||
}
|
||||
|
||||
func saveVetCache() goCacheStep {
|
||||
return goCacheStep{name: "Save vet Go build cache", family: cacheSaveActionFamily, key: goVetCacheKey(), path: goVetCachePath(), ifCondition: failureSurvivingCacheSaveCondition(cacheMissCondition(goVetCacheRestoreID()))}
|
||||
}
|
||||
|
||||
// The golangci-lint cache family (F5.2): one cache per PR Lint matrix leg,
|
||||
// keyed on file content and the leg. A miss on the exact content key falls
|
||||
// back to the leg's prefix (its most recent entry); golangci-lint revalidates
|
||||
// a restored cache against the current config and inputs, so a stale restore
|
||||
// cannot mask an issue. pr.yml only ever restores; main.yml's matrix is the
|
||||
// only saver. The key deliberately omits the commit sha (SF-4): with it,
|
||||
// every green push to main saved a new multi-GB entry per leg regardless of
|
||||
// whether lint's inputs changed; keying on content alone means a save only
|
||||
// happens when .golangci.yml or go.sum change. Only the native leg's path
|
||||
// includes the lint GOCACHE (SF-4): the windows/darwin legs cross-compile and
|
||||
// their GOCACHE is large relative to the small golangci-lint result cache
|
||||
// that is the part actually worth keeping warm for them.
|
||||
func golangciCachePrefix() string {
|
||||
return "beads-golangci-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-v2.10.1-${{ matrix.target }}-"
|
||||
}
|
||||
|
||||
func golangciCacheKey() string {
|
||||
return golangciCachePrefix() + "${{ hashFiles('.golangci.yml', 'go.sum') }}"
|
||||
}
|
||||
|
||||
func golangciCacheRestoreKeys() string { return golangciCachePrefix() }
|
||||
|
||||
func golangciCachePath() string {
|
||||
return "~/.cache/golangci-lint\n${{ matrix.target == 'native' && format('{0}/go-cache/lint-{1}', runner.temp, matrix.target) || '' }}\n"
|
||||
}
|
||||
|
||||
func golangciCacheRestoreID() string { return "restore-golangci-lint-cache" }
|
||||
|
||||
func restoreGolangciCache() goCacheStep {
|
||||
return goCacheStep{name: "Restore golangci-lint cache", family: cacheRestoreActionFamily, key: golangciCacheKey(), restoreKeys: golangciCacheRestoreKeys(), path: golangciCachePath()}
|
||||
}
|
||||
|
||||
func mainRestoreGolangciCache() goCacheStep {
|
||||
step := restoreGolangciCache()
|
||||
step.id = golangciCacheRestoreID()
|
||||
return step
|
||||
}
|
||||
|
||||
func saveGolangciCache() goCacheStep {
|
||||
return goCacheStep{name: "Save golangci-lint cache", family: cacheSaveActionFamily, key: golangciCacheKey(), path: golangciCachePath(), ifCondition: failureSurvivingCacheSaveCondition(cacheMissCondition(golangciCacheRestoreID()))}
|
||||
}
|
||||
|
||||
func assertGoCacheInventory(t *testing.T, job ciWorkflowJob, want []goCacheStep) {
|
||||
t.Helper()
|
||||
|
||||
|
|
@ -1546,6 +1901,8 @@ type ciWorkflowStrategy struct {
|
|||
type ciWorkflowMatrix struct {
|
||||
OS []string `yaml:"os"`
|
||||
Shard []int `yaml:"shard"`
|
||||
Target []string `yaml:"target"`
|
||||
Check []string `yaml:"check"`
|
||||
Include []ciWorkflowMatrixInclude `yaml:"include"`
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -27,6 +27,11 @@ const (
|
|||
lintReportedTimeout = "5m"
|
||||
)
|
||||
|
||||
// defaultLintTargets is BD_LINT_TARGETS' value when unset or empty: the
|
||||
// no-argument usage contract (`make ci-pr-lint`, `bd preflight`) must keep
|
||||
// running all three passes locally and from the pre-commit hook.
|
||||
var defaultLintTargets = []string{"native", "windows", "darwin"}
|
||||
|
||||
type commandSpec struct {
|
||||
name string
|
||||
args []string
|
||||
|
|
@ -89,6 +94,16 @@ func run(args []string, dir string, environ []string, stdout, stderr io.Writer,
|
|||
return 2
|
||||
}
|
||||
|
||||
rawTargets, _ := environmentValue(environ, "BD_LINT_TARGETS", runtime.GOOS == "windows")
|
||||
targets, err := parseLintTargets(rawTargets)
|
||||
if err != nil {
|
||||
fmt.Fprintln(stderr, err)
|
||||
return 2
|
||||
}
|
||||
runNative := containsTarget(targets, "native")
|
||||
runWindows := containsTarget(targets, "windows")
|
||||
runDarwin := containsTarget(targets, "darwin")
|
||||
|
||||
goPath, err := runner.lookPath("go")
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "Go toolchain not found in PATH: %v\n", err)
|
||||
|
|
@ -130,7 +145,7 @@ func run(args []string, dir string, environ []string, stdout, stderr io.Writer,
|
|||
nativeEnv := effectiveEnv
|
||||
if runtime.GOOS == "windows" {
|
||||
nativeEnv = preferSelectedGo(probeCtx, dir, effectiveEnv, native, stderr, runner)
|
||||
if !skipWindows {
|
||||
if runWindows && !skipWindows {
|
||||
// GOWORK=off can select a different toolchain. Probe the original
|
||||
// Go executable and PATH again, before either lint pass starts.
|
||||
selected, code := readNativeGoEnvironment(probeCtx, dir, windowsEnv, goPath, stderr, runner)
|
||||
|
|
@ -138,7 +153,7 @@ func run(args []string, dir string, environ []string, stdout, stderr io.Writer,
|
|||
windowsEnv = preferSelectedGo(probeCtx, dir, windowsEnv, selected, stderr, runner)
|
||||
}
|
||||
}
|
||||
if !skipDarwin {
|
||||
if runDarwin && !skipDarwin {
|
||||
selected, code := readNativeGoEnvironment(probeCtx, dir, darwinEnv, goPath, stderr, runner)
|
||||
if code == 0 {
|
||||
darwinEnv = preferSelectedGo(probeCtx, dir, darwinEnv, selected, stderr, runner)
|
||||
|
|
@ -146,28 +161,36 @@ func run(args []string, dir string, environ []string, stdout, stderr io.Writer,
|
|||
}
|
||||
}
|
||||
cancel() // All discovery shares the existing 30-second probe budget.
|
||||
if code := runLintPass(
|
||||
"golangci-lint (native)",
|
||||
commandSpec{name: lintPath, args: argsForLint, dir: dir, env: nativeEnv},
|
||||
stdout,
|
||||
stderr,
|
||||
runner,
|
||||
); code != 0 {
|
||||
return code
|
||||
|
||||
if runNative {
|
||||
if code := runLintPass(
|
||||
"golangci-lint (native)",
|
||||
commandSpec{name: lintPath, args: argsForLint, dir: dir, env: nativeEnv},
|
||||
stdout,
|
||||
stderr,
|
||||
runner,
|
||||
); code != 0 {
|
||||
return code
|
||||
}
|
||||
}
|
||||
|
||||
if skipWindows {
|
||||
fmt.Fprintln(stdout, "==> golangci-lint (windows/amd64, non-CGO) already covered by native pass")
|
||||
} else if code := runLintPass(
|
||||
"golangci-lint (windows/amd64, non-CGO)",
|
||||
commandSpec{name: lintPath, args: argsForLint, dir: dir, env: windowsEnv},
|
||||
stdout,
|
||||
stderr,
|
||||
runner,
|
||||
); code != 0 {
|
||||
return code
|
||||
if runWindows {
|
||||
if skipWindows {
|
||||
fmt.Fprintln(stdout, "==> golangci-lint (windows/amd64, non-CGO) already covered by native pass")
|
||||
} else if code := runLintPass(
|
||||
"golangci-lint (windows/amd64, non-CGO)",
|
||||
commandSpec{name: lintPath, args: argsForLint, dir: dir, env: windowsEnv},
|
||||
stdout,
|
||||
stderr,
|
||||
runner,
|
||||
); code != 0 {
|
||||
return code
|
||||
}
|
||||
}
|
||||
|
||||
if !runDarwin {
|
||||
return 0
|
||||
}
|
||||
if skipDarwin {
|
||||
fmt.Fprintln(stdout, "==> golangci-lint (darwin/arm64, non-CGO) already covered by native pass")
|
||||
return 0
|
||||
|
|
@ -182,6 +205,39 @@ func run(args []string, dir string, environ []string, stdout, stderr io.Writer,
|
|||
)
|
||||
}
|
||||
|
||||
// parseLintTargets parses BD_LINT_TARGETS: a comma-separated list of
|
||||
// "native", "windows" and "darwin", in any order and with any repetition.
|
||||
// An empty value (the variable unset, or set to only whitespace and commas)
|
||||
// selects all three, so the no-argument usage contract is unchanged. Any
|
||||
// other token is an error: callers exit 2, matching the usage-error exit
|
||||
// code above.
|
||||
func parseLintTargets(raw string) ([]string, error) {
|
||||
var targets []string
|
||||
for _, field := range strings.Split(raw, ",") {
|
||||
target := strings.TrimSpace(field)
|
||||
if target == "" {
|
||||
continue
|
||||
}
|
||||
if !containsTarget(defaultLintTargets, target) {
|
||||
return nil, fmt.Errorf("BD_LINT_TARGETS: unknown target %q (want a comma list of native, windows, darwin)", target)
|
||||
}
|
||||
targets = append(targets, target)
|
||||
}
|
||||
if len(targets) == 0 {
|
||||
return defaultLintTargets, nil
|
||||
}
|
||||
return targets, nil
|
||||
}
|
||||
|
||||
func containsTarget(targets []string, target string) bool {
|
||||
for _, candidate := range targets {
|
||||
if candidate == target {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func readNativeGoEnvironment(
|
||||
ctx context.Context,
|
||||
dir string,
|
||||
|
|
|
|||
|
|
@ -142,6 +142,136 @@ func TestRunUsesCanonicalNativeAndCrossTargetPasses(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestRunHonorsBDLintTargetsSelection covers the pr.yml PR Lint matrix
|
||||
// (F5.2): each leg sets BD_LINT_TARGETS to exactly one of native, windows or
|
||||
// darwin, and only that pass must run.
|
||||
func TestRunHonorsBDLintTargetsSelection(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
targets string
|
||||
wantRuns []string // GOOS set on the one command's env, in order
|
||||
}{
|
||||
{"native", []string{"linux"}},
|
||||
{"windows", []string{"windows"}},
|
||||
{"darwin", []string{"darwin"}},
|
||||
{"native,windows", []string{"linux", "windows"}},
|
||||
{" windows , darwin ", []string{"windows", "darwin"}},
|
||||
{"windows,windows", []string{"windows"}},
|
||||
} {
|
||||
t.Run(tc.targets, func(t *testing.T) {
|
||||
runner := &recordingRunner{
|
||||
paths: map[string]string{
|
||||
"go": "/tools/go",
|
||||
"golangci-lint": "/tools/golangci-lint",
|
||||
},
|
||||
goEnvOutput: `{"GOOS":"linux","CGO_ENABLED":"1"}`,
|
||||
}
|
||||
var stdout, stderr bytes.Buffer
|
||||
environ := []string{"PATH=/tools", "BD_LINT_TARGETS=" + tc.targets}
|
||||
|
||||
code := run(nil, "/repo", environ, &stdout, &stderr, runner)
|
||||
if code != 0 {
|
||||
t.Fatalf("run exit = %d, want 0; stderr=%s", code, stderr.String())
|
||||
}
|
||||
if len(runner.runCommands) != len(tc.wantRuns) {
|
||||
t.Fatalf("lint calls = %d, want %d (%v); commands=%#v", len(runner.runCommands), len(tc.wantRuns), tc.wantRuns, runner.runCommands)
|
||||
}
|
||||
for index, wantGOOS := range tc.wantRuns {
|
||||
if wantGOOS == "linux" {
|
||||
// The native pass does not force a GOOS override.
|
||||
if got, found := environmentValue(runner.runCommands[index].env, "GOOS", false); found && got != "" && got != "linux" {
|
||||
t.Fatalf("native pass GOOS override = %q", got)
|
||||
}
|
||||
continue
|
||||
}
|
||||
assertEnvironmentValue(t, runner.runCommands[index].env, "GOOS", wantGOOS, false)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunDefaultsBDLintTargetsToAllThree(t *testing.T) {
|
||||
runner := &recordingRunner{
|
||||
paths: map[string]string{
|
||||
"go": "/tools/go",
|
||||
"golangci-lint": "/tools/golangci-lint",
|
||||
},
|
||||
goEnvOutput: `{"GOOS":"linux","CGO_ENABLED":"1"}`,
|
||||
}
|
||||
var stdout, stderr bytes.Buffer
|
||||
// No BD_LINT_TARGETS at all, and a second case with it set but empty:
|
||||
// both must run the full native+windows+darwin contract (the no-argument
|
||||
// usage contract `make ci-pr-lint` and `bd preflight` depend on).
|
||||
for _, environ := range [][]string{
|
||||
{"PATH=/tools"},
|
||||
{"PATH=/tools", "BD_LINT_TARGETS="},
|
||||
{"PATH=/tools", "BD_LINT_TARGETS= , ,"},
|
||||
} {
|
||||
runner.runCommands = nil
|
||||
code := run(nil, "/repo", environ, &stdout, &stderr, runner)
|
||||
if code != 0 {
|
||||
t.Fatalf("run exit = %d, want 0; stderr=%s", code, stderr.String())
|
||||
}
|
||||
if len(runner.runCommands) != 3 {
|
||||
t.Fatalf("env=%v: lint calls = %d, want 3", environ, len(runner.runCommands))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRejectsUnknownBDLintTarget(t *testing.T) {
|
||||
runner := &recordingRunner{
|
||||
paths: map[string]string{
|
||||
"go": "/tools/go",
|
||||
"golangci-lint": "/tools/golangci-lint",
|
||||
},
|
||||
goEnvOutput: `{"GOOS":"linux","CGO_ENABLED":"1"}`,
|
||||
}
|
||||
var stdout, stderr bytes.Buffer
|
||||
environ := []string{"PATH=/tools", "BD_LINT_TARGETS=native,solaris"}
|
||||
|
||||
code := run(nil, "/repo", environ, &stdout, &stderr, runner)
|
||||
if code != 2 {
|
||||
t.Fatalf("run exit = %d, want 2; stderr=%s", code, stderr.String())
|
||||
}
|
||||
if len(runner.runCommands) != 0 {
|
||||
t.Fatalf("an unknown target must run nothing, got %d lint calls", len(runner.runCommands))
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "solaris") {
|
||||
t.Fatalf("missing the unknown target in the diagnostic: %q", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseLintTargets(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
raw string
|
||||
want []string
|
||||
wantErr bool
|
||||
}{
|
||||
{"", []string{"native", "windows", "darwin"}, false},
|
||||
{" , ", []string{"native", "windows", "darwin"}, false},
|
||||
{"native", []string{"native"}, false},
|
||||
{"native,darwin", []string{"native", "darwin"}, false},
|
||||
{" native , darwin ", []string{"native", "darwin"}, false},
|
||||
{"bogus", nil, true},
|
||||
{"native,bogus", nil, true},
|
||||
} {
|
||||
t.Run(tc.raw, func(t *testing.T) {
|
||||
got, err := parseLintTargets(tc.raw)
|
||||
if tc.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("parseLintTargets(%q) = %v, want an error", tc.raw, got)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("parseLintTargets(%q) unexpected error: %v", tc.raw, err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, tc.want) {
|
||||
t.Fatalf("parseLintTargets(%q) = %#v, want %#v", tc.raw, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLintArgsKeepsHostileMergeBaseInOneArgument(t *testing.T) {
|
||||
mergeBase := `origin/main; printf injected >&2`
|
||||
args := lintArgs(mergeBase)
|
||||
|
|
|
|||
|
|
@ -575,6 +575,14 @@ func TestPRRunsGoTestsBazelSkips(t *testing.T) {
|
|||
t.Errorf("%s: if %q, continue-on-error %v, needs %v, runs-on %q, timeout %d; want an unconditional ubuntu-latest job with a timeout",
|
||||
prScriptsChecksJob, job.If, job.ContinueOnError, job.Needs, job.RunsOn, job.TimeoutMinutes)
|
||||
}
|
||||
// F5.3: three legs behind matrix.check, not fail-fast (a vet regression
|
||||
// must not hide an allowlisted regression, or vice versa).
|
||||
if job.Strategy.FailFast {
|
||||
t.Errorf("%s strategy.fail-fast = true, want false", prScriptsChecksJob)
|
||||
}
|
||||
if want := []string{"scripts-test", "vet", "allowlisted"}; !equalStrings(job.Strategy.Matrix.Check, want) {
|
||||
t.Errorf("%s matrix.check = %v, want %v", prScriptsChecksJob, job.Strategy.Matrix.Check, want)
|
||||
}
|
||||
var names []string
|
||||
for _, st := range job.Steps {
|
||||
names = append(names, st.Name)
|
||||
|
|
@ -582,27 +590,45 @@ func TestPRRunsGoTestsBazelSkips(t *testing.T) {
|
|||
t.Errorf("%s step %q has continue-on-error", prScriptsChecksJob, st.Name)
|
||||
}
|
||||
}
|
||||
wantNames := []string{"", "Set up Go", "Restore Go module cache", "Restore race Go build cache", "Install Dolt",
|
||||
"Configure Git and Dolt identity", "Go test the scripts packages", "Go vet with go test's checks", prAllowlistedStep}
|
||||
wantNames := []string{"", "Set up Go", "Restore Go module cache", "Restore race Go build cache", "Restore vet Go build cache",
|
||||
"Restore non-race Go build cache", "Install Dolt", "Configure Git and Dolt identity", "Go test the scripts packages",
|
||||
"Go vet with go test's checks", prAllowlistedStep}
|
||||
if !reflect.DeepEqual(names, wantNames) {
|
||||
t.Errorf("%s steps %q, want %q", prScriptsChecksJob, names, wantNames)
|
||||
}
|
||||
// The environment PR Core's job gives its go test.
|
||||
// The environment PR Core's job gives its go test (the race leg's cache
|
||||
// restore and Dolt setup stay byte-for-byte the same as pr-core-wrapper's).
|
||||
core := pr.job(t, "pr-core-wrapper")
|
||||
for _, name := range []string{"Install Dolt", "Configure Git and Dolt identity", "Restore race Go build cache"} {
|
||||
if got, want := job.step(t, name), core.step(t, name); got.Run != want.Run || !reflect.DeepEqual(got.With, want.With) || got.Uses != want.Uses {
|
||||
t.Errorf("%s step %q differs from pr-core-wrapper's", prScriptsChecksJob, name)
|
||||
}
|
||||
}
|
||||
untilDone := "${{ !cancelled() && steps.setup-go.outcome == 'success' }}"
|
||||
legIf := func(checks ...string) string {
|
||||
parts := make([]string, len(checks))
|
||||
for i, c := range checks {
|
||||
parts[i] = "matrix.check == '" + c + "'"
|
||||
}
|
||||
return strings.Join(parts, " || ")
|
||||
}
|
||||
// Dolt is only needed by the legs that use it; the vet leg skips it.
|
||||
for _, name := range []string{"Install Dolt", "Configure Git and Dolt identity"} {
|
||||
if got, want := job.step(t, name).If, legIf("scripts-test", "allowlisted"); got != want {
|
||||
t.Errorf("%s step %q if = %q, want %q", prScriptsChecksJob, name, got, want)
|
||||
}
|
||||
}
|
||||
for name, want := range map[string]struct {
|
||||
run, ifc string
|
||||
env map[string]string
|
||||
}{
|
||||
"Go test the scripts packages": {"bash scripts/ci/scripts-go-test.sh", "", map[string]string{
|
||||
"Go test the scripts packages": {"bash scripts/ci/scripts-go-test.sh", legIf("scripts-test"), map[string]string{
|
||||
"BEADS_TEST_REQUIRE_EXCLUDE_PERMISSION": "1", "GOCACHE": "${{ runner.temp }}/go-cache/race"}},
|
||||
"Go vet with go test's checks": {"bash scripts/ci/go-test-vet.sh", untilDone, nil},
|
||||
prAllowlistedStep: {"bash scripts/ci/allowlisted-go-tests.sh", untilDone, nil},
|
||||
"Go vet with go test's checks": {"bash scripts/ci/go-test-vet.sh",
|
||||
"${{ !cancelled() && steps.setup-go.outcome == 'success' && matrix.check == 'vet' }}",
|
||||
map[string]string{"GOCACHE": "${{ runner.temp }}/go-cache/vet"}},
|
||||
prAllowlistedStep: {"bash scripts/ci/allowlisted-go-tests.sh",
|
||||
"${{ !cancelled() && steps.setup-go.outcome == 'success' && matrix.check == 'allowlisted' }}",
|
||||
map[string]string{"GOCACHE": "${{ runner.temp }}/go-cache/non-race"}},
|
||||
} {
|
||||
st := job.step(t, name)
|
||||
if st.Run != want.run || st.If != want.ifc || st.Shell != "" || (len(st.Env) != 0 || len(want.env) != 0) && !reflect.DeepEqual(st.Env, want.env) {
|
||||
|
|
@ -625,6 +651,21 @@ func TestPRRunsGoTestsBazelSkips(t *testing.T) {
|
|||
}
|
||||
}
|
||||
}
|
||||
// main.yml's push-only go-vet-cache job (F5.3) is the one explicit
|
||||
// exception: it warms the vet cache pr.yml's vet leg restores, and
|
||||
// nothing else in main.yml may run these scripts either.
|
||||
for name, j := range readCIWorkflow(t, "main.yml").Jobs {
|
||||
for _, st := range j.Steps {
|
||||
for _, script := range []string{"allowlisted-go-tests.sh", "scripts-go-test.sh", "go-test-vet.sh"} {
|
||||
if strings.Contains(st.Run, script) && name != "go-vet-cache" {
|
||||
t.Errorf("main.yml %s step %q runs %s; only go-vet-cache may (go-test-vet.sh)", name, st.Name, script)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if got := readCIWorkflow(t, "main.yml").job(t, "go-vet-cache").step(t, "Go vet with go test's checks").Run; got != "bash scripts/ci/go-test-vet.sh" {
|
||||
t.Errorf("main.yml go-vet-cache does not run go-test-vet.sh: %q", got)
|
||||
}
|
||||
if os.Getenv("TEST_SRCDIR") != "" {
|
||||
return // scripts_test's runfiles hold none of the scripts (this part runs in that job itself)
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue