pr.yml and pr-risk.yml (and the three advisory workflows that already list release/** beside it -- conformance.yml, cross-version-smoke.yml, regression.yml) trigger pull_request only on main and release/**. A PR based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1, showed 17 checks -- 15 cross-version historical smokes and triage/label jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI Gate / Required, no PR Risk Gate / Required. pr.yml's own comment states exactly why release/** is there: release-prep and release-fix PRs are based on the release branch, not main, and without it they run almost no CI before the tag build. A hotfix line is in the identical position. Added hotfix/** alongside release/** in all five files. New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/** presence in each file's pull_request.branches list -- red before this change (observed [main release/**] in all five), green after. The existing TestPRRiskBazelCoverageJob equality check (pr.yml and pr-risk.yml's pull_request triggers must match) still passes, since both were updated identically. go vet/gofmt clean. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
177 lines
8.4 KiB
YAML
177 lines
8.4 KiB
YAML
name: Regression Tests
|
|
|
|
on:
|
|
# Run on merges to main
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main, 'release/**', 'hotfix/**']
|
|
# Allow manual trigger
|
|
workflow_dispatch:
|
|
|
|
env:
|
|
BD_DISABLE_METRICS: "1"
|
|
BD_DISABLE_EVENT_FLUSH: "1"
|
|
|
|
jobs:
|
|
regression:
|
|
name: Differential Regression (v0.49.6 baseline)
|
|
# F7c: detect-regression-need folded in as this job's own first step
|
|
# (id: detect). A non-applicable PR now costs one short Blacksmith-sized
|
|
# job instead of two GitHub-hosted jobs; every later step is gated on
|
|
# steps.detect.outputs.run_regression instead of a separate job's output.
|
|
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
|
|
timeout-minutes: 45
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Decide whether to run differential regression tests
|
|
id: detect
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
run: |
|
|
set -euo pipefail
|
|
|
|
if [[ "$EVENT_NAME" != "pull_request" ]]; then
|
|
echo "run_regression=true" >> "$GITHUB_OUTPUT"
|
|
echo "reason=$EVENT_NAME event always runs regression tests" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
|
|
labels="$(jq -r '.pull_request.labels[].name // empty' "$GITHUB_EVENT_PATH")"
|
|
if grep -Fxq "run-regression" <<<"$labels"; then
|
|
echo "run_regression=true" >> "$GITHUB_OUTPUT"
|
|
echo "reason=run-regression label present" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
if grep -Fxq "skip-regression" <<<"$labels"; then
|
|
echo "run_regression=false" >> "$GITHUB_OUTPUT"
|
|
echo "reason=skip-regression label present" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
|
|
changed="$(git diff --name-only "$PR_BASE_SHA" "$PR_HEAD_SHA")"
|
|
echo "Changed files:"
|
|
printf '%s\n' "$changed"
|
|
|
|
regression_candidates="$(
|
|
printf '%s\n' "$changed" |
|
|
grep -Ev '^(cmd/bd|internal/storage|internal/types)/.*_test\.go$' ||
|
|
true
|
|
)"
|
|
|
|
risky_pattern='^(\.github/workflows/regression\.yml|Makefile|go\.(mod|sum)|cmd/bd/|internal/storage/|internal/types/|tests/regression/)'
|
|
if grep -Eq "$risky_pattern" <<<"$regression_candidates"; then
|
|
echo "run_regression=true" >> "$GITHUB_OUTPUT"
|
|
echo "reason=PR changes CLI, storage, types, regression, or build inputs" >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
|
|
echo "run_regression=false" >> "$GITHUB_OUTPUT"
|
|
echo "reason=PR does not touch regression-sensitive paths" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Show regression trigger reason
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: echo "${{ steps.detect.outputs.reason }}"
|
|
|
|
- name: Set up Go
|
|
id: setup-go
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
cache: ${{ runner.environment == 'github-hosted' }}
|
|
|
|
# F7c review fix S5: this job's own key copy must match main.yml's
|
|
# seeder key byte-for-byte (TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers),
|
|
# which now keys by UTC calendar day instead of github.sha so main
|
|
# pushes don't churn a multi-GB cache entry on every commit; this step
|
|
# computes that same day value locally so the key expression below
|
|
# resolves (steps.* references only resolve within the same job).
|
|
- name: Compute cache date
|
|
id: cache-date
|
|
if: steps.detect.outputs.run_regression == 'true' && runner.environment == 'self-hosted'
|
|
run: echo "today=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Restore Blacksmith setup-go cache
|
|
if: steps.detect.outputs.run_regression == 'true' && runner.environment == 'self-hosted'
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
|
|
with:
|
|
path: |
|
|
~/go/pkg/mod
|
|
~/.cache/go-build
|
|
key: blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-${{ steps.cache-date.outputs.today }}
|
|
restore-keys: |
|
|
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-
|
|
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
|
|
|
|
- name: Configure Git
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: |
|
|
git config --global user.name "CI Bot"
|
|
git config --global user.email "ci@beads.test"
|
|
|
|
- name: Install Dolt
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: ./scripts/ci/install-dolt.sh
|
|
|
|
- name: Configure Dolt
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: |
|
|
dolt config --global --add user.name "CI Bot"
|
|
dolt config --global --add user.email "ci@beads.test"
|
|
dolt version
|
|
|
|
- name: Pull Dolt sql-server image
|
|
# Without this, isDoltImageCached() returns false in TestMain and the
|
|
# regression suite falls back to embedded Dolt with HOME=<tmpdir>, which
|
|
# has a known cleanup race (see beads issue: TestCreateWithParentFlag
|
|
# TempDir RemoveAll "directory not empty" flake). Keep the tag in sync
|
|
# with internal/testutil/testdoltcommon.go:DoltDockerImage.
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: ./scripts/ci/pull-dolt-image.sh
|
|
|
|
# Restore-only: actions/cache (bare) auto-saves on any key miss, which
|
|
# would let a same-repo PR that bumps BASELINE_VERSION write a baseline
|
|
# binary cache entry a later PR or main could read back unverified. The
|
|
# matching save below only runs off pull_request (B2, F7c implementation
|
|
# report).
|
|
- name: Restore baseline binary cache
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
|
|
with:
|
|
path: ~/.cache/beads-regression
|
|
key: regression-baseline-${{ hashFiles('tests/regression/BASELINE_VERSION') }}
|
|
|
|
# Passing-side wall time already overlapped the go deadline below, so
|
|
# ordinary GitHub-runner speed variance -- not a real hang -- was
|
|
# tripping this step. Measured over the 100 most recent Regression
|
|
# Tests runs (2026-09-03/04, ~29.5h): p50 593s (~10m), max passing
|
|
# 1189s (~19.8m) against the prior 1200s (20m) cap, and 5 runs in that
|
|
# window hit the identical `panic: test timed out after 20m0s` with
|
|
# zero `--- FAIL` lines -- a budget defect, not a regression. Match
|
|
# main.yml's Test job fix for the same failure mode: raise the go
|
|
# deadline to cover the observed slowdown and keep it inside a larger
|
|
# finite job budget (see timeout-minutes above), so the go deadline
|
|
# fires first and prints goroutine stacks -- a REAL hang stays
|
|
# diagnosable instead of being cut off by a bare job kill.
|
|
- name: Run regression tests
|
|
if: steps.detect.outputs.run_regression == 'true'
|
|
run: go test -tags=regression,gms_pure_go -timeout=40m -v ./tests/regression/...
|
|
|
|
- name: Save baseline binary cache
|
|
# Allow-list push/workflow_dispatch rather than deny-list
|
|
# pull_request: a deny-list also admits a hypothetical future
|
|
# merge_group event, which this job's runs-on expression already
|
|
# resolves to Blacksmith for. Pinned by
|
|
# TestBlacksmithReachableAdvisoryJobsNeverSaveACache (checks both the
|
|
# pull_request and merge_group trust contexts independently).
|
|
if: steps.detect.outputs.run_regression == 'true' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
|
|
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
|
|
with:
|
|
path: ~/.cache/beads-regression
|
|
key: regression-baseline-${{ hashFiles('tests/regression/BASELINE_VERSION') }}
|