beads/.github/workflows/regression.yml
Rongjun GENG 3c65c99c29
ci: trigger pull_request CI on hotfix/** branches, not just release/** (#7148) (#7198)
pr.yml and pr-risk.yml (and the three advisory workflows that already
list release/** beside it -- conformance.yml, cross-version-smoke.yml,
regression.yml) trigger pull_request only on main and release/**. A PR
based on hotfix/1.3.1 (or any sibling hotfix/** line) matched neither
pattern, so it ran almost no CI: #7146, a backport to hotfix/1.3.1,
showed 17 checks -- 15 cross-version historical smokes and triage/label
jobs, 2 skipped -- no Lint, no unit shards, no embedded tier, no CI
Gate / Required, no PR Risk Gate / Required.

pr.yml's own comment states exactly why release/** is there:
release-prep and release-fix PRs are based on the release branch, not
main, and without it they run almost no CI before the tag build. A
hotfix line is in the identical position. Added hotfix/** alongside
release/** in all five files.

New TestPullRequestWorkflowsTriggerOnHotfixBranches pins hotfix/**
presence in each file's pull_request.branches list -- red before this
change (observed [main release/**] in all five), green after. The
existing TestPRRiskBazelCoverageJob equality check (pr.yml and
pr-risk.yml's pull_request triggers must match) still passes, since
both were updated identically. go vet/gofmt clean.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
2026-10-04 20:30:49 -07:00

177 lines
8.4 KiB
YAML

name: Regression Tests
on:
# Run on merges to main
push:
branches: [main]
pull_request:
branches: [main, 'release/**', 'hotfix/**']
# Allow manual trigger
workflow_dispatch:
env:
BD_DISABLE_METRICS: "1"
BD_DISABLE_EVENT_FLUSH: "1"
jobs:
regression:
name: Differential Regression (v0.49.6 baseline)
# F7c: detect-regression-need folded in as this job's own first step
# (id: detect). A non-applicable PR now costs one short Blacksmith-sized
# job instead of two GitHub-hosted jobs; every later step is gated on
# steps.detect.outputs.run_regression instead of a separate job's output.
runs-on: ${{ (github.event_name == 'merge_group' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]')) && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
with:
fetch-depth: 0
- name: Decide whether to run differential regression tests
id: detect
env:
EVENT_NAME: ${{ github.event_name }}
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
if [[ "$EVENT_NAME" != "pull_request" ]]; then
echo "run_regression=true" >> "$GITHUB_OUTPUT"
echo "reason=$EVENT_NAME event always runs regression tests" >> "$GITHUB_OUTPUT"
exit 0
fi
labels="$(jq -r '.pull_request.labels[].name // empty' "$GITHUB_EVENT_PATH")"
if grep -Fxq "run-regression" <<<"$labels"; then
echo "run_regression=true" >> "$GITHUB_OUTPUT"
echo "reason=run-regression label present" >> "$GITHUB_OUTPUT"
exit 0
fi
if grep -Fxq "skip-regression" <<<"$labels"; then
echo "run_regression=false" >> "$GITHUB_OUTPUT"
echo "reason=skip-regression label present" >> "$GITHUB_OUTPUT"
exit 0
fi
changed="$(git diff --name-only "$PR_BASE_SHA" "$PR_HEAD_SHA")"
echo "Changed files:"
printf '%s\n' "$changed"
regression_candidates="$(
printf '%s\n' "$changed" |
grep -Ev '^(cmd/bd|internal/storage|internal/types)/.*_test\.go$' ||
true
)"
risky_pattern='^(\.github/workflows/regression\.yml|Makefile|go\.(mod|sum)|cmd/bd/|internal/storage/|internal/types/|tests/regression/)'
if grep -Eq "$risky_pattern" <<<"$regression_candidates"; then
echo "run_regression=true" >> "$GITHUB_OUTPUT"
echo "reason=PR changes CLI, storage, types, regression, or build inputs" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "run_regression=false" >> "$GITHUB_OUTPUT"
echo "reason=PR does not touch regression-sensitive paths" >> "$GITHUB_OUTPUT"
- name: Show regression trigger reason
if: steps.detect.outputs.run_regression == 'true'
run: echo "${{ steps.detect.outputs.reason }}"
- name: Set up Go
id: setup-go
if: steps.detect.outputs.run_regression == 'true'
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# F7c review fix S5: this job's own key copy must match main.yml's
# seeder key byte-for-byte (TestBlacksmithSetupGoCacheKeysMatchAcrossSeederAndConsumers),
# which now keys by UTC calendar day instead of github.sha so main
# pushes don't churn a multi-GB cache entry on every commit; this step
# computes that same day value locally so the key expression below
# resolves (steps.* references only resolve within the same job).
- name: Compute cache date
id: cache-date
if: steps.detect.outputs.run_regression == 'true' && runner.environment == 'self-hosted'
run: echo "today=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
- name: Restore Blacksmith setup-go cache
if: steps.detect.outputs.run_regression == 'true' && runner.environment == 'self-hosted'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-${{ steps.cache-date.outputs.today }}
restore-keys: |
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('go.sum') }}-
blacksmith-sg-v1-${{ runner.os }}-${{ runner.arch }}-go-${{ steps.setup-go.outputs.go-version }}-
- name: Configure Git
if: steps.detect.outputs.run_regression == 'true'
run: |
git config --global user.name "CI Bot"
git config --global user.email "ci@beads.test"
- name: Install Dolt
if: steps.detect.outputs.run_regression == 'true'
run: ./scripts/ci/install-dolt.sh
- name: Configure Dolt
if: steps.detect.outputs.run_regression == 'true'
run: |
dolt config --global --add user.name "CI Bot"
dolt config --global --add user.email "ci@beads.test"
dolt version
- name: Pull Dolt sql-server image
# Without this, isDoltImageCached() returns false in TestMain and the
# regression suite falls back to embedded Dolt with HOME=<tmpdir>, which
# has a known cleanup race (see beads issue: TestCreateWithParentFlag
# TempDir RemoveAll "directory not empty" flake). Keep the tag in sync
# with internal/testutil/testdoltcommon.go:DoltDockerImage.
if: steps.detect.outputs.run_regression == 'true'
run: ./scripts/ci/pull-dolt-image.sh
# Restore-only: actions/cache (bare) auto-saves on any key miss, which
# would let a same-repo PR that bumps BASELINE_VERSION write a baseline
# binary cache entry a later PR or main could read back unverified. The
# matching save below only runs off pull_request (B2, F7c implementation
# report).
- name: Restore baseline binary cache
if: steps.detect.outputs.run_regression == 'true'
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
with:
path: ~/.cache/beads-regression
key: regression-baseline-${{ hashFiles('tests/regression/BASELINE_VERSION') }}
# Passing-side wall time already overlapped the go deadline below, so
# ordinary GitHub-runner speed variance -- not a real hang -- was
# tripping this step. Measured over the 100 most recent Regression
# Tests runs (2026-09-03/04, ~29.5h): p50 593s (~10m), max passing
# 1189s (~19.8m) against the prior 1200s (20m) cap, and 5 runs in that
# window hit the identical `panic: test timed out after 20m0s` with
# zero `--- FAIL` lines -- a budget defect, not a regression. Match
# main.yml's Test job fix for the same failure mode: raise the go
# deadline to cover the observed slowdown and keep it inside a larger
# finite job budget (see timeout-minutes above), so the go deadline
# fires first and prints goroutine stacks -- a REAL hang stays
# diagnosable instead of being cut off by a bare job kill.
- name: Run regression tests
if: steps.detect.outputs.run_regression == 'true'
run: go test -tags=regression,gms_pure_go -timeout=40m -v ./tests/regression/...
- name: Save baseline binary cache
# Allow-list push/workflow_dispatch rather than deny-list
# pull_request: a deny-list also admits a hypothetical future
# merge_group event, which this job's runs-on expression already
# resolves to Blacksmith for. Pinned by
# TestBlacksmithReachableAdvisoryJobsNeverSaveACache (checks both the
# pull_request and merge_group trust contexts independently).
if: steps.detect.outputs.run_regression == 'true' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
with:
path: ~/.cache/beads-regression
key: regression-baseline-${{ hashFiles('tests/regression/BASELINE_VERSION') }}