beads/scripts/ci/detect-package-gates.sh
Julian Knutsen e72cd8b315
ci: merge-queue readiness for the required gates; cache test results (fresh nightly) (#7264)
* ci: make the required gates merge-queue ready

The default-branch ruleset requires "CI Gate / Required" and "PR Risk
Gate / Required". Prepare both for a GitHub merge queue (ALLGREEN,
squash), so enabling the ruleset's merge_queue rule is the only step left.

- pr.yml, pr-risk.yml: merge_group pinned to types [checks_requested].
- D2: bazel-coverage covers every merge_group run (same repository, CI
  secrets, actor github-merge-queue[bot], bazel.yml mode remote), so the
  retired legacy tiers stay retired in the queue and ci-gate requires
  their Bazel lanes to have run remotely and passed; a missing secret or
  the farm switch off is red, as on a same-repo PR.
- Diff bases on merge_group: the package gates diff
  merge_group.base_sha..head_sha (not HEAD^, which misses all but the
  last commit of a rebase-method entry), the .beads/issues.jsonl guard
  now runs on merge_group against base_sha (ci-gate no longer excuses
  its skip), and check-doc-flags passes BD_DOCS_DIFF_BASE so CLI-docs
  drift already on main cannot eject every queued PR.
- Test result reuse: on a merge_group run and on attempt 2+ of a PR run,
  the six always-execute Bazel lanes append --config=reuse-test-results
  (--cache_test_results=auto), so tests whose action key is unchanged
  reuse the farm's recorded PASS and only what the merge changed runs.
  Failures are never reused, eviction retries stay 0, and each PR's first
  attempt, push to main and nightly still execute every test.
- scripts/ci_merge_queue_test.go simulates merge_group through the shared
  evalGHExpr evaluator (now with true/false/null literals, hyphenated
  identifiers and case-insensitive ==) and the workflows' own scripts:
  triggers and gate names, concurrency groups, every pull_request field
  read, the coverage and rbe decisions, result reuse, the package-gate,
  .beads and doc-drift diff bases.
- engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md: new "Merge Queue" section (how
  to queue, settings, what runs, result reuse rationale, ejection and
  flakes, rollback).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: cache Bazel test results except nightly; queue trust and sizing

Review follow-ups to the merge-queue readiness change.

- Test results (S1): --nocache_test_results also stops Bazel writing
  results, so a PR's first run never fed the cache and the merge-queue
  reuse config could only hit on later reuse runs. Drop it from
  test:embedded, test:doltserver-proxied, test:doltserver-integration and
  test:sole-run (which keeps --experimental_remote_cache_eviction_retries=0),
  and drop test:reuse-test-results and BAZEL_REUSE_RESULTS. Results are now
  cached (read and write) on PRs, merge groups and pushes to main.
  nightly.yml passes the new fresh-test-results input, so every `bazel
  test` appends --config=fresh (--nocache_test_results) and the suite
  re-executes nightly. The docker lane keeps --nocache_test_results.
  Measured locally with a disk cache: a default run writes and the next
  reads ("(cached) PASSED"); a fresh run neither reads nor writes.
- Security headers (S2): bazel.yml now states that a merge_group run
  executes the queued commit's workflows, so a queued fork or Dependabot
  PR's CI edits run with every CI secret, the pre-warm app key included
  (merging-level trust), and records the maintainer rule: review fork
  changes to .github/, .bazelrc, tools/bazel, MODULE.bazel, BUILD files
  and CI scripts before queueing. Same note in the topology doc.
- Queue sizing (S3): recommend max_entries_to_build 3 and
  check_response_timeout_minutes 60 (max_entries_to_merge 5, ALLGREEN,
  SQUASH unchanged).
- Nits: the .beads guard fails closed on a missing diff base, and its
  test runs under GitHub's default `bash -e`; evalGHExpr follows GitHub's
  loose equality (true == 'true' is false, false == '' is true, missing
  context keys are null) with a typed-context variant; the
  detect-package-gates comment notes the SQUASH/HEAD^ equivalence and
  path-filter scope; the doc covers re-run reuse, flake re-sampling via
  dispatch, stale queue runs and how to confirm reuse on the farm.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 13:54:15 -07:00

124 lines
4.5 KiB
Bash
Executable file

#!/usr/bin/env bash
# Decide which package gates apply to the current CI event.
set -euo pipefail
event_name="${GITHUB_EVENT_NAME:-}"
pr_base_sha="${PR_BASE_SHA:-}"
pr_head_sha="${PR_HEAD_SHA:-}"
push_before_sha="${PUSH_BEFORE_SHA:-}"
push_after_sha="${PUSH_AFTER_SHA:-${GITHUB_SHA:-HEAD}}"
mcp_package=false
npm_package=false
reason=""
changed_files=""
write_outputs() {
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
{
echo "mcp_package=$mcp_package"
echo "npm_package=$npm_package"
echo "reason=$reason"
} >>"$GITHUB_OUTPUT"
fi
}
run_all() {
reason="$1"
mcp_package=true
npm_package=true
write_outputs
echo "$reason"
exit 0
}
if [[ "${CI_PACKAGE_GATES_FORCE:-}" == "all" ]]; then
run_all "CI_PACKAGE_GATES_FORCE=all"
fi
case "$event_name" in
pull_request)
if [[ -z "$pr_base_sha" || -z "$pr_head_sha" ]]; then
run_all "PR diff bounds unavailable; running all package gates"
fi
if ! changed_files="$(git diff --name-only "$pr_base_sha" "$pr_head_sha")"; then
run_all "PR diff failed; running all package gates"
fi
;;
push)
if [[ -z "$push_before_sha" || "$push_before_sha" =~ ^0+$ ]]; then
if ! changed_files="$(git diff-tree --no-commit-id --name-only -r "$push_after_sha")"; then
run_all "push diff unavailable; running all package gates"
fi
elif ! changed_files="$(git diff --name-only "$push_before_sha" "$push_after_sha")"; then
run_all "push diff failed; running all package gates"
fi
;;
merge_group)
# PR_BASE_SHA/PR_HEAD_SHA carry merge_group.base_sha/head_sha here:
# the commit the queue entry is built on and the entry itself. Not
# HEAD^: a rebase-method queue entry stacks every commit of the PR,
# so its first parent would hide all but the last one. With the
# ALLGREEN grouping strategy every entry of a batch is checked, so
# each entry's own diff is enough. (With SQUASH, HEAD^ already was
# base_sha; this matters for the merge and rebase methods.) As with
# PR-time path filtering, entry N's gate does not re-test entry
# N-1's package change combined with N's; N-1's own run tested it.
if [[ -z "$pr_base_sha" || -z "$pr_head_sha" ]]; then
run_all "merge-group diff bounds unavailable; running all package gates"
fi
if ! changed_files="$(git diff --name-only "$pr_base_sha" "$pr_head_sha")"; then
run_all "merge-group diff failed; running all package gates"
fi
;;
*)
run_all "non-PR/push/merge-group event; running all package gates"
;;
esac
while IFS= read -r path; do
[[ -n "$path" ]] || continue
case "$path" in
integrations/beads-mcp/*|scripts/ci/package-mcp.sh|scripts/ci/detect-package-gates.sh|.github/workflows/pr.yml|.github/workflows/main.yml|.github/workflows/pr-risk.yml|.github/workflows/ci-measurements.yml|.github/workflows/release.yml|Makefile)
mcp_package=true
;;
esac
case "$path" in
npm-package/*|scripts/ci/package-npm.sh|scripts/ci/detect-package-gates.sh|.github/workflows/pr.yml|.github/workflows/main.yml|.github/workflows/pr-risk.yml|.github/workflows/ci-measurements.yml|.github/workflows/release.yml|Makefile)
npm_package=true
;;
esac
case "$path" in
# F3: package-mcp/package-npm's job definitions (this detect step
# included) now live in bazel.yml, not pr.yml, so a change there
# must run both gates the same way a pr.yml/main.yml change does.
# package-bazel-bd.sh packages the Bazel-built bd both gates use on
# the remote-execution path.
scripts/ci/detect-package-gates.sh|scripts/ci/package-bazel-bd.sh|.github/workflows/bazel.yml|.github/workflows/pr.yml|.github/workflows/main.yml|.github/workflows/pr-risk.yml|.github/workflows/ci-measurements.yml|Makefile)
mcp_package=true
npm_package=true
;;
esac
done <<<"$changed_files"
if [[ "$mcp_package" == "true" || "$npm_package" == "true" ]]; then
reason="package paths changed"
else
reason="no package-gate paths changed"
fi
write_outputs
echo "$reason"
echo "Changed files:"
if [[ -n "$changed_files" ]]; then
while IFS= read -r path; do
printf ' %s\n' "$path"
done <<<"$changed_files"
else
echo " <none>"
fi
echo "Package gates:"
echo " mcp_package=$mcp_package"
echo " npm_package=$npm_package"