Fix: check-testing-short.sh flags comments and misnames the offending function (round 2) (#5809)
* test(feat): red — Fix: check-testing-short.sh flags comments and misnames the offending function (refs be-0v13) * feat: green — Fix: check-testing-short.sh flags comments and misnames the offending function (refs be-0v13) Skip comment-only mentions of testing.Short() before the allowlist check: read the exact matched line and treat a line whose first non-whitespace characters are // as non-offending. Existing func-attribution logic already reports "unknown" for a hit above the first func in a file, so no change was needed there. Also fixes two test-fixture bugs found while confirming RED: the RED test file's own source contained the literal substring testing.Short() (now built via string concatenation into a shortCall const so the checker doesn't flag this file when it scans the real repo tree), and single-file temp-dir fixtures omitted the grep filename prefix (now every fixture dir gets a companion .go file so find+grep always sees 2+ filenames, matching real repo-root invocations). * test(scripts): red — check-testing-short.sh must not attribute a hit between two functions to the preceding one (refs be-r56j) * fix(scripts): green — resolve testing.Short() hits to the containing function, not the nearest preceding one (refs be-r56j) * chore: release gate PASS for be-053u-check-testing-short-round2 * ci: retrigger — unrelated Ryuk reaper flake in storage/uow/tracker job The "Test (storage domain + uow)" job failed on an infrastructure flake unrelated to this PR's diff. That job runs three package trees in one go test invocation (.github/workflows/pr.yml:676), so Go runs their test binaries in parallel and each starts its own Dolt testcontainer. The binaries share a single testcontainers Ryuk reaper; in this run the internal/tracker binary reused a sibling's reaper and its handshake was reset: 23:12:21 Reaper obtained from Docker for this test session 6a74346c 23:12:21 Reaper handshake failed: read ack: ... connection reset by peer With no live connection holding the session open, Ryuk reaped the session's containers. The tracker package's Dolt container went down mid-suite and every subsequent store-backed test failed with "connection refused" on 127.0.0.1:32772. This is be-2on, root-caused on a different PR (#5339) with the identical signature. The durable fix is PR #5770 (disable Ryuk on the Dolt testcontainer steps), which is open and awaiting upstream merge. This PR touches only scripts/check-testing-short.sh, its test, and a release-gate record — nothing in internal/tracker or the Dolt path. Empty commit to re-trigger CI.
This commit is contained in:
parent
28d65c1329
commit
5941096915
3 changed files with 261 additions and 1 deletions
142
release-gates/be-053u-check-testing-short-round2-gate.md
Normal file
142
release-gates/be-053u-check-testing-short-round2-gate.md
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
# Release Gate: be-053u
|
||||
|
||||
**Feature:** Fix: check-testing-short.sh flags comments and misnames the offending function (round 2)
|
||||
**Deploy bead:** be-053u
|
||||
**Review bead:** be-d8z3 (verdict: pass, closed 2026-08-15T22:23:23Z)
|
||||
**Build beads:** be-0v13 (round 1) → be-r56j (round-1 review, request-changes) → round-2 resume (this deploy)
|
||||
**Deploy commit:** `604a672d5cc9138ad589b4be9ccee0b62a2fd33b`
|
||||
**Source branch:** `builder/be-0v13` (provenance only — never a push target)
|
||||
**Deploy mode:** remote (origin=github.com/gastownhall/beads, fork=github.com/quad341/beads)
|
||||
**Base ref:** `origin/main` @ `7505e173f` ("chore(release): forward-port v1.2.2 to main", #5782)
|
||||
|
||||
## Pre-flight: already merged?
|
||||
|
||||
`gh api repos/gastownhall/beads/commits/604a672d5cc9138ad589b4be9ccee0b62a2fd33b/pulls` → `[]`.
|
||||
No PR exists for this commit yet. Normal (non-reconciliation) flow applies.
|
||||
|
||||
## Criterion 6 — Branch diverges cleanly from BASE_REF (evaluated first)
|
||||
|
||||
**PASS.** `git merge-base --is-ancestor origin/main 604a672d5c...` → rc=0. origin/main
|
||||
(7505e173f) is a direct ancestor of the deploy commit:
|
||||
|
||||
```
|
||||
7505e173f (origin/main tip)
|
||||
→ d79c5a2f2 test(feat): red
|
||||
→ d7f274e06 feat: green (round 1)
|
||||
→ 6b9ed10c3 test(scripts): red (round 2)
|
||||
→ 604a672d5 fix(scripts): green (round 2, DEPLOY_SHA)
|
||||
```
|
||||
|
||||
Strict fast-forward descendant, zero divergence. No self-rebase needed.
|
||||
|
||||
## Criterion 1 — Reviewer PASS verdict present
|
||||
|
||||
**PASS.** be-d8z3, verdict: pass, close_reason: "pass". Round 2 re-reviewed after round 1
|
||||
(be-r56j) issued request-changes for an unmet exit_contract item (see criterion 2).
|
||||
|
||||
## Criterion 2 — Acceptance criteria met
|
||||
|
||||
**PASS.** This repo has no `work-packages/` directory and no `docs/PROJECT_MANIFEST.md` —
|
||||
both checked, neither exists. Acceptance criteria live as bead-native `exit_contract` items
|
||||
instead (the operative mechanism here). be-r56j's round-1 review left one item unmet:
|
||||
|
||||
> UNMET: a hit that sits between two functions (not truly inside either) must resolve to
|
||||
> 'unknown', not the preceding function's name. Round-1's test only covered the degenerate
|
||||
> "no preceding func at all" case, which already passed pre-fix via an unrelated fallback.
|
||||
|
||||
Round 2 adds `TestCheckTestingShortReportsUnknownBetweenFunctions` and fixes the awk
|
||||
attribution logic. be-d8z3 independently hand-read the new fixture (`TestA(){}` → blank
|
||||
line → bare `testing.Short()` → `TestB(){}`) and confirmed it exercises the genuine
|
||||
between-two-functions case, not a degenerate repeat. All 5 exit_contract items now `[x]`.
|
||||
|
||||
## Criterion 3 — Tests pass (required CI-equivalent command)
|
||||
|
||||
**PASS.** Ran `make ci-pr-core` at DEPLOY_SHA (the actual required PR test lane:
|
||||
`go test -race -short -skip '^TestEmbedded' ./...`, hermetic env via `beads_test_env_enter`).
|
||||
Exit 0, every package `ok`, runtime 379s. `DOCKER_HOST`/`TESTCONTAINERS_RYUK_DISABLED` set
|
||||
beforehand per protocol; this diff has no container/storage surface so no container-backed
|
||||
tests apply, but the env was live regardless to avoid a false-green skip.
|
||||
|
||||
Diff-owned test file: `scripts/check_testing_short_test.go` (only `_test.go` file in the
|
||||
diff; `git diff --name-only origin/main...604a672d5c | grep -E '_test\.(go|py|rb)$|...'`).
|
||||
Independently re-ran verbosely, by name (not just the aggregate "ok"):
|
||||
|
||||
| Test | Result |
|
||||
|---|---|
|
||||
| TestCheckTestingShortIgnoresCommentOnlyMention | PASS |
|
||||
| TestCheckTestingShortStillFlagsRealCallAndNamesFunc | PASS |
|
||||
| TestCheckTestingShortReportsUnknownAboveFirstFunc | PASS |
|
||||
| TestCheckTestingShortReportsUnknownBetweenFunctions | PASS |
|
||||
| TestCheckTestingShortPassesOnCleanRepoTree | PASS |
|
||||
|
||||
test_counts: 5/5 diff-owned PASS, 0 FAIL, 0 SKIP. One pre-existing SKIP elsewhere in the
|
||||
`scripts` package (`TestTestScriptPrebuiltBinaryLaunchProbe`, self-gated to the test.sh
|
||||
fake-go re-exec driver, not diff-owned — file untouched by this diff). No diff-owned SKIP,
|
||||
so no waiver needed. waiver_ref: none.
|
||||
|
||||
## Criterion 3b — Policy/lint lane (part of criterion 3, not optional)
|
||||
|
||||
**PASS**, after excluding a confirmed local-environment false positive.
|
||||
|
||||
`make ci-pr-policy` initially failed at "check version consistency":
|
||||
`.githooks/commit-msg: no 'BEGIN BEADS INTEGRATION' marker found`. Investigated before
|
||||
accepting this as a real finding, since the diff never touches `.githooks/`:
|
||||
|
||||
- `git log -- .githooks/commit-msg` → empty. The path has **zero commit history** in this repo.
|
||||
- `git ls-files .githooks/commit-msg` → empty (untracked).
|
||||
- `git check-ignore -v .githooks/commit-msg` → matched by
|
||||
`/home/jaword/projects/beads/.git/info/exclude:40` — a **local-only**, machine-specific
|
||||
exclude rule, never present in any other clone or in CI.
|
||||
- File content: a gc-management "commit-gate shim... installed by worktree-setup.sh
|
||||
(be-xug guardrail A)... rewritten on every session start", pointing at a local
|
||||
gc-management path — session-orchestrator machinery, not beads-repo content.
|
||||
- `scripts/check-versions.sh` enumerates via a bare `for hook in .githooks/*` filesystem
|
||||
glob (confirmed: `grep -n githooks scripts/check-versions.sh` → line 95), with no
|
||||
git-tracked-file filter. A real GitHub Actions checkout would never have this file, so
|
||||
this failure cannot reproduce in actual CI.
|
||||
|
||||
Backed up the shim (mode 755, byte contents), relocated it, re-ran `make ci-pr-policy`
|
||||
clean end-to-end (exit 0) — including "check version consistency" itself, and including
|
||||
"check testing.Short boundaries" (the exact lane this bead's diff targets):
|
||||
|
||||
```
|
||||
check build-tag policy .......... PASS (96 files scanned)
|
||||
check go install guidance ....... PASS
|
||||
check version consistency ....... PASS
|
||||
build bd for docs checks ........ PASS
|
||||
check doc flags .................. PASS
|
||||
check doc freshness .............. PASS
|
||||
check testing.Short boundaries ... PASS
|
||||
check workapi frontend boundary .. PASS
|
||||
check no .beads/issues.jsonl changes . PASS
|
||||
check openapi spec gate (api-check) .. PASS
|
||||
```
|
||||
|
||||
Restored the shim immediately after (byte-identical `diff -q`, mode 755 confirmed).
|
||||
|
||||
`make ci-pr-lint` with `BD_LINT_NEW_FROM_MERGE_BASE=origin/main` (matching
|
||||
`.github/workflows/pr.yml`'s actual `pr-lint-wrapper` job exactly): exit 0.
|
||||
gofmt clean; golangci-lint 0 issues (native + windows cross-lint). Note: locally installed
|
||||
golangci-lint is v2.12.0 vs CI-pinned v2.10.1 — version delta not reconciled (diff is 2
|
||||
files / +119/-1 in `scripts/`, low risk), flagged here for transparency rather than silently
|
||||
assumed equivalent.
|
||||
|
||||
policy_lane: `make ci-pr-policy` + `make ci-pr-lint`, both PASS (see above).
|
||||
|
||||
## Criterion 4 — No high-severity review findings open
|
||||
|
||||
**PASS.** be-d8z3 round-2 style_findings: none blocking (one pre-existing non-blocking note
|
||||
at scripts/check-testing-short.sh:29, unchanged by this diff, already known from round 1).
|
||||
security_findings: none — full OWASP Top 10 walk against the true diff, explicitly including
|
||||
a note that the net effect is gate-tightening (closes a misattribution that could let an
|
||||
unauthorized `testing.Short()` slip past the allowlist under a legitimate function's name).
|
||||
|
||||
## Criterion 5 — Feature branch clean (no uncommitted changes)
|
||||
|
||||
**PASS.** `git status --short` at DEPLOY_SHA: zero modified/staged tracked files. Only
|
||||
pre-existing untracked leftovers from unrelated, already-closed beads (release-gates/be-hi97,
|
||||
release-gates/be-uoat, scripts/rebase-resolve-lib.sh — a stray untracked copy, not sourced).
|
||||
|
||||
## Overall verdict: PASS
|
||||
|
||||
All 6 criteria pass. Proceeding to push-and-pr.
|
||||
|
|
@ -25,13 +25,21 @@ while IFS=: read -r file line _; do
|
|||
continue
|
||||
fi
|
||||
|
||||
content=$(sed -n "${line}p" "$file")
|
||||
if [[ "$content" =~ ^[[:space:]]*// ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
func=$(
|
||||
awk -v target="$line" '
|
||||
NR <= target && /^func [A-Za-z0-9_]+\(/ {
|
||||
NR > target { exit }
|
||||
/^func [A-Za-z0-9_]+\(/ {
|
||||
current = $0
|
||||
sub(/^func /, "", current)
|
||||
sub(/\(.*/, "", current)
|
||||
next
|
||||
}
|
||||
current != "" && /^}/ { current = "" }
|
||||
END { print current }
|
||||
' "$file"
|
||||
)
|
||||
|
|
|
|||
110
scripts/check_testing_short_test.go
Normal file
110
scripts/check_testing_short_test.go
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
package scripts_test
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Built via concatenation so this file's own source never contains the
|
||||
// literal substring check-testing-short.sh scans for — otherwise this test
|
||||
// file would flag itself when TestCheckTestingShortPassesOnCleanRepoTree
|
||||
// runs the checker against the real repo tree.
|
||||
const shortCall = "testing" + ".Short()"
|
||||
|
||||
func TestCheckTestingShortIgnoresCommentOnlyMention(t *testing.T) {
|
||||
out, err := runCheckTestingShort(t, map[string]string{
|
||||
"pkg_test.go": "package pkg\n\nimport \"testing\"\n\nfunc TestFoo(t *testing.T) {\n\t// this comment mentions " + shortCall + " and should not fail the gate\n}\n",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("comment-only mention of %s should not fail the gate; err=%v output=%s", shortCall, err, out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckTestingShortStillFlagsRealCallAndNamesFunc(t *testing.T) {
|
||||
out, err := runCheckTestingShort(t, map[string]string{
|
||||
"pkg_test.go": "package pkg\n\nimport \"testing\"\n\nfunc TestBar(t *testing.T) {\n\tif " + shortCall + " {\n\t\tt.Skip(\"slow\")\n\t}\n}\n",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatalf("real %s call should still fail the gate; output=%s", shortCall, out)
|
||||
}
|
||||
if !strings.Contains(out, "TestBar") {
|
||||
t.Errorf("expected offending function TestBar named in output, got: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckTestingShortReportsUnknownAboveFirstFunc(t *testing.T) {
|
||||
out, err := runCheckTestingShort(t, map[string]string{
|
||||
"pkg_test.go": "package pkg\n\nimport \"testing\"\n\nvar shortMode = " + shortCall + "\n\nfunc TestBaz(t *testing.T) {}\n",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatalf("real %s call above any func should still fail the gate; output=%s", shortCall, out)
|
||||
}
|
||||
if !strings.Contains(out, "unknown") {
|
||||
t.Errorf("expected 'unknown' function name for a hit above the first func, got: %s", out)
|
||||
}
|
||||
if strings.Contains(out, "TestBaz") {
|
||||
t.Errorf("hit above the first func must not be attributed to a later func TestBaz: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckTestingShortReportsUnknownBetweenFunctions(t *testing.T) {
|
||||
out, err := runCheckTestingShort(t, map[string]string{
|
||||
"pkg_test.go": "package pkg\n\nimport \"testing\"\n\nfunc TestA(t *testing.T) {\n\tt.Log(\"a\")\n}\n\nvar shortAgain = " + shortCall + "\n\nfunc TestB(t *testing.T) {}\n",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatalf("real %s call between two functions should still fail the gate; output=%s", shortCall, out)
|
||||
}
|
||||
if !strings.Contains(out, "unknown") {
|
||||
t.Errorf("expected 'unknown' function name for a hit between two functions (not truly inside either body), got: %s", out)
|
||||
}
|
||||
if strings.Contains(out, "TestA") {
|
||||
t.Errorf("hit between two functions must not be attributed to the preceding func TestA: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckTestingShortPassesOnCleanRepoTree(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("checker is a Bash boundary")
|
||||
}
|
||||
repo := sourceRepoRoot(t)
|
||||
cmd := exec.Command("bash", filepath.Join(repo, "scripts", "check-testing-short.sh"))
|
||||
cmd.Dir = repo
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("check-testing-short.sh failed on clean repo tree (all 9 allowlist entries must still be recognized): %v\noutput: %s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
func runCheckTestingShort(t *testing.T, files map[string]string) (string, error) {
|
||||
t.Helper()
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("checker is a Bash boundary")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
// A second, innocuous .go file keeps find+grep invoked with 2+ filenames,
|
||||
// matching real usage (repo root always has many .go files): grep only
|
||||
// prefixes matches with the filename when given more than one file, so a
|
||||
// single-fixture temp dir would silently corrupt the file:line parse in
|
||||
// check-testing-short.sh's read loop.
|
||||
if err := os.WriteFile(filepath.Join(dir, "companion.go"), []byte("package pkg\n"), 0600); err != nil {
|
||||
t.Fatalf("write companion.go: %v", err)
|
||||
}
|
||||
for rel, content := range files {
|
||||
full := filepath.Join(dir, rel)
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0700); err != nil {
|
||||
t.Fatalf("mkdir %s: %v", filepath.Dir(full), err)
|
||||
}
|
||||
if err := os.WriteFile(full, []byte(content), 0600); err != nil {
|
||||
t.Fatalf("write %s: %v", rel, err)
|
||||
}
|
||||
}
|
||||
script := filepath.Join(sourceRepoRoot(t), "scripts", "check-testing-short.sh")
|
||||
cmd := exec.Command("bash", script)
|
||||
cmd.Dir = dir
|
||||
out, err := cmd.CombinedOutput()
|
||||
return string(out), err
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue