fix(test): stdio leak guard baselines at first test, not var-init (#5881) (#5902)

Under go test -json (what gotestsum runs, so every CI coverage leg) the
testing framework itself reassigns os.Stderr = os.Stdout inside M.Run so
both streams interleave into one test2json event stream (go1.26
testing/testing.go:2391, go.dev/issue/33419). That swap lands after
package var-init and before the first test, so a guard baselined at
var-init convicts the framework on every -json run — the ubuntu-only
zz_stdio_leak_guard failures bisected in #5881 (probe branch
bee/zzleak-probe-5881: every TestMain checkpoint clean through pre-m.Run,
first test already polluted; reproducible on darwin with plain
go test -json).

Capture the baseline in the alphabetically-first test instead
(aaa_stdio_baseline_test.go), so the guard measures exactly what it
promises: did any TEST leak a stdio swap. When a -run filter excludes
the baseline test the guard skips rather than comparing against a
pre-framework baseline.

Verified: cmd/bd guard tests pass in plain mode and -json mode (both
previously failed under -json), and skip cleanly under
-run TestZZStdioNotLeaked alone.
This commit is contained in:
Bee 2026-08-27 06:26:16 -07:00 • committed by GitHub
commit b2cad50584
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 38 additions and 15 deletions

View file

@ -0,0 +1,22 @@
package main
import (
"os"
"testing"
)
// baselineStdout/baselineStderr are the os.Stdout/os.Stderr objects as the
// FIRST test in this package saw them, captured here rather than at package
// var-init: under `go test -json` (what gotestsum runs, so every CI coverage
// leg) the testing framework itself reassigns os.Stderr = os.Stdout inside
// M.Run so both streams interleave cleanly into one test2json event stream
// (go1.26 testing/testing.go:2391, go.dev/issue/33419). That swap is the
// framework's, not a test's; a leak guard that baselines at var-init convicts
// the framework on every -json run (#5881) while measuring nothing about the
// tests. This file is named aaa_* so its test is declared first: go test runs
// tests in declaration order across files sorted by name (no -shuffle here).
var baselineStdout, baselineStderr *os.File
func TestAAAStdioBaseline(t *testing.T) {
baselineStdout, baselineStderr = os.Stdout, os.Stderr
}

View file

@ -5,28 +5,29 @@ import (
"testing"
)
// realStdout/realStderr are captured at package-variable initialization, before
// any test runs, so they are the process's genuine streams.
var (
realStdout = os.Stdout
realStderr = os.Stderr
)
// TestZZStdioNotLeaked fails if any test in this package reassigned os.Stdout or
// os.Stderr and did not restore it. A capture helper that restores in a defer
// cannot trip this; one that restores on the happy path only will trip it as soon
// as its callback calls t.Fatal. See be-gh02.
// TestZZStdioNotLeaked fails if any test in this package reassigned os.Stdout
// or os.Stderr and did not restore it. A capture helper that restores in a
// defer cannot trip this; one that restores on the happy path only will trip
// it as soon as its callback calls t.Fatal. See be-gh02.
//
// The baseline is what the FIRST test saw (aaa_stdio_baseline_test.go), not
// the var-init streams: under `go test -json` the testing framework swaps
// os.Stderr to os.Stdout inside M.Run (go.dev/issue/33419), after var-init
// and before any test, and that framework swap is not a leak (#5881).
func TestZZStdioNotLeaked(t *testing.T) {
if os.Stdout != realStdout {
if baselineStdout == nil || baselineStderr == nil {
t.Skip("stdio baseline not captured (TestAAAStdioBaseline filtered out); nothing to compare against")
}
if os.Stdout != baselineStdout {
t.Errorf("os.Stdout was leaked by an earlier test (now fd=%d name=%q); "+
"a capture helper restored it on the happy path only - move the restore into a defer",
os.Stdout.Fd(), os.Stdout.Name())
os.Stdout = realStdout
os.Stdout = baselineStdout
}
if os.Stderr != realStderr {
if os.Stderr != baselineStderr {
t.Errorf("os.Stderr was leaked by an earlier test (now fd=%d name=%q); "+
"a capture helper restored it on the happy path only - move the restore into a defer",
os.Stderr.Fd(), os.Stderr.Name())
os.Stderr = realStderr
os.Stderr = baselineStderr
}
}