fix(metrics): keep the flusher child's mem diagnostics off the parent's path
Maintainer review fixup for #6507 (adopt-pr review iteration 1). Change Impact / Blast Radius (major). flusherChildEnv hands the detached `bd send-metrics` child the parent's environment verbatim apart from the endpoint and the flusher marker, so BEADS_MEM_PROFILE / BEADS_MEM_STATS arrived holding the same absolute paths the parent had just resolved. The spawn runs on main()'s post-ExecuteC tail (ExecuteC -> CloseAndFlush -> MaybeSpawnFlusher), i.e. after PersistentPostRunE already wrote them, so before this commit the child's trivial profile silently replaced the profile of the command the user actually asked about -- no error, no size or format anomaly. Error exits were worse: CheckReadonly and the pre-run gates call CloseAndFlush while PersistentPostRunE never runs, leaving the child's file as the only one written. This was PR-introduced rather than base behavior: before the PR the child inherited the vars and ignored them. Fixed by memDiagnosticsDest(), which suffixes the resolved destination with ".send-metrics" when BD_IS_FLUSHER=1. flusherChildEnv sets that marker only on the spawned child, so a human running `bd send-metrics` directly still gets the plain path. The suffix is applied after the flag/env fallback so both knobs and both sources follow one rule. This is the synthesis's option (a); option (b) (scrubbing the mem vars in flusherChildEnv) was explicitly not taken, because it would reinstate exactly the inertness be-wwy2.2 removes. Contract & Interface Fidelity (minor). --mem-profile is registered on rootCmd.PersistentFlags(), so the hidden send-metrics command inherits it and prints it in --help, but the call site hardcoded "" -- the flag was parsed and then discarded, leaving the advertised half of the PR's own premise in the same "accepted but silently inert" state it exists to remove, one line below the fix. It now passes memProfilePath exactly as PersistentPostRunE does, and the false "there is no --mem-profile flag on this hidden command" claim is removed from both comments that carried it (send_metrics.go and writeMemDiagnostics's own doc comment). Tests: two new subtests, each mutation-proved to red on its own while the other six stay green. - "flusher child writes beside the parent's files instead of over them" seeds the parent's two files, runs the child with BD_IS_FLUSHER=1, and asserts the parent's bytes survive and the child's own diagnostics are still produced at the suffixed paths. Reverting memDiagnosticsDest to `return dest` reds it alone, with the parent's profile replaced by a real gzip stream -- the finding's exact damage. - "--mem-profile flag is honored, not just the env var" drives the flag with no BEADS_MEM_PROFILE set, so only the flag can produce the file. Restoring writeMemDiagnostics("") reds it alone. Validation: gofmt clean, `go build ./cmd/bd`, `go vet ./cmd/bd`, and `go test ./cmd/bd -run TestSendMetricsHonorsMemDiagnostics -count=1 -v` (7/7 subtests, 15.0s). Not taken, per the synthesis's explicit "do not block this PR": moving writeMemDiagnostics next to CloseAndFlush so error exits are covered, and warning to stderr when os.Create fails. Both are verbatim pre-existing carry-overs whose remedies change behavior for every bd subcommand rather than this one, so they stay follow-ups instead of riding a scoped fixup. Contributor commits are preserved. --mem-profile's help text is unchanged, so no CLI-docs regeneration is implied, and BEADS_MEM_* appears in no other Go file, test, or doc in the tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
8895e6cb33
commit
f4d6ad26a3
3 changed files with 141 additions and 25 deletions
|
|
@ -2243,17 +2243,48 @@ var rootCmd = &cobra.Command{
|
|||
},
|
||||
}
|
||||
|
||||
// flusherDiagnosticsSuffix separates the detached send-metrics child's memory
|
||||
// diagnostics from the parent command's. See memDiagnosticsDest.
|
||||
const flusherDiagnosticsSuffix = ".send-metrics"
|
||||
|
||||
// memDiagnosticsDest returns the destination writeMemDiagnostics should write
|
||||
// dest to, suffixed when this process is the detached flusher child.
|
||||
//
|
||||
// MaybeSpawnFlusher hands the child the parent's environment minus the endpoint
|
||||
// (flusherChildEnv in internal/metrics/spawn.go), so BEADS_MEM_PROFILE and
|
||||
// BEADS_MEM_STATS arrive holding the same absolute paths the parent resolved.
|
||||
// The spawn happens on main()'s post-ExecuteC tail, i.e. after
|
||||
// PersistentPostRunE already wrote them, so an unsuffixed child would silently
|
||||
// replace the profile of the command the user actually asked about with a
|
||||
// profile of the trivial flusher -- no error, no size anomaly. Error exits are
|
||||
// worse: CheckReadonly and the pre-run gates call CloseAndFlush while
|
||||
// PersistentPostRunE never runs, leaving the child's file as the only one.
|
||||
//
|
||||
// BD_IS_FLUSHER=1 is set only by flusherChildEnv, so a human running
|
||||
// `bd send-metrics` directly still gets the plain path. The suffix is applied
|
||||
// to the resolved destination, after the flag/env fallback, so both knobs and
|
||||
// both sources follow one rule.
|
||||
func memDiagnosticsDest(dest string) string {
|
||||
if dest == "" || os.Getenv(metrics.EnvIsFlusher) != "1" {
|
||||
return dest
|
||||
}
|
||||
return dest + flusherDiagnosticsSuffix
|
||||
}
|
||||
|
||||
// writeMemDiagnostics honors the heap-profile and MemStats diagnostic knobs
|
||||
// (--mem-profile / BEADS_MEM_PROFILE / BEADS_MEM_PROFILE_NOGC / BEADS_MEM_STATS).
|
||||
// memProfileFlag is the --mem-profile flag value ("" if the caller has no
|
||||
// such flag, e.g. the send-metrics child, which calls this directly since its
|
||||
// Run exits before Cobra ever reaches PersistentPostRunE below).
|
||||
// memProfileFlag is the --mem-profile flag value, which wins over
|
||||
// BEADS_MEM_PROFILE. Both call sites pass memProfilePath: --mem-profile is
|
||||
// registered on rootCmd.PersistentFlags(), so every subcommand inherits it,
|
||||
// including the hidden send-metrics one -- which calls this directly because its
|
||||
// Run exits before Cobra ever reaches PersistentPostRunE below.
|
||||
func writeMemDiagnostics(memProfileFlag string) {
|
||||
// Runs a GC first by default; BEADS_MEM_PROFILE_NOGC=1 skips it to capture peak.
|
||||
heapDest := memProfileFlag
|
||||
if heapDest == "" {
|
||||
heapDest = os.Getenv("BEADS_MEM_PROFILE")
|
||||
}
|
||||
heapDest = memDiagnosticsDest(heapDest)
|
||||
if heapDest != "" {
|
||||
if os.Getenv("BEADS_MEM_PROFILE_NOGC") == "" {
|
||||
runtime.GC()
|
||||
|
|
@ -2264,7 +2295,7 @@ func writeMemDiagnostics(memProfileFlag string) {
|
|||
}
|
||||
}
|
||||
// Optional one-line MemStats summary: BEADS_MEM_STATS=/path/to/stats.txt
|
||||
if statsDest := os.Getenv("BEADS_MEM_STATS"); statsDest != "" {
|
||||
if statsDest := memDiagnosticsDest(os.Getenv("BEADS_MEM_STATS")); statsDest != "" {
|
||||
var ms runtime.MemStats
|
||||
runtime.ReadMemStats(&ms)
|
||||
if f, err := os.Create(statsDest); err == nil { // #nosec G304 -- user-supplied profiling path
|
||||
|
|
|
|||
|
|
@ -22,11 +22,12 @@ var sendMetricsCmd = &cobra.Command{
|
|||
// That same early exit means this Run returns before Cobra ever reaches
|
||||
// PersistentPostRunE in main.go, so --mem-profile/BEADS_MEM_PROFILE/
|
||||
// BEADS_MEM_STATS are silently inert for this subcommand unless honored
|
||||
// here directly (be-wwy2.2). There is no --mem-profile flag on this
|
||||
// hidden command, so pass "" and let it fall through to the env var,
|
||||
// same as PersistentPostRunE does.
|
||||
// here directly (be-wwy2.2). --mem-profile is registered on
|
||||
// rootCmd.PersistentFlags(), so this hidden command inherits it and
|
||||
// prints it in --help; pass memProfilePath exactly as PersistentPostRunE
|
||||
// does, or the advertised flag would be parsed and then discarded.
|
||||
code := metrics.RunSendMetrics()
|
||||
writeMemDiagnostics("")
|
||||
writeMemDiagnostics(memProfilePath)
|
||||
os.Exit(code)
|
||||
},
|
||||
}
|
||||
|
|
|
|||
|
|
@ -12,6 +12,34 @@ import (
|
|||
"github.com/steveyegge/beads/internal/metrics"
|
||||
)
|
||||
|
||||
// memStatsLine is the exact shape writeMemDiagnostics' MemStats summary writes.
|
||||
var memStatsLine = regexp.MustCompile(`^HeapAlloc=\d+ HeapSys=\d+ HeapInuse=\d+ HeapObjects=\d+\n$`)
|
||||
|
||||
// requireHeapProfile fails unless path holds a real heap profile.
|
||||
// runtime/pprof.WriteHeapProfile always gzips its protobuf output, so
|
||||
// decompressing cleanly to a non-empty body is strong evidence a genuine profile
|
||||
// was written -- without pulling in a pprof-parsing dependency just for this
|
||||
// test. knob names the surface under test so a failure says which one broke.
|
||||
func requireHeapProfile(t *testing.T, knob, path string) {
|
||||
t.Helper()
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("%s file was not written: %v", knob, err)
|
||||
}
|
||||
defer f.Close()
|
||||
gz, err := gzip.NewReader(f)
|
||||
if err != nil {
|
||||
t.Fatalf("%s is not valid gzip: %v", knob, err)
|
||||
}
|
||||
body, err := io.ReadAll(gz)
|
||||
if err != nil {
|
||||
t.Fatalf("%s gzip stream corrupt: %v", knob, err)
|
||||
}
|
||||
if len(body) == 0 {
|
||||
t.Fatalf("%s decompressed to 0 bytes, want a real heap profile", knob)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSendMetricsHonorsMemDiagnostics is the be-wwy2.2 regression: send-metrics's
|
||||
// Run calls os.Exit() directly, so it returns before Cobra ever reaches
|
||||
// PersistentPostRunE (main.go) -- the one place --mem-profile / BEADS_MEM_PROFILE /
|
||||
|
|
@ -63,9 +91,8 @@ func TestSendMetricsHonorsMemDiagnostics(t *testing.T) {
|
|||
if err != nil {
|
||||
t.Fatalf("BEADS_MEM_STATS file was not written: %v", err)
|
||||
}
|
||||
want := regexp.MustCompile(`^HeapAlloc=\d+ HeapSys=\d+ HeapInuse=\d+ HeapObjects=\d+\n$`)
|
||||
if !want.Match(data) {
|
||||
t.Fatalf("BEADS_MEM_STATS content = %q, want match of %s", data, want)
|
||||
if !memStatsLine.Match(data) {
|
||||
t.Fatalf("BEADS_MEM_STATS content = %q, want match of %s", data, memStatsLine)
|
||||
}
|
||||
})
|
||||
|
||||
|
|
@ -76,24 +103,81 @@ func TestSendMetricsHonorsMemDiagnostics(t *testing.T) {
|
|||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("bd send-metrics: %v\noutput: %s", err, out)
|
||||
}
|
||||
f, err := os.Open(profilePath)
|
||||
if err != nil {
|
||||
t.Fatalf("BEADS_MEM_PROFILE file was not written: %v", err)
|
||||
requireHeapProfile(t, "BEADS_MEM_PROFILE", profilePath)
|
||||
})
|
||||
|
||||
// --mem-profile is registered on rootCmd.PersistentFlags() (main.go), so this
|
||||
// hidden subcommand inherits it and advertises it in --help. The call site
|
||||
// used to hardcode "", which parsed the flag and then discarded it -- the same
|
||||
// "accepted but silently inert" state be-wwy2.2 exists to remove, one line
|
||||
// below the fix. No BEADS_MEM_PROFILE here, so only the flag can produce the
|
||||
// file.
|
||||
t.Run("--mem-profile flag is honored, not just the env var", func(t *testing.T) {
|
||||
profilePath := filepath.Join(t.TempDir(), "flag.pprof")
|
||||
cmd := exec.Command(bdBin, metrics.SendMetricsSubcommand, "--mem-profile="+profilePath)
|
||||
cmd.Env = baseEnv(t.TempDir())
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("bd send-metrics --mem-profile: %v\noutput: %s", err, out)
|
||||
}
|
||||
defer f.Close()
|
||||
// runtime/pprof.WriteHeapProfile always gzips its protobuf output;
|
||||
// decompressing cleanly is strong evidence a real profile was written
|
||||
// without pulling in a pprof-parsing dependency just for this test.
|
||||
gz, err := gzip.NewReader(f)
|
||||
if err != nil {
|
||||
t.Fatalf("BEADS_MEM_PROFILE is not valid gzip: %v", err)
|
||||
requireHeapProfile(t, "--mem-profile", profilePath)
|
||||
})
|
||||
|
||||
// The detached flusher child inherits the parent's BEADS_MEM_* paths verbatim
|
||||
// (flusherChildEnv strips only the endpoint and the flusher marker), and
|
||||
// MaybeSpawnFlusher runs on main()'s post-ExecuteC tail -- after
|
||||
// PersistentPostRunE already wrote them. Without a distinct destination the
|
||||
// child's trivial profile would silently replace the profile of the command
|
||||
// the user actually asked about. BD_IS_FLUSHER=1 is what production sets on
|
||||
// the child, so driving it here is the real shape, not a contrivance.
|
||||
t.Run("flusher child writes beside the parent's files instead of over them", func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
profilePath := filepath.Join(dir, "heap.pprof")
|
||||
statsPath := filepath.Join(dir, "stats.txt")
|
||||
|
||||
// Stand in for the parent command's own writeMemDiagnostics call, which
|
||||
// has already happened by the time the flusher is spawned.
|
||||
parentProfile := []byte("parent heap profile, must survive the spawn\n")
|
||||
parentStats := []byte("parent memstats, must survive the spawn\n")
|
||||
if err := os.WriteFile(profilePath, parentProfile, 0o600); err != nil {
|
||||
t.Fatalf("seed parent profile: %v", err)
|
||||
}
|
||||
body, err := io.ReadAll(gz)
|
||||
if err != nil {
|
||||
t.Fatalf("BEADS_MEM_PROFILE gzip stream corrupt: %v", err)
|
||||
if err := os.WriteFile(statsPath, parentStats, 0o600); err != nil {
|
||||
t.Fatalf("seed parent stats: %v", err)
|
||||
}
|
||||
if len(body) == 0 {
|
||||
t.Fatalf("BEADS_MEM_PROFILE decompressed to 0 bytes, want a real heap profile")
|
||||
|
||||
cmd := exec.Command(bdBin, metrics.SendMetricsSubcommand)
|
||||
cmd.Env = append(baseEnv(t.TempDir()),
|
||||
"BEADS_MEM_PROFILE="+profilePath,
|
||||
"BEADS_MEM_STATS="+statsPath,
|
||||
metrics.EnvIsFlusher+"=1",
|
||||
)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("bd send-metrics as flusher child: %v\noutput: %s", err, out)
|
||||
}
|
||||
|
||||
// The parent's data is untouched, byte for byte.
|
||||
if got, err := os.ReadFile(profilePath); err != nil {
|
||||
t.Fatalf("parent profile disappeared: %v", err)
|
||||
} else if string(got) != string(parentProfile) {
|
||||
t.Fatalf("flusher child overwrote the parent's heap profile: got %q, want %q", got, parentProfile)
|
||||
}
|
||||
if got, err := os.ReadFile(statsPath); err != nil {
|
||||
t.Fatalf("parent stats disappeared: %v", err)
|
||||
} else if string(got) != string(parentStats) {
|
||||
t.Fatalf("flusher child overwrote the parent's MemStats: got %q, want %q", got, parentStats)
|
||||
}
|
||||
|
||||
// ...and the child's own diagnostics are still produced, just beside
|
||||
// them. This is the half that distinguishes suffixing the destination
|
||||
// from simply scrubbing the vars out of the child's environment, which
|
||||
// would reinstate the inertness be-wwy2.2 removes.
|
||||
requireHeapProfile(t, "flusher child BEADS_MEM_PROFILE", profilePath+".send-metrics")
|
||||
childStats, err := os.ReadFile(statsPath + ".send-metrics")
|
||||
if err != nil {
|
||||
t.Fatalf("flusher child wrote no MemStats of its own: %v", err)
|
||||
}
|
||||
if !memStatsLine.Match(childStats) {
|
||||
t.Fatalf("flusher child MemStats content = %q, want match of %s", childStats, memStatsLine)
|
||||
}
|
||||
})
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue