ci(proxied-shard): discover TestServerMode* entry points

Review finding (major, Test Evidence Quality): TestServerModeDeferAutoWake —
this PR's only end-to-end guard for the classified-read defer-wake fix — was
selected by no CI lane. The two lanes that set BEADS_TEST_PROXIED_SERVER=1
both shard through proxied-test-shard.sh, whose discovery grep matched only
`^func TestProxiedServer`, so the wiring lines at cmd/bd/main.go:1675 and
internal/storage/dolt/store.go:2136 had zero CI-executed coverage: the one
CI-reachable new test pins the pure predicate and cannot catch a mis-wired
flag.

Widen discovery to `^func Test(ProxiedServer|ServerMode)` in the shard script
and in scripts/ci/gen_proxied_shard_manifest.py so the two discovery patterns
stay in step, and widen the shard script's manifest-name validation to match.
That last part is load-bearing, not cosmetic: without it a regenerated
manifest containing a TestServerMode* row is rejected outright ("Invalid
shard manifest line") and the whole lane exits 1.

This also rescues the two pre-existing TestServerModeServe* suites, which
share the identical requireSharedProxiedServer gate and the same shared dolt
sql-server harness and were equally invisible — the unclosed class this PR
was following.

Verified: discovery goes 151 -> 154; a 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
dry run assigns each of the 154 to exactly one shard with no pre-existing
test moved (the three land on shards 4/13/15 via hash fallback); the
pre-change script selects none of the three.

Also clarifies the reviewers' nit: the "foreign-project" exclusion named in
the new comments is not tested at the guard or in the policy expression —
those opens build their own dolt.Config and leave ClassifiedRead false, so
the guarantee is fail-closed by default.

Iteration-2 review (two minors, both non-gating): the fixup's own new prose
asserted two universals that measurement falsifies — the same class as the
nit above, reintroduced in the CI files this fixup added.

  * proxied-test-shard.sh claimed "Both families gate on
    requireSharedProxiedServer", contradicted two lines earlier by its own
    mention of the TestProxiedServerPathHelpers unit test. Transitive gate
    census over the 154 discovered entry points: 143 reach
    requireSharedProxiedServer, 10 reach the different helper
    requireProxiedServerEnv, and TestProxiedServerPathHelpers
    (cmd/bd/proxied_server_test.go, bare `package main`, three assert.Equal
    calls on path helpers) has no gate and no build tag, so it also runs in
    every ordinary `go test ./cmd/bd/` lane. Now scoped to the
    TestServerMode* family, which does gate uniformly (3/3), and the
    TestProxiedServer* side is described as mixed rather than generalized
    over.

  * main.yml claimed TestServerMode* discovery means "neither family can be
    silently left out of CI". Discovery is a non-recursive cmd/bd/*_test.go
    glob: 154 top-level matches versus 156 recursive, the difference being
    cmd/bd/doctor/database_test.go:60 and cmd/bd/doctor/dolt_test.go:126.
    Now scoped to the top-level entry points, and the comment names the
    residual hazard (a container-gated suite added under cmd/bd/<subpkg>/ is
    still never selected here) instead of implying it cannot happen.

Both edits are comment-only and behaviorally inert: non-comment lines are
byte-identical in both files, and the 15-shard BEADS_TEST_SHARD_LIST_ONLY=1
sweep is byte-identical before and after (154 assignments, 0 duplicates,
assigned set == discovered set, the three TestServerMode* still at 4/13/15).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Gas City Adopt-PR 2026-09-28 13:10:04 +00:00
commit 0a3f5e0047
7 changed files with 62 additions and 25 deletions

View file

@ -5,8 +5,9 @@
# 15-shard split, bin-packed longest-processing-time-first by
# estimated cost (bd-init count; per-init migration chains dominate).
# Regenerate with scripts/ci/gen_proxied_shard_manifest.py after adding,
# splitting, or reweighting TestProxiedServer* functions. Newly-added
# tests not listed here hash-distribute via proxied-test-shard.sh.
# splitting, or reweighting TestProxiedServer*/TestServerMode*
# functions. Newly-added tests not listed here hash-distribute via
# proxied-test-shard.sh.
#
# This file is generated: notes added here are erased by the next
# regeneration. Add them to the generator instead. Known blind spots

View file

@ -3,13 +3,24 @@
#
# Usage: proxied-test-shard.sh <shard_number> <total_shards>
#
# Discovers all TestProxiedServer* top-level functions across cmd/bd/*_test.go
# (the proxied integration suites plus TestProxiedServerExternalCreate and the
# TestProxiedServerPathHelpers unit test, matching the lane's historical
# `-run '^TestProxiedServer'`). Tests listed in proxied-cmd-test-shards.txt for
# the requested shard count use that committed assignment; newly-added tests
# fall back to hash(name) % total. Runs the matching subset using the pre-built
# test binary at BEADS_TEST_CMD_BINARY (or /tmp/bd-cmd-test).
# Discovers all TestProxiedServer* and TestServerMode* top-level functions
# across cmd/bd/*_test.go (the proxied integration suites plus
# TestProxiedServerExternalCreate and the TestProxiedServerPathHelpers unit
# test, widening the lane's historical `-run '^TestProxiedServer'`). The
# TestServerMode* suites all gate on requireSharedProxiedServer and drive the
# same shared dolt sql-server, and this lane is the only one that sets
# BEADS_TEST_PROXIED_SERVER=1, so it is the only place they run: before the
# widening a top-level TestServerMode* entry point matched no discovery pattern
# here, so it skipped everywhere else and was never executed by CI at all.
# The TestProxiedServer* side is mixed and should not be generalized over: most
# gate on requireSharedProxiedServer, the rest on requireProxiedServerEnv, and
# TestProxiedServerPathHelpers (pure path helpers) has no gate and no build tag,
# so that one also runs in every ordinary `go test ./cmd/bd/` lane.
#
# Tests listed in proxied-cmd-test-shards.txt for the requested shard count use
# that committed assignment; newly-added tests fall back to hash(name) % total.
# Runs the matching subset using the pre-built test binary at
# BEADS_TEST_CMD_BINARY (or /tmp/bd-cmd-test).
#
# Environment:
# BEADS_TEST_PROXIED_SERVER=1 required (tests skip without it)
@ -36,16 +47,17 @@ fi
SHARD_INDEX=$(( SHARD_NUMBER - 1 ))
MANIFEST="${BEADS_TEST_SHARD_MANIFEST:-.github/scripts/proxied-cmd-test-shards.txt}"
# Discover all top-level TestProxiedServer* functions across every cmd/bd test
# file (integration suites live in *_proxied_integration_test.go, but
# TestProxiedServerExternalCreate and TestProxiedServerPathHelpers do not, so
# match by name rather than by file glob).
ALL_TESTS=$(grep -rh '^func TestProxiedServer' cmd/bd/*_test.go \
| sed 's/func \(TestProxiedServer[A-Za-z0-9_]*\).*/\1/' \
# Discover all top-level TestProxiedServer*/TestServerMode* functions across
# every cmd/bd test file (integration suites live in
# *_proxied_integration_test.go, but TestProxiedServerExternalCreate,
# TestProxiedServerPathHelpers and the TestServerMode* suites do not, so match
# by name rather than by file glob).
ALL_TESTS=$(grep -rhE '^func Test(ProxiedServer|ServerMode)' cmd/bd/*_test.go \
| sed -E 's/func (Test(ProxiedServer|ServerMode)[A-Za-z0-9_]*).*/\1/' \
| sort -u)
if [ -z "$ALL_TESTS" ]; then
echo "No TestProxiedServer* functions found" >&2
echo "No TestProxiedServer*/TestServerMode* functions found" >&2
exit 1
fi
@ -62,7 +74,7 @@ if [ -f "$MANIFEST" ]; then
if [ -z "${manifest_total:-}" ]; then
continue
fi
if [ -n "${extra:-}" ] || ! [[ "$manifest_total" =~ ^[0-9]+$ ]] || ! [[ "$manifest_shard" =~ ^[0-9]+$ ]] || ! [[ "$test_name" =~ ^TestProxiedServer[A-Za-z0-9_]+$ ]]; then
if [ -n "${extra:-}" ] || ! [[ "$manifest_total" =~ ^[0-9]+$ ]] || ! [[ "$manifest_shard" =~ ^[0-9]+$ ]] || ! [[ "$test_name" =~ ^Test(ProxiedServer|ServerMode)[A-Za-z0-9_]+$ ]]; then
echo "Invalid shard manifest line: $line" >&2
exit 1
fi

View file

@ -868,9 +868,14 @@ jobs:
# Reuses the cmd/bd test binary compiled once by build-artifacts (bd-cmd-test):
# the proxied tests share the package and are gated at runtime by
# BEADS_TEST_PROXIED_SERVER=1. Shard assignment is committed in
# .github/scripts/proxied-cmd-test-shards.txt; newly-added TestProxiedServer*
# tests hash-distribute until listed there. Adopt the
# `TestProxiedServer<Subcommand>` naming convention for new entry points.
# .github/scripts/proxied-cmd-test-shards.txt; newly-added
# TestProxiedServer*/TestServerMode* tests hash-distribute until listed there.
# Prefer the `TestProxiedServer<Subcommand>` naming convention for new entry
# points; server-mode-topology suites named TestServerMode* are also
# discovered among the cmd/bd/*_test.go entry points, so neither family is
# left out of this lane. That glob is non-recursive: a container-gated suite
# added under cmd/bd/<subpkg>/ is still never selected here whatever it is
# named, so keep new proxied/server-mode entry points at the top level.
test-proxied-cmd:
name: Test (proxied-server cmd ${{ matrix.shard }}/15)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'

View file

@ -64,7 +64,8 @@ jobs:
# tests live in the same package and are gated at runtime by
# BEADS_TEST_PROXIED_SERVER=1, so no separate build is needed. Shard
# assignment is committed in .github/scripts/proxied-cmd-test-shards.txt;
# newly-added TestProxiedServer* tests hash-distribute until listed there.
# newly-added TestProxiedServer*/TestServerMode* tests hash-distribute until
# listed there.
test-proxied-cmd:
name: Test (Proxied Dolt Cmd ${{ matrix.shard }}/${{ strategy.job-total }})
needs: [detect-ci-tier, build-embedded]

View file

@ -53,6 +53,13 @@ func (s *DoltStore) SearchIssuesWithCounts(ctx context.Context, query string, fi
// CURRENT project) is eligible despite readOnly=true because the store is
// genuinely writable underneath; strict --readonly, preview, and
// foreign-project opens are not (be-vbhpf).
//
// Only the strict --readonly and preview cases are decided here, by
// classifiedRead being false. Foreign-project and other auxiliary opens are
// excluded one layer up: they build their own dolt.Config literals and never
// set ClassifiedRead, so they reach this guard already fail-closed. Do not
// look for a foreign-project term in this predicate — there is none, and
// setting ClassifiedRead on such an open would defeat the exclusion.
func deferWakeSweepEligible(readOnly, classifiedRead bool) bool {
return !readOnly || classifiedRead
}

View file

@ -384,6 +384,12 @@ type Config struct {
// the CURRENT project) rather than strict --readonly, an explicit preview,
// or a foreign-project lookup. Only a classified-read open is eligible for
// the lazy defer-wake sweep (be-vbhpf) — the others must never mutate.
//
// Strict --readonly and preview are excluded by the policy expression that
// computes this field; foreign-project and auxiliary opens are excluded
// because they construct their own Config and leave this at its false zero
// value. That default is the guarantee — setting it true on such an open
// would make it eligible to sweep.
ClassifiedRead bool
// LenientOpen opens the store leniently: a migration gate refusal (#4259)

View file

@ -1,7 +1,11 @@
#!/usr/bin/env python3
"""Regenerate .github/scripts/proxied-cmd-test-shards.txt by cost-balancing.
Cost proxy for each top-level TestProxiedServer* function is its bd-init count
Discovery must stay in step with proxied-test-shard.sh: both select the
TestProxiedServer* and TestServerMode* entry points, which share the
requireSharedProxiedServer gate and the shared dolt sql-server.
Cost proxy for each such top-level function is its bd-init count
(newSharedProxiedProject occurrences) — inits dominate proxied wall-time. Tests
are packed into TOTAL shards longest-processing-time-first so the heaviest shard
is minimized. Prints the manifest to stdout.
@ -13,7 +17,7 @@ import re
import sys
TOTAL = int(sys.argv[1]) if len(sys.argv) > 1 else 15
func_re = re.compile(r'^func (TestProxiedServer[A-Za-z0-9_]+)\(')
func_re = re.compile(r'^func (Test(?:ProxiedServer|ServerMode)[A-Za-z0-9_]+)\(')
costs = {}
for path in glob.glob('cmd/bd/*_test.go'):
@ -51,8 +55,9 @@ out.append('#')
out.append(f'# {TOTAL}-shard split, bin-packed longest-processing-time-first by')
out.append('# estimated cost (bd-init count; per-init migration chains dominate).')
out.append('# Regenerate with scripts/ci/gen_proxied_shard_manifest.py after adding,')
out.append('# splitting, or reweighting TestProxiedServer* functions. Newly-added')
out.append('# tests not listed here hash-distribute via proxied-test-shard.sh.')
out.append('# splitting, or reweighting TestProxiedServer*/TestServerMode*')
out.append('# functions. Newly-added tests not listed here hash-distribute via')
out.append('# proxied-test-shard.sh.')
out.append('#')
out.append('# This file is generated: notes added here are erased by the next')
out.append('# regeneration. Add them to the generator instead. Known blind spots')