* ci(bazel): raise proxied-server tier from 15 to 30 shards (F2)
bazel.yml's bazel-proxied lane (//cmd/bd:bd_proxied_test, run only with
remote execution) was sharing PR Risk's legacy 15-shard, frozen
bd-init-cost-proxy manifest block, so its longest shard was the long
pole of the Bazel-only lanes. Give it its own 30-shard block in
.github/scripts/proxied-cmd-test-shards.txt, bin-packed
longest-processing-time-first by measured wall-time
(scripts/ci/proxied_test_durations.json, captured from two bazel-proxied
job failures' testlogs) instead of the bd-init count proxy, via
scripts/ci/gen_proxied_shard_manifest.py's new --weights=duration mode.
Coverage is proven equal to the full discovered test set by two
independent methods (sorted-name diff and an actual 30-shard invocation
of proxied-test-shard.sh in list-only mode), both exact matches with no
duplicates.
cmd/bd:bd_proxied_test's shard_count moves to 30; PR Risk's and
main.yml's own test-proxied-cmd jobs keep their 15-shard matrix and
frozen manifest block untouched, per D2's retirement design (the Bazel
lane only needs to run the same tests, not the same split). Updated the
two coupled policy tests (scripts/ci_workflow_test.go's
TestBazelDoltServerTiersMirrorPRRisk, scripts/pr_risk_bazel_coverage_test.go's
TestBazelRetiredLanesCheckListedTestsRan) to allow the Bazel target's
shard_count to differ from its legacy job's own matrix size, and
engdocs/CI_REQUIRED_CHECK_TOPOLOGY.md / engdocs/TESTING.md to document
the split.
Verified: go test ./scripts/...; bazel --nohome_rc test
//scripts:scripts_test; actionlint on bazel.yml; make bazel-sync-check;
make ci-pr-policy; 3 representative shards (1, 15, 30 of 30) run
directly against Bazel-built race bd_test/bd_for_tests binaries and the
pinned dolt 2.2.0 release, all green, 55-81s wall time each (the
duration data's absolute seconds are from a contention-era sample and
run far faster here; the LPT balance itself, not the absolute SLA, is
the property that matters).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): close fork-PR gap in proxied 30-shard manifest validation (review B1/S1)
TestShardScriptsListOnlyRealTests only ran proxied-test-shard.sh at PR
Risk's legacy 15-shard matrix, never at the Bazel-only bazel-proxied
lane's 30-shard manifest block. bazel-proxied only runs in mode=remote
(skipped on fork PRs), so a fork PR could rename, duplicate, or corrupt
a TestProxiedServer*/TestServerMode* manifest entry in the 30-block and
merge green via go test ./scripts/... (which runs on every PR), only
breaking the required lane afterward on main and every same-repo PR.
Fix: the test now computes every shard total it must validate per
script (this job's own PR Risk matrix size, every distinct total found
in that script's own committed manifest, and - for proxied-test-shard.sh
- cmd/bd:bd_proxied_test's live shard_count from BUILD.bazel) and runs
the full per-shard validation loop at each one, relying on the script's
own duplicate/stale-name guards. It also sums each total's "manifest: N"
counts across all shards and fails if they sum to zero, catching a
silently-deleted block that would otherwise fall back to 100% hashing
undetected.
Also fold in S1: replace the independently hard-coded `30` overrides in
ci_workflow_test.go and pr_risk_bazel_coverage_test.go with
bazelProxiedShardCount(t), a new shared helper that reads
cmd/bd:bd_proxied_test's shard_count live from cmd/bd/BUILD.bazel. This
turns the check_shard_coverage.py argument comparison in
pr_risk_bazel_coverage_test.go into a genuine cross-file pin (BUILD.bazel
vs bazel.yml) instead of two literals that could drift together
unnoticed. ci_workflow_test.go's tiers loop no longer asserts
BUILD.bazel's shard_count against a value read from the same file
(tautological); it instead asserts pr-risk.yml's and main.yml's legacy
test-proxied-cmd matrices equal each other, restoring the pin S1 found
removed on this branch relative to base.
Verified against the reviewer's mutations (duplicate entry, bogus
entry, whole-block deletion, stale rename, matrix divergence, BUILD
shard_count drift, bazel.yml arg drift): each now fails go test
./scripts/..., where it previously passed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): add generator --write/--check and freeze legacy manifest block (review S3/N1/N4/N5)
gen_proxied_shard_manifest.py:
- Add --write (rewrite only the target total_shards block in place,
leaving every other block's header/body text untouched) and --check
(regenerate in memory and diff against the committed block, exiting
1 with the exact --write command needed if stale). Both operate via
new split_blocks()/write_block()/check_block() helpers that round-trip
the real manifest file losslessly.
- N1: fix the duration-weights header's false claim about the
fallback ratio.
- N5: warn on stderr when proxied_test_durations.json has duration
captures for tests that no longer exist in cmd/bd/*_test.go (renamed,
split, or deleted since the last capture).
proxied-cmd-test-shards.txt:
- N4: mark the legacy 15-shard block's header as frozen for
pr-risk.yml's/main.yml's legacy fork/push jobs: it is deliberately
not kept in sync with newly-added tests (those hash-fallback
instead), so regenerating it is not routine maintenance. Only the
header text changed; all 149 explicit assignment lines and the
"known blind spots" callouts are untouched (verified: a fresh
--write would pack all 164 currently-discovered tests explicitly,
silently changing which shard runs which test for this block, which
is exactly what "frozen" means to avoid).
- Regenerate the Bazel-only 30-shard block's header via
`gen_proxied_shard_manifest.py 30 --weights=duration --write` to
describe --write/--check instead of the old manual-copy-paste
instructions. proxied_test_durations.json has not changed yet (S2
is still in progress), so this run only touched header text; the 164
assignment lines are unchanged.
pr_risk_bazel_coverage_test.go:
- Add TestProxiedShardManifestGeneratorNotStale: runs
gen_proxied_shard_manifest.py 30 --weights=duration --check under
plain `go test` (skipped under `bazel test`, whose runfiles hold
neither the generator's sources nor cmd/bd) so a stale 30-block
(e.g. committed by hand, or left behind after adding tests without
regenerating) fails scripts-go-checks on every PR, including forks.
tools/bazel/equivalence_allowlist.txt: allowlist the new test's guarded
Bazel skip.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): rename 14 undiscovered proxied-gated tests; N2/N3/N6 doc fixes
cmd/bd/*_test.go: 14 tests call requireProxiedServerEnv/
requireSharedProxiedServer but their names did not match the shard
scripts' discovery regex ^Test(ProxiedServer|ServerMode), so no CI lane
ever set BEADS_TEST_PROXIED_SERVER=1 for them and they silently skipped
everywhere (pre-existing gap, not caused by F2; list was in
~/beads-bazel-plan/f2/review/undiscovered_gated.txt).
Fixed by renaming rather than widening discovery: 162 of the 164
already-discovered tests call one of the two gate functions in the first
few lines of their body, so name and gate already correlate almost
perfectly, but the 14 tests' names are each styled differently with no
single regex term that would cover all of them without a hand-maintained
list. Renaming lets the existing discovery mechanism, hash-fallback
behavior, and both shard-coverage policy tests pick them up with zero
other changes. Checked each of the 14 names for cross-references first;
each appeared only in its own file plus at most one doc-comment in a
sibling file, both updated alongside the rename. No collisions with
existing test names.
Discovery went from 164 to 178 tests. Regenerated the Bazel-only 30-shard
block (gen_proxied_shard_manifest.py 30 --weights=duration --write) to
pack the 14 new tests; this is the first --write this round to actually
change assignments, not just header text. The frozen legacy 15-shard
block (N4) was deliberately left alone: the 14 renamed tests now
hash-fallback into it, exactly like every other test added since it was
last generated.
cmd/bd/BUILD.bazel (N6): restore the bd_proxied_test comment sentence
bazel.yml's mirror comment kept but BUILD.bazel's had dropped: PR Risk
builds its bd-cmd-test with -race, as here; main.yml's test-proxied-cmd
twin uses build-artifacts' non-race bd-cmd-test, so this lane is the
stricter of the two.
.github/workflows/bazel.yml, .bazelrc (N2/N3): remove stale timing
language ("30 shards of ~5-10 minutes" / ".bazelrc's leftover "15 shards
of ~10 minutes") never reconciled with this task's own local
representative-shard measurements (55-81s/shard); replaced with the
concurrency argument (30 simultaneous dolt sql-servers don't fit a
GitHub-hosted runner regardless of any single shard's wall time). Also
removes the "within 5s of each other" balance claim (S2): the underlying
duration weights are contaminated by -test.parallel overlap in the
top-level <testsuite time> figures they were derived from; a corrected
re-measurement is in progress (separate commit once it lands).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied duration weights without overlap double-counting (review S2)
scripts/ci/proxied_test_durations.json's 178 entries are now each a
clean, single-job (no concurrent heavy bazel/dolt job on the box),
-test.parallel=1 measurement: a fresh manifest-aligned 30-shard sweep's
own <testsuite time> per top-level test where errors="0" (including
failing-but-complete suites, whose elapsed time is still trustworthy),
plus, for the handful too slow to finish inside Bazel's default
per-target timeout, a `bazel run` leaf-subtest-only sum that never adds
a parallel group's own PASS/FAIL line on top of its already-overlapping
children's lines. An earlier pass at this data summed every PASS/FAIL
line indiscriminately, double-counting parallel subtest groups.
A first attempt at re-measuring the slow stragglers overlapped in
wall-clock time with a second, unrelated heavy sweep and came out 2-3x
inflated versus the same tests measured cleanly; that contaminated data
was discarded in favor of isolated re-runs. TestProxiedServerDep2
initially looked hung during its isolated re-run (15+ minutes with no
PASS/FAIL output and no CPU growth in the top-level test binary) but
was not: each of its subtests does real `bd init --proxied-server`
work in a short-lived child process, and Go buffers a parallel
subtest group's output until the group's last member finishes. Letting
it run to completion (2357.72s) rather than guessing avoided reporting
bad data.
seconds_per_init_fallback is recomputed as the weighted average
seconds-per-init across all 178 measured tests (sum of durations / sum
of inits), for use only when a brand-new test has no entry yet.
Re-packing the 30-shard manifest with this corrected data changes the
shard-balance picture: TestProxiedServerDep2 and TestProxiedServerClose
are each large enough, alone, to dominate their own shard (~2358s and
~1105s locally) while every other shard clusters around 741-876s. The
"within 5s" claim this replaces was never earned by the underlying
data; .github/workflows/bazel.yml now documents the real, uneven
spread and flags that the two outlier shards' local numbers already
exceed the job's 25m step timeout on their own merits (unverified
against RBE's actual per-shard throughput, since this job has only
ever run remote).
Verified: go vet ./scripts/..., go test ./scripts/... (includes the
manifest/coverage policy tests), actionlint, make bazel-sync-check,
make ci-pr-policy all pass. Exact-once coverage re-confirmed: 178
manifest assignments for 178 discovered tests, zero duplicates, zero
gaps.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cmd/bd): apply re-review candidate fixes; revert 2 unready renames (R-B1)
Re-review found the prior round's 14 renames would turn the required
Bazel + legacy 15-shard lanes red once discovered:
- TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix/external-unix and
TestProxiedServerOutageReconnectAcceptanceMatrix/external-unix-socket
built a dolt.sock path under t.TempDir(), which nests Bazel's hermetic
tmp-root plus the (64-byte-truncated) test name deep enough to exceed
sockaddr_un's 108-byte sun_path limit. Both now use a new shortSocketPath
helper (a short os.MkdirTemp("", "bds") directory) instead.
- runProxiedDeadline raced a deadline-watcher goroutine against
exec.Cmd.Start()/Run()'s write to cmd.Process, caught by -race. Replaced
with cmd.Cancel, which exec.CommandContext only invokes after Start, so
the read is ordered after Start's write.
With those two fixes applied, TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix
passes outright. TestProxiedServerOutageReconnectAcceptanceMatrix's race is
fixed, but both its subtests still fail a real "exceeded 3s bound" assertion
against a stopped external upstream -- a genuine latency issue, not a test
bug -- so it is reverted back to its original name
(TestProxiedOutageReconnectAcceptanceMatrix) with a comment explaining why,
pending a bead against the underlying latency.
TestServerModeSharedServeGlobalReportsTheServedDatabase also reverted back
to TestSharedServerModeServeGlobalReportsTheServedDatabase: `bd serve
--global` refuses to auto-apply 4 pending schema migrations (v65->v69) to a
shared/proxied server database and exits 1 with a consent-required usage
message instead of starting, per the #5920 migration-consent guard. Also a
genuine product/test-fixture question, not fixed here; reverted with a
comment, pending a bead.
The other 12 of the original 14 renames are kept; all 12 verified passing
in both lanes (see commits that follow for the manifest update and full
verification).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): re-derive proxied weights from parallel=4 CI-identical sweep (R-S1)
The prior round's duration data (committed in 6be9605d9f) was captured at
-test.parallel=1 (serial). That fixed a real double-counting bug, but
serial timings don't model this lane, which always runs -test.parallel=4
(--config=doltserver-proxied's own setting). Re-review measured near-zero
correlation (r=0.08) between the serial-derived predicted shard load and
actual parallel=4 wall time -- e.g. TestProxiedServerRecomputeBlocked was
weighted 812s serially but measured 33s actual at parallel=4.
Replaced with a full fresh capture: `bazel --nohome_rc test
--config=doltserver-proxied //cmd/bd:bd_proxied_test` (local execution
only, the exact config bazel.yml's proxied-server tier uses), 30/30 shards
passing, 0 failures, 0 races. Every one of the 176 discovered tests'
top-level <testsuite time> was captured directly from this run's
bazel-testlogs test.xml files -- no leaf-summation workaround was needed
this time, since none of the 176 hit Bazel's per-target default timeout
running alongside the rest of its shard.
Re-ran `gen_proxied_shard_manifest.py 30 --weights=duration --repack
--write` with these corrected weights: a deliberate full reweight, not an
incremental add, since every weight changed. The new packing clusters all
30 shards within a few seconds of ~355-360s each (176 tests summing to
10712s / 30), a dramatic improvement over the serial data's wrongly
implied two-outlier-shard spread (TestProxiedServerDep2 alone at ~2358s,
TestProxiedServerClose alone at ~1105s -- both artifacts of serial
timing, not real per-shard cost). Updated bazel.yml's paragraph to report
this honestly instead of the prior, false "exceeds 25-minute timeout"
claim, which was itself downstream of the serial/parallel mismatch.
Verified: the 12 tests kept renamed in the previous commit, plus
TestProxiedServerHistoryRemoteRefusalFrontDoorMatrix's socket-path fix,
all pass under the new 30-shard Bazel manifest (this capture run) and
under the legacy 15-shard hash-fallback script (ran the 7 shards their
hash lands in: 2, 3, 4, 5, 8, 10, 11 -- all pass).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(ci,bazel): incremental --write, --repack, name-coverage --check (R-S2)
The prior --check compared the committed 30-shard block byte-for-byte
against a fresh from-scratch LPT pack, and --write always did a full
repack. That meant any weight drift (even from proxied_test_durations.json
alone, with no test added/renamed/removed) failed --check, and two
unrelated PRs each adding one proxied test would both force a full
repack and conflict on unrelated shard lines -- re-review demonstrated
one new test moving 73 assignment lines under the old full-repack
behavior.
Redesigned:
- --check now verifies exact-once NAME coverage of the currently
discovered TestProxiedServer*/TestServerMode* set against the
committed block -- missing, stale, or duplicated names fail with an
actionable message -- but does NOT compare shard assignments, so a
weights-only change no longer forces --check to fail.
- --write is now incremental by default: every already-assigned test
keeps its current shard, stale (no longer discovered) names are
dropped, and only newly-discovered names are LPT-placed onto whichever
shard is currently lightest. A full from-scratch LPT pack now requires
the explicit --repack flag (used in the preceding commit for the
deliberate full reweight; ordinary new-test additions should use the
incremental default).
Mutation-tested directly (not just reviewed): verified --check correctly
flags a stale name as "stale/unknown" and a newly added test as "missing"
with the exact --write command to fix it; verified incremental --write
touches only the lines for the added/removed names; verified --repack
--write produces a materially different (full rebalance) diff from the
incremental default.
Updated TestProxiedShardManifestGeneratorNotStale's comment in
pr_risk_bazel_coverage_test.go to describe the new name-coverage-only
--check semantics (it previously described the old byte-identical-packing
behavior; the test's own assertion -- exit code only -- was already
correct and unaffected).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
251 lines
9.8 KiB
Go
251 lines
9.8 KiB
Go
//go:build cgo && unix
|
|
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"net/http"
|
|
"testing"
|
|
)
|
|
|
|
// The events journal over HTTP, end to end through a real `bd serve`.
|
|
//
|
|
// The handler tests in internal/httpapi drive fakes, which is the right shape
|
|
// for the refusal vocabulary and cannot answer the question this file exists
|
|
// for: does a mutation that arrives over the HTTP surface actually reach the
|
|
// journal the same surface then serves? Every layer between those two facts is
|
|
// real here — the server-mode provider serve builds for itself, the activation
|
|
// it resolves from the workspace, the journal writes inside the mutation's own
|
|
// transaction, and the read plumbing that pages them back out.
|
|
//
|
|
// It is the composition that has failed before (see the note on
|
|
// TestProxiedServerServeActivatesTheEventsJournal): each half looked correct in isolation
|
|
// while the server committed with an empty journal and every response looked
|
|
// normal.
|
|
|
|
// TestProxiedServerServeReadsTheJournalItJustWrote: mutate over HTTP, then read the journal
|
|
// over HTTP, and require the records to be there, gapless, and paced by a head
|
|
// that matches.
|
|
func TestProxiedServerServeReadsTheJournalItJustWrote(t *testing.T) {
|
|
requireSharedProxiedServer(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
p := newServerModeProject(t, bd, "hje")
|
|
p.env = append(p.env, "BD_EVENTS_JOURNAL=1")
|
|
|
|
sp := startServe(t, bd, p.dir, p.env)
|
|
|
|
// Nothing has been mutated yet: an ENABLED but untouched journal is a 200
|
|
// with an empty page and a head of zero, which is the answer a disabled
|
|
// workspace must never give.
|
|
status, body, _ := sp.get(t, "/v0/beads/events?since=0")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("GET /v0/beads/events on a fresh workspace = %d: %v\nstderr:\n%s", status, body, sp.stderr.String())
|
|
}
|
|
if head, _ := body["head"].(float64); head != 0 {
|
|
t.Errorf("head on a fresh workspace = %v, want 0", body["head"])
|
|
}
|
|
if records, _ := body["records"].([]any); len(records) != 0 {
|
|
t.Errorf("records on a fresh workspace = %v, want none", records)
|
|
}
|
|
|
|
const created = 4
|
|
for i := range created {
|
|
status, body := sp.postJSON(t, "/v0/beads/issues:batchCreate",
|
|
fmt.Sprintf(`{"actor":"tester","items":[{"title":"served mutation %d","issue_type":"task"}]}`, i))
|
|
if status != http.StatusOK {
|
|
t.Fatalf("POST batchCreate %d = %d: %v\nstderr:\n%s", i, status, body, sp.stderr.String())
|
|
}
|
|
}
|
|
|
|
status, body, _ = sp.get(t, "/v0/beads/events?since=0")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("GET /v0/beads/events = %d: %v\nstderr:\n%s", status, body, sp.stderr.String())
|
|
}
|
|
records, _ := body["records"].([]any)
|
|
if len(records) != created {
|
|
t.Fatalf("records = %d, want %d — the HTTP mutations did not reach the journal this surface serves: %v",
|
|
len(records), created, body)
|
|
}
|
|
|
|
// GAPLESS AND IN ORDER, from 1. A consumer's whole contract is that it can
|
|
// resume from the last seq it processed, which is only true if the sequence
|
|
// it sees has no holes.
|
|
for i, raw := range records {
|
|
rec, ok := raw.(map[string]any)
|
|
if !ok {
|
|
t.Fatalf("records[%d] is not an object: %v", i, raw)
|
|
}
|
|
if seq, _ := rec["seq"].(float64); int(seq) != i+1 {
|
|
t.Fatalf("records[%d].seq = %v, want %d — the journal has a gap", i, rec["seq"], i+1)
|
|
}
|
|
if op, _ := rec["op"].(string); op != "create" {
|
|
t.Errorf("records[%d].op = %v, want create", i, rec["op"])
|
|
}
|
|
// The published envelope, not a shape this surface invented: `issue` is
|
|
// always present, and a create carries no dependency half.
|
|
if _, ok := rec["issue"]; !ok {
|
|
t.Errorf("records[%d] has no `issue` member", i)
|
|
}
|
|
if _, ok := rec["dep"]; ok {
|
|
t.Errorf("records[%d] carries `dep` on a create", i)
|
|
}
|
|
}
|
|
|
|
// The head paces the consumer: caught up means last seq == head.
|
|
head, _ := body["head"].(float64)
|
|
if int(head) != created {
|
|
t.Fatalf("head = %v, want %d", body["head"], created)
|
|
}
|
|
|
|
// Resuming from the head is the steady state of a poller, and it must be an
|
|
// ordinary empty 200 rather than any kind of miss.
|
|
status, body, _ = sp.get(t, fmt.Sprintf("/v0/beads/events?since=%d", int(head)))
|
|
if status != http.StatusOK {
|
|
t.Fatalf("caught-up GET = %d: %v", status, body)
|
|
}
|
|
if records, _ := body["records"].([]any); len(records) != 0 {
|
|
t.Errorf("caught-up records = %v, want none", records)
|
|
}
|
|
|
|
// A bounded page still reports the JOURNAL's head, not the page's — that is
|
|
// how the consumer knows to keep reading.
|
|
status, body, _ = sp.get(t, "/v0/beads/events?since=0&limit=2")
|
|
if status != http.StatusOK {
|
|
t.Fatalf("bounded GET = %d: %v", status, body)
|
|
}
|
|
if records, _ := body["records"].([]any); len(records) != 2 {
|
|
t.Fatalf("bounded records = %d, want 2", len(records))
|
|
}
|
|
if h, _ := body["head"].(float64); int(h) != created {
|
|
t.Errorf("bounded page head = %v, want %d — the head must describe the journal", body["head"], created)
|
|
}
|
|
|
|
sp.shutdown(t)
|
|
|
|
// The same records, through the CLI, on the same workspace. This is the
|
|
// claim the shared record projection makes and the reason it is shared: an
|
|
// operator debugging a consumer must be able to reach for `bd events export`
|
|
// and see what the consumer saw.
|
|
exported := decodeEventRecords(t, p.run(t, bd, "events", "export"))
|
|
if len(exported) != created {
|
|
t.Fatalf("bd events export returned %d records, want %d — the CLI and HTTP reads disagree", len(exported), created)
|
|
}
|
|
for i, rec := range exported {
|
|
if rec.Seq != int64(i+1) {
|
|
t.Errorf("exported[%d].seq = %d, want %d", i, rec.Seq, i+1)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestProxiedServerServeRefusesAStaleCheckpointWithGone is the retention boundary over HTTP.
|
|
// A consumer whose checkpoint was pruned past must be TOLD, with the window the
|
|
// server can still serve — not handed the surviving suffix as though it were a
|
|
// complete history, and not handed an empty success it would read as "caught
|
|
// up" and stall on forever.
|
|
//
|
|
// Both retention floors are zeroed so the prune can actually cut: the shipped
|
|
// defaults (7 days / 100k rows) protect everything a test could create, which
|
|
// is the point of them and the reason this has to say so explicitly.
|
|
func TestProxiedServerServeRefusesAStaleCheckpointWithGone(t *testing.T) {
|
|
requireSharedProxiedServer(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
p := newServerModeProject(t, bd, "hjt")
|
|
p.env = append(p.env,
|
|
"BD_EVENTS_JOURNAL=1",
|
|
"BD_EVENTS_JOURNAL_RETAIN_DAYS=0",
|
|
"BD_EVENTS_JOURNAL_RETAIN_ROWS=0",
|
|
// The prune below is the one this test performs. A maintenance ticker
|
|
// cutting on its own schedule would make the window nondeterministic.
|
|
"BD_EVENTS_JOURNAL_AUTO_PRUNE=0",
|
|
)
|
|
|
|
sp := startServe(t, bd, p.dir, p.env)
|
|
|
|
const created = 5
|
|
for i := range created {
|
|
status, body := sp.postJSON(t, "/v0/beads/issues:batchCreate",
|
|
fmt.Sprintf(`{"actor":"tester","items":[{"title":"pruned mutation %d","issue_type":"task"}]}`, i))
|
|
if status != http.StatusOK {
|
|
t.Fatalf("POST batchCreate %d = %d: %v\nstderr:\n%s", i, status, body, sp.stderr.String())
|
|
}
|
|
}
|
|
|
|
// Cut the first three records out from under a consumer sitting at 0.
|
|
p.run(t, bd, "events", "prune", "--before", "4")
|
|
|
|
status, body, _ := sp.get(t, "/v0/beads/events?since=0")
|
|
if status != http.StatusGone {
|
|
t.Fatalf("GET /v0/beads/events?since=0 after a prune = %d, want 410: %v\nstderr:\n%s",
|
|
status, body, sp.stderr.String())
|
|
}
|
|
if code, _ := body["code"].(string); code != "events_journal_truncated" {
|
|
t.Errorf("code = %v, want events_journal_truncated", body["code"])
|
|
}
|
|
// The window, which is the whole payload of this refusal: the consumer's two
|
|
// recoveries are both computed from these numbers.
|
|
for _, tc := range []struct {
|
|
member string
|
|
want int
|
|
}{{"since", 0}, {"floor", 4}, {"head", created}} {
|
|
got, _ := body[tc.member].(float64)
|
|
if int(got) != tc.want {
|
|
t.Errorf("%s = %v, want %d", tc.member, body[tc.member], tc.want)
|
|
}
|
|
}
|
|
|
|
// The documented recovery actually works: resume from floor-1 and the read
|
|
// succeeds with the surviving suffix, gap explicitly accepted.
|
|
floor, _ := body["floor"].(float64)
|
|
status, body, _ = sp.get(t, fmt.Sprintf("/v0/beads/events?since=%d", int(floor)-1))
|
|
if status != http.StatusOK {
|
|
t.Fatalf("resume from floor-1 = %d, want 200: %v", status, body)
|
|
}
|
|
records, _ := body["records"].([]any)
|
|
if len(records) != created-int(floor)+1 {
|
|
t.Fatalf("resumed records = %d, want %d", len(records), created-int(floor)+1)
|
|
}
|
|
first, _ := records[0].(map[string]any)
|
|
if seq, _ := first["seq"].(float64); int(seq) != int(floor) {
|
|
t.Errorf("resumed records[0].seq = %v, want the floor %d", first["seq"], int(floor))
|
|
}
|
|
|
|
sp.shutdown(t)
|
|
}
|
|
|
|
// TestProxiedServerServeRefusesTheJournalWhenItIsDisabled is the distinction the data cannot
|
|
// make. This workspace never enabled the journal, so it records nothing and
|
|
// never will — and the read has to say so rather than answer the empty page an
|
|
// enabled-but-untouched workspace answers, which a consumer would poll against
|
|
// forever.
|
|
func TestProxiedServerServeRefusesTheJournalWhenItIsDisabled(t *testing.T) {
|
|
requireSharedProxiedServer(t)
|
|
t.Parallel()
|
|
bd := buildEmbeddedBD(t)
|
|
p := newServerModeProject(t, bd, "hjd")
|
|
// Deliberately NOT enabling the journal. The default is off.
|
|
|
|
sp := startServe(t, bd, p.dir, p.env)
|
|
|
|
// A mutation lands, and is not journaled, because nothing is journaling.
|
|
status, postBody := sp.postJSON(t, "/v0/beads/issues:batchCreate",
|
|
`{"actor":"tester","items":[{"title":"unjournaled","issue_type":"task"}]}`)
|
|
if status != http.StatusOK {
|
|
t.Fatalf("POST batchCreate = %d: %v\nstderr:\n%s", status, postBody, sp.stderr.String())
|
|
}
|
|
|
|
status, body, _ := sp.get(t, "/v0/beads/events?since=0")
|
|
if status != http.StatusConflict {
|
|
t.Fatalf("GET /v0/beads/events on a disabled workspace = %d, want 409: %v\nstderr:\n%s",
|
|
status, body, sp.stderr.String())
|
|
}
|
|
if code, _ := body["code"].(string); code != "events_journal_disabled" {
|
|
t.Errorf("code = %v, want events_journal_disabled", body["code"])
|
|
}
|
|
if detail, _ := body["detail"].(string); detail == "" {
|
|
t.Error("no detail on the disabled refusal")
|
|
}
|
|
|
|
sp.shutdown(t)
|
|
}
|