beads/internal/storage/doltutil
matt wilkie a4cc7ee29c
perf(dolt): pin MaxAllowedPacket in the server DSN to drop a per-connection round trip (#5273 item 4) (#5316)
* perf(dolt): pin MaxAllowedPacket in the server DSN (#5273 item 4)

Both DSN builders construct mysql.Config as a composite literal rather than
via mysql.NewConfig(), so MaxAllowedPacket is left at zero. FormatDSN emits
maxAllowedPacket=0, and go-sql-driver/mysql then runs

    SELECT @@max_allowed_packet

while establishing every new connection, because its connector takes that
probe whenever cfg.MaxAllowedPacket <= 0. That is an extra round-trip per
connection, paid on every pool expansion — the cost that matters against a
high-latency remote Dolt server, where connections are opened far more often
than the packet limit ever changes.

Pin the value to the driver's own defaultMaxAllowedPacket (64 MiB), which is
what a mysql.NewConfig() caller already gets, so the packet limit itself is
unchanged. The constant is duplicated because the driver keeps it unexported.
FormatDSN omits the parameter entirely when it equals that default and
ParseDSN starts from NewConfig(), so the value survives the round-trip
through the DSN string without being written into it.

Tests assert the parsed value rather than the DSN text, for that reason: a
substring check would pass for the wrong reason. Verified RED by reverting
just the field — both new tests fail with "got 0", and the failure message
shows the offending DSN carrying maxAllowedPacket=0.

Companion to #4617 (interpolateParams in the same file), and item 4 of the
PR #4167 showcase salvage list.

Credit: the round-trip analysis originates in the showcase work of
Hans Boese (@Shockwave2k) in #4167 / #4165.

Agent-Signature: claude-opus-5-high on behalf of maphew

* fix(dolt): pin the packet ceiling to Dolt's 1 GiB, not the driver's 64 MiB

Cross-vendor review (codex gpt-5.6-sol) caught a real regression in the
previous commit: pinning MaxAllowedPacket to go-sql-driver/mysql's own
defaultMaxAllowedPacket (64 MiB) does skip the per-connection probe, but it
also lowers the client's ceiling far below the server's.

Verified against the engine rather than assumed. go-mysql-server declares:

    "max_allowed_packet": &sql.MysqlSystemVariable{
        Type:    types.NewSystemUintType("max_allowed_packet", 1024, 1073741824),
        Default: int64(1073741824),
    }

so Dolt defaults to 1 GiB and its sysvar type caps the value there — no Dolt
server can be configured to accept a larger packet. Under the old probing
behavior the driver set its ceiling from that server value; a 64 MiB pin
would instead have made the driver reject oversized statements locally with
ErrPktTooLarge, breaking large imports and base64 content that previously
succeeded.

Pin 1 GiB instead. The round-trip saving is unchanged (the probe is taken
whenever cfg.MaxAllowedPacket <= 0, not when it differs from any particular
value), and the client now never rejects a packet the server would accept.

Trade-off, stated in the code: the probe read the server's *configured*
value, so an operator who lowers max_allowed_packet below 1 GiB no longer
gets a client-side rejection — the server rejects instead. That moves the
error to the authoritative side and never silently truncates.

Both tests now assert the ceiling is at least Dolt's 1 GiB, so a future
change back to the driver default fails rather than silently shrinking the
limit.

Agent-Signature: claude-opus-5-high on behalf of maphew

* build(bazel): add the mysql dep to the two dsn go_test targets (make bazel-sync)

Both dsn_test.go files now import github.com/go-sql-driver/mysql directly
(for mysql.ParseDSN), so gazelle emits the dep on each go_test. Without it,
bazel.yml's "BUILD files in sync" step fails in the required bazel-test lane.
Generated by `make bazel-sync`; matches the drift that step reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(dolt): correct the max_allowed_packet trade-off comment

Maintainer fixup for PR #5316 review iteration 1. The synthesis and the
quality scorecard both approve (989/1000, no blocker or major findings).
This is the one non-gating minor; both review lanes flagged it
independently.

The maxAllowedPacketBytes doc comment said that when an operator lowers
max_allowed_packet below 1 GiB, "the server rejects the statement instead".
Dolt does not do that. In the engine versions this module pins, the sysvar
has one non-test reader, go-mysql-server's LOAD_FILE
(sql/expression/function/load_file.go). vitess's MaxPacketSize is only the
16 MiB wire-framing constant, and vitess reassembles multi-packet payloads
without any size check. dolt/go never reads the sysvar. A lowered setting
is therefore honored nowhere on the statement path, which matches the PR
description's own analysis. The last paragraph now says so.

The paragraph above it rested on the same premise ("no Dolt server can be
configured to accept a larger packet than this"). Left alone, it would
contradict the corrected paragraph. It now states what the type cap does
guarantee: no Dolt server can report a larger value, so the old probe
could never have produced a higher ceiling than this pin.

Comment-only; no code or test change. The contributor's second commit
message still has the old wording. Contributor commits stay untouched to
preserve authorship, so this comment is the corrected record.

Two optional nits were left as-is:
- Asserting == 1<<30 instead of >= in both tests. The tests are correct
  as written, and their stated purpose is a lower bound ("must not shrink
  what the client will send"). The upward-drift harm the nit describes
  assumes the server enforces the sysvar, which the fix above shows it
  does not.
- The pin is one byte above the old probe's n-1 ceiling. That difference
  can't be reached in practice or observed without server-side
  enforcement, and the synthesis lists leaving it as-is as its first option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 01:10:37 -07:00
..
BUILD.bazel perf(dolt): pin MaxAllowedPacket in the server DSN to drop a per-connection round trip (#5273 item 4) (#5316) 2026-10-01 01:10:37 -07:00
cli_mirror_ref_test.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
close.go test: drive close timeout with a deadline event (bd-c3qkw) (#5262) 2026-08-01 19:49:49 -07:00
close_test.go test: drive close timeout with a deadline event (bd-c3qkw) (#5262) 2026-08-01 19:49:49 -07:00
dsn.go perf(dolt): pin MaxAllowedPacket in the server DSN to drop a per-connection round trip (#5273 item 4) (#5316) 2026-10-01 01:10:37 -07:00
dsn_test.go perf(dolt): pin MaxAllowedPacket in the server DSN to drop a per-connection round trip (#5273 item 4) (#5316) 2026-10-01 01:10:37 -07:00
identifier.go fix(doctor,doltserver): escape SHOW DATABASES identifiers at raw-interpolation sites (#6434) (#6638) 2026-09-26 14:26:04 -07:00
identifier_test.go fix(doctor,doltserver): escape SHOW DATABASES identifiers at raw-interpolation sites (#6434) (#6638) 2026-09-26 14:26:04 -07:00
persisted_remotes_test.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
probe_pid_other.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
probe_pid_unix.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
probe_pid_windows.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
remotes.go feat(storage): carry the git data ref of a git-backed Dolt remote (#6521) 2026-09-27 15:56:18 -07:00
remotes_test.go perf(dolt): eliminate 12s slow path on bd commands when .local_version is stale (#3710) 2026-07-24 00:53:06 -07:00