Maintainer fixup for the choose-your-own-storage-backend PR, amended across the
pre-review CI-repair and the review-loop apply-fixes iterations so contributor
commits stay intact under one maintainer commit.
CI repair (pre-review):
- default.nix: bump vendorHash for the added Postgres/MySQL driver deps.
- conformance.yml: provision the Postgres service DB with a C-locale cluster so
text ORDER BY sorts by code point, matching Dolt's utf8mb4_0900_bin oracle.
Review fixes — blockers:
- uow/doltserver_tx: on a non-serialization DOLT_COMMIT failure, ROLLBACK on the
pinned session before releasing it, and poison (pool-discard via driver.ErrBadConn)
the connection when the rollback also fails, so a later borrower can't inherit an
open transaction and implicitly commit orphaned writes. Satisfies the existing
doltserver_tx_test rollback/poison cases.
- sqlitedialect: translate GREATEST/LEAST to SQLite scalar max/min so the
ReconcileChildCounters upsert (GREATEST(last_child, ?)) no longer aborts creation
or import of hierarchical dotted issue IDs (e.g. x-1.2) on the SQLite backend.
Review fixes — majors:
- conformance: add a portable dotted hierarchical child-ID scenario (x-1, x-1.1,
x-1.2, x-1.2.1) exercising the ReconcileChildCounters GREATEST path on every
backend — the coverage hole that let the SQLite blocker ship green. Verified
against the embedded-Dolt oracle and the real SQLite driver.
- postgres: warn at provision/open when the database collation is not code-point
ordered (C/POSIX/C.*), since text ORDER BY otherwise diverges from Dolt silently.
- sqlite: pin the pool to one connection and add busy_timeout + WAL to the DSN so
overlapping operations wait instead of surfacing a raw "database is locked"
(matches the Dolt backend's single-connection posture). Adds a concurrency guard.
- cmd/bd: gate explicit `bd dolt commit` / `bd vc commit` with a typed
storage.ErrUnsupported on non-Dolt backends instead of the no-op Commit printing
"Committed."; internal opportunistic auto-commit is untouched.
Review fixes — attempt 2 (stale init backend-flag test contract):
- cmd/bd/init_test, init_embedded_test: the init backend-flag tests still asserted
the pre-#3151 one-backend contract (--backend=sqlite must fail "SQLite backend
has been removed"; --backend=postgres is an "unknown backend"), so
`go test ./cmd/bd -run '^TestInitBackendFlag$'` failed in a clean environment now
that sqlite/postgres/mysql are supported pluggable backends. Rewrite
TestInitBackendFlag and the cgo TestEmbeddedInit sibling (rejected_backends →
sql_backend_flags) for the new contract: --backend=sqlite initializes and
persists backend=sqlite; --backend=postgres is recognized and fails only on
missing connection config (not "unknown backend"); a genuinely unknown backend
is still rejected. Pin BEADS_DIR and scrub BEADS_*/BD_* so the subtests are
hermetic against an ambient .beads or Dolt server.
Cleanups (non-gating):
- cmd/bd/init: extract checkExistingSQLBackend so the added SQL-backend branches
don't inflate checkExistingBeadsDataAt's complexity, and load config once.
- configfile: correct the stale "Deprecated: always dolt" Backend field comment.
Review fixes -- attempt 3 (SQLite datetime parity blocker + conformance/rewrite
hardening of the storage-parity gate):
- sqlite/dsn: add _time_format=datetime. Without it modernc binds time.Time via
t.String() ("2026-07-09 12:34:56.123 +0000 UTC"), a shape SQLite's date functions
cannot parse, so the counts mega-query's DATE_FORMAT(created_at,...) (strftime on
SQLite) returns NULL and deps_json exposes a zero created_at while Dolt emits the
real timestamp. datetime stores whole-second "2026-07-09 12:34:56", matching
Dolt/MySQL datetime(0). Verified end-to-end against the real modernc driver.
- conformance: add ReadyWorkDepCreatedAtParity -- imports a dependency with a known
created_at (the bd import / CreateIssuesWithFullOptions bound-time.Time path) and
asserts the counts-rendered deps_json created_at equals it, closing the blind spot
that let the SQLite blocker ship green (the counts tests only checked DependencyCount).
Proven to fail on the pre-fix DSN and pass on both the fixed SQLite backend and the
embedded-Dolt oracle.
- conformance: add ClaimReadyIssueConcurrentExclusivity -- races 24 workers over 8
ready issues and asserts each is claimed exactly once, exercising the SQL backends'
multi-writer claim property that was previously only asserted, never raced. Gated to
the multi-connection dialects (postgres/mysql) via DialectName; SQLite (single-conn
pool) and embedded-Dolt skip.
- conformance.sh: run the embedded-Dolt reference conformance with
BEADS_TEST_EMBEDDED_DOLT=1 and fail loudly if it skips. It self-skips without the env
var, so the parity gate was reporting success while never running the reference oracle.
- mysqldialect: drive the is_blocked-recompute error-1093 rewrite test from the real
issueops templates (new issueops.SelfReferentialRecomputeTemplates) instead of a
hand-copied fixture, so template drift fails a non-gated test rather than only the
env-gated MySQL conformance suite. All four mark/unmark x issue/wisp templates
verified rewritten into the materialized derived-table form.
Note: the review re-flagged TestInitBackendFlag as failing, but it passes in a clean run
(plain and gms_pure_go, matching the scorecard's own rerun); the only cmd/bd failure in
this sandbox is an unrelated pre-existing embedded-Dolt/CGO test not in this PR's diff.
Review fixes -- attempt 4 (conformance-gate self-brick + all-wisps inline-dependency
drop blockers; Postgres error-log password-leak major):
- conformance.sh: anchor the embedded-Dolt reference skip/pass guards to top-level
result lines (^--- SKIP/PASS: TestConformance). The attempt-3 loud-fail guard used
an unanchored grep, so the indented, legitimate subtest skip of
ClaimReadyIssueConcurrentExclusivity (embedded-Dolt is single-writer) matched it and
false-failed the entire storage-parity gate on every run even when the top-level
suite passed. Whole-suite-skip protection is preserved.
- sqlkit/portable + embeddeddolt/create_issue: the all-wisps arm looped the
single-issue issueops.CreateIssueInTx, which runs no batch dependency-persist pass,
so an all-wisps batch carrying inline dependencies (e.g. importing no-history beads
with inter-bead edges -- export DOES include no-history beads, GH#2619) silently
dropped every edge (not even reported via OnSkippedDependency) and, one transaction
per issue, forfeited batch atomicity. Both now mark ephemeral, then run the whole
batch in ONE transaction through issueops.CreateIssuesInTx, matching the server-mode
DoltStore all-wisps arm. The embedded-Dolt reference oracle had the same latent drop,
so the oracle and the new SQL backends were corrected together to keep conformance
parity (the finding compared sqlkit against the server-mode DoltStore, which already
persisted these edges).
- conformance: add CreateAllWispsInlineDependencies -- an all-wisp batch with one valid
inter-wisp edge and one unresolvable edge; asserts the valid edge persists and the
unresolvable one is reported via OnSkippedDependency. Fails on the pre-fix per-issue
loop, passes on the batch path; verified on SQLite and the embedded-Dolt oracle, with
a full embedded-Dolt conformance re-run clean.
- pgdialect: scrub query-param/sslpassword secrets from pgx ParseConfig errors in Open
and OpenRaw (new ScrubDSNError). pgx's ParseConfigError redacts only URL userinfo, so
a postgres://u@h/db?password=SECRET DSN (the shape bd init connects with) leaked the
cleartext secret through the wrapped parse/TLS-config error into logs. Connect-time
errors were verified not to carry the password. Adds redact_error_test coverage for
the query-param, sslpassword, userinfo, and libpq-keyword shapes plus the real
Open/OpenRaw parse-failure path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
190 lines
5.7 KiB
Go
190 lines
5.7 KiB
Go
package pgdialect
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"database/sql/driver"
|
|
|
|
"github.com/jackc/pgx/v5"
|
|
"github.com/jackc/pgx/v5/stdlib"
|
|
)
|
|
|
|
// Open returns a *sql.DB over pgx whose Prepare/PrepareContext translate bd's
|
|
// canonical (MySQL-dialect) SQL to Postgres via Translate. searchPath pins the
|
|
// workspace schema.
|
|
//
|
|
// This wrapper deliberately exposes no Queryer/Execer fast path, so database/sql
|
|
// routes every statement through Prepare — translation therefore happens in
|
|
// exactly one place, and an untranslatable construct fails loudly at PREPARE.
|
|
func Open(dsn, searchPath string) (*sql.DB, error) {
|
|
cfg, err := pgx.ParseConfig(dsn)
|
|
if err != nil {
|
|
// pgx's ParseConfigError leaves ?password= URL query params cleartext; scrub
|
|
// before the error can reach any log.
|
|
return nil, ScrubDSNError(dsn, err)
|
|
}
|
|
if searchPath != "" {
|
|
if cfg.RuntimeParams == nil {
|
|
cfg.RuntimeParams = map[string]string{}
|
|
}
|
|
cfg.RuntimeParams["search_path"] = searchPath
|
|
}
|
|
return sql.OpenDB(&connector{base: stdlib.GetConnector(*cfg)}), nil
|
|
}
|
|
|
|
// OpenRaw returns a *sql.DB over pgx WITHOUT the translating connector wrapper,
|
|
// so SQL text reaches Postgres verbatim. It exists for schema DDL: the DDL is
|
|
// already native Postgres, and routing it through Translate mangles the
|
|
// $$-quoted function bodies and rewrites DDL constructs (e.g. now()). searchPath
|
|
// pins the workspace schema exactly as Open does. Callers must use native $n
|
|
// placeholders, not ?, since Translate is bypassed.
|
|
func OpenRaw(dsn, searchPath string) (*sql.DB, error) {
|
|
cfg, err := pgx.ParseConfig(dsn)
|
|
if err != nil {
|
|
// See Open: pgx does not redact query-param passwords in parse errors.
|
|
return nil, ScrubDSNError(dsn, err)
|
|
}
|
|
if searchPath != "" {
|
|
if cfg.RuntimeParams == nil {
|
|
cfg.RuntimeParams = map[string]string{}
|
|
}
|
|
cfg.RuntimeParams["search_path"] = searchPath
|
|
}
|
|
return sql.OpenDB(stdlib.GetConnector(*cfg)), nil
|
|
}
|
|
|
|
type connector struct{ base driver.Connector }
|
|
|
|
func (c *connector) Connect(ctx context.Context) (driver.Conn, error) {
|
|
cn, err := c.base.Connect(ctx)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &conn{base: cn}, nil
|
|
}
|
|
|
|
func (c *connector) Driver() driver.Driver { return c.base.Driver() }
|
|
|
|
// conn wraps a pgx driver.Conn and translates SQL text at prepare time.
|
|
type conn struct{ base driver.Conn }
|
|
|
|
func (c *conn) Prepare(query string) (driver.Stmt, error) {
|
|
q, err := Translate(query)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
base, err := c.base.Prepare(q)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &stmt{base: base, conn: c.base}, nil
|
|
}
|
|
|
|
func (c *conn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
|
|
q, err := Translate(query)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
var base driver.Stmt
|
|
if p, ok := c.base.(driver.ConnPrepareContext); ok {
|
|
base, err = p.PrepareContext(ctx, q)
|
|
} else {
|
|
base, err = c.base.Prepare(q)
|
|
}
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &stmt{base: base, conn: c.base}, nil
|
|
}
|
|
|
|
func (c *conn) Close() error { return c.base.Close() }
|
|
|
|
//nolint:staticcheck // SA1019: driver.Conn requires the deprecated Begin; BeginTx is preferred and forwarded below.
|
|
func (c *conn) Begin() (driver.Tx, error) { return c.base.Begin() }
|
|
|
|
func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
|
|
if b, ok := c.base.(driver.ConnBeginTx); ok {
|
|
return b.BeginTx(ctx, opts)
|
|
}
|
|
return c.base.Begin() //nolint:staticcheck
|
|
}
|
|
|
|
func (c *conn) ResetSession(ctx context.Context) error {
|
|
if rs, ok := c.base.(driver.SessionResetter); ok {
|
|
return rs.ResetSession(ctx)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (c *conn) IsValid() bool {
|
|
if v, ok := c.base.(driver.Validator); ok {
|
|
return v.IsValid()
|
|
}
|
|
return true
|
|
}
|
|
|
|
func (c *conn) Ping(ctx context.Context) error {
|
|
if p, ok := c.base.(driver.Pinger); ok {
|
|
return p.Ping(ctx)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// stmt wraps a pgx driver.Stmt so bd's MySQL TINYINT(1) discipline survives on
|
|
// Postgres: bd binds Go bool arguments to columns that are smallint here (not
|
|
// boolean), so CheckNamedValue coerces bool -> 0/1 before binding. Every other
|
|
// argument type is forwarded to pgx's own NamedValueChecker so pgx's rich type
|
|
// handling (arrays, time, etc.) is preserved.
|
|
type stmt struct {
|
|
base driver.Stmt
|
|
conn driver.Conn
|
|
}
|
|
|
|
func (s *stmt) CheckNamedValue(nv *driver.NamedValue) error {
|
|
if b, ok := nv.Value.(bool); ok {
|
|
nv.Value = boolToInt64(b)
|
|
return nil
|
|
}
|
|
if chk, ok := s.conn.(driver.NamedValueChecker); ok {
|
|
return chk.CheckNamedValue(nv)
|
|
}
|
|
return driver.ErrSkip // fall back to database/sql's default conversion
|
|
}
|
|
|
|
func (s *stmt) Close() error { return s.base.Close() }
|
|
func (s *stmt) NumInput() int { return s.base.NumInput() }
|
|
|
|
//nolint:staticcheck // driver.Stmt requires the deprecated Exec; ExecContext is preferred and forwarded below.
|
|
func (s *stmt) Exec(args []driver.Value) (driver.Result, error) { return s.base.Exec(args) }
|
|
|
|
//nolint:staticcheck // driver.Stmt requires the deprecated Query; QueryContext is preferred and forwarded below.
|
|
func (s *stmt) Query(args []driver.Value) (driver.Rows, error) { return s.base.Query(args) }
|
|
|
|
func (s *stmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
|
|
if e, ok := s.base.(driver.StmtExecContext); ok {
|
|
return e.ExecContext(ctx, args)
|
|
}
|
|
return s.base.Exec(namedToValues(args)) //nolint:staticcheck
|
|
}
|
|
|
|
func (s *stmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
|
|
if q, ok := s.base.(driver.StmtQueryContext); ok {
|
|
return q.QueryContext(ctx, args)
|
|
}
|
|
return s.base.Query(namedToValues(args)) //nolint:staticcheck
|
|
}
|
|
|
|
func boolToInt64(b bool) int64 {
|
|
if b {
|
|
return 1
|
|
}
|
|
return 0
|
|
}
|
|
|
|
func namedToValues(named []driver.NamedValue) []driver.Value {
|
|
vals := make([]driver.Value, len(named))
|
|
for i, n := range named {
|
|
vals[i] = n.Value
|
|
}
|
|
return vals
|
|
}
|