* fix(doctor,doltserver): escape SHOW DATABASES identifiers at raw-interpolation sites Three call sites interpolated a SHOW DATABASES-derived name into SQL without doubling backticks, so a database name containing a backtick could break out of the identifier quoting: - cmd/bd/doctor/dolt.go probeForCorrectDatabase (issues probe) - internal/doltserver/doltserver.go flush routine (dolt_status probe and USE) Extract each query into a small helper (issuesProbeQuery, doltStatusQuery, useDatabaseStatement) that applies the same escape already used by inspectServerMetadataDatabases, and unit-test them without a live Dolt server. Fixes #6434 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(doctor): escape the bd doctor --fix probe via a shared doltutil helper Maintainer review fixes for PR #6638 (review iterations 1 and 2). The review's one major (Change Impact / Blast Radius) is that the PR escapes three of four identical SHOW DATABASES interpolation sites. probeForCorrectDoltDatabase (cmd/bd/doctor/fix/metadata.go) is a near-line-for-line clone of the probeForCorrectDatabase this PR fixed -- same enumeration, same system-DB skip map, same testdb_/doctest_/ doctortest_ prefix filter, byte-identical query string -- and the two are the check and fix halves of one GH#2160 wrong-database repair. So after this PR "bd doctor" detects a backticked database and emits "Fix: Run 'bd doctor --fix' to set dolt_database=X", while "bd doctor --fix" reaches the still-raw twin, whose SELECT errors, yielding correctDB == "" and a bare return nil: no fix, no error, no output. The PR turned a consistent double failure into a repair tool that tells the user to run a command that silently does nothing. Take the review's preferred single remedy, which also closes the Architectural Consistency minor: land one exported doltutil.QuoteIdentifier and route every site through it. internal/storage/doltutil was already imported by the affected packages (cmd/bd/doctor, cmd/bd/doctor/fix, internal/doltserver), so only cmd/bd gains an import edge. - new doltutil.QuoteIdentifier, returning the backtick-quoted form so quoting and escaping cannot be applied separately; unit-tested for plain, empty, embedded/leading backtick, and backtick-only names - route all five hand-inlined backtick-doubling copies through it: the three builders this PR added, the pre-existing DROP DATABASE in cmd/bd/dolt_clean_databases.go, and inspectServerMetadataDatabases. Non-test inline copies go from five to one -- the helper itself - drop the surviving //nolint:gosec rationale "dbName from SHOW DATABASES, not user input" at fix/metadata.go, which documented as safe exactly the premise this PR exists to refute - pin the call site, not only the builder: a sqlmock test asserts probeForCorrectDoltDatabase issues the escaped query. Reverting the escape fails it with probeForCorrectDoltDatabase() = "", the silent no-op described above, while the plain-name case stays green Left raw deliberately, as both review lanes concluded: doltserver.go's CREATE DATABASE takes the constant GlobalDatabaseName, and the raw USE in internal/testutil takes test-controlled names. Review iteration 2 found no blocker or major. Its one minor is that the consolidation re-created the ambiguity it removed, one level up: the repo already exported a QuoteIdentifier in internal/storage/domain/db with the opposite contract -- allowlist-based, returns an error, rejects anything outside validIdentifier -- whose docstring claimed to be "the one place this repository's identifiers are quoted", a sentence the new helper falsified while staking a competing near-universal claim of its own. - rename doltutil.QuoteIdentifier to QuoteIdentifierUnvalidated, and have both docstrings state their input class and cross-reference the other: db.QuoteIdentifier for bd-minted names that must satisfy the allowlist before the database or table is addressed, the doltutil helper for server-reported names bd does not control. The escape-over-reject behavior does not change -- rejecting a SHOW DATABASES name would skip the candidate, which is the defect this PR exists to fix. The new name also keeps Quote, so callers do not re-wrap an already-quoted result - scope the helper's "every identifier that is not a compile-time constant or already validated" rule to non-test code: internal/testutil still interpolates raw, on test-controlled names, and has no non-test importer - pin dropStaleDatabases (cmd/bd/dolt_clean_databases.go), the one destructive statement in the set. versioncontrolops.DBConn is already a parameter there, so a recording fake needs no server and no sqlmock, and the test is tagless where the sibling file is cgo-gated. Reverting that call site to raw interpolation fails it with "DROP DATABASE `evil`; DROP TABLE x`" -- the injected statement escaping the identifier -- while the plain-name statement stays byte-identical The remaining unpinned site, inspectServerMetadataDatabases, opens its own *sql.DB inside openServerCatalogDB, so pinning it needs a production seam rather than a test. It was already correctly escaped before this PR and is unchanged by it; left as reviewer-noted follow-up, as is nolintlint, which .golangci.yml still disables. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
153 lines
4.3 KiB
Go
153 lines
4.3 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"os"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/steveyegge/beads/internal/storage/doltutil"
|
|
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
|
|
)
|
|
|
|
type cleanDatabasesOptions struct {
|
|
dryRun bool
|
|
purgeDropped bool
|
|
}
|
|
|
|
const (
|
|
cleanDatabasesListTimeout = 30 * time.Second
|
|
cleanDatabasesDropTimeout = 30 * time.Second
|
|
cleanDatabasesBatchSize = 5
|
|
cleanDatabasesBatchPause = 2 * time.Second
|
|
cleanDatabasesBackoffPause = 10 * time.Second
|
|
cleanDatabasesTimeoutTrip = 3
|
|
cleanDatabasesMaxFailures = 10
|
|
)
|
|
|
|
func listStaleDatabases(ctx context.Context, conn versioncontrolops.DBConn) ([]string, error) {
|
|
listCtx, cancel := context.WithTimeout(ctx, cleanDatabasesListTimeout)
|
|
defer cancel()
|
|
|
|
rows, err := conn.QueryContext(listCtx, "SHOW DATABASES")
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
defer rows.Close()
|
|
|
|
var stale []string
|
|
for rows.Next() {
|
|
var dbName string
|
|
if err := rows.Scan(&dbName); err != nil {
|
|
return nil, err
|
|
}
|
|
for _, prefix := range staleDatabasePrefixes {
|
|
if strings.HasPrefix(dbName, prefix) {
|
|
stale = append(stale, dbName)
|
|
break
|
|
}
|
|
}
|
|
}
|
|
if err := rows.Err(); err != nil {
|
|
return nil, err
|
|
}
|
|
return stale, nil
|
|
}
|
|
|
|
func cleanDatabases(ctx context.Context, conn versioncontrolops.DBConn, opts cleanDatabasesOptions) error {
|
|
stale, err := listStaleDatabases(ctx, conn)
|
|
if err != nil {
|
|
return HandleError("listing databases: %v", err)
|
|
}
|
|
|
|
if len(stale) == 0 {
|
|
fmt.Println("No stale databases found.")
|
|
} else {
|
|
fmt.Printf("Found %d stale databases:\n", len(stale))
|
|
for _, name := range stale {
|
|
fmt.Printf(" %s\n", name)
|
|
}
|
|
}
|
|
|
|
if opts.dryRun {
|
|
if len(stale) > 0 {
|
|
fmt.Println("\n(dry run — no databases dropped)")
|
|
}
|
|
if opts.purgeDropped {
|
|
fmt.Println("(dry run — --purge-dropped ignored; no purge performed)")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
dropped, err := dropStaleDatabases(ctx, conn, stale)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
fmt.Println()
|
|
if shouldPurgeDroppedDatabases(opts.purgeDropped, dropped) {
|
|
if err := purgeDroppedDatabases(ctx, conn); err != nil {
|
|
fmt.Fprintf(os.Stderr, " WARN: PURGE_DROPPED_DATABASES failed: %v\n", err)
|
|
fmt.Fprintln(os.Stderr, " Try `dolt sql -q 'CALL DOLT_PURGE_DROPPED_DATABASES()'`.")
|
|
} else {
|
|
fmt.Println("Purged all dropped databases on this server (server-global, irreversible —")
|
|
fmt.Println("CALL DOLT_UNDROP is no longer available for any of them, not just this run's).")
|
|
}
|
|
} else {
|
|
fmt.Println("Dropped databases remain recoverable via `CALL DOLT_UNDROP(name)` until purged.")
|
|
fmt.Println("Pass --purge-dropped to permanently reclaim their disk. This purges ALL dropped")
|
|
fmt.Println("databases on the server (server-global), not just the ones from this run.")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func dropStaleDatabases(ctx context.Context, conn versioncontrolops.DBConn, stale []string) (int, error) {
|
|
if len(stale) == 0 {
|
|
return 0, nil
|
|
}
|
|
|
|
fmt.Println()
|
|
dropped := 0
|
|
failures := 0
|
|
consecutiveTimeouts := 0
|
|
|
|
for i, name := range stale {
|
|
if consecutiveTimeouts >= cleanDatabasesTimeoutTrip {
|
|
fmt.Fprintf(os.Stderr, " ⚠ %d consecutive timeouts — backing off %s\n",
|
|
consecutiveTimeouts, cleanDatabasesBackoffPause)
|
|
time.Sleep(cleanDatabasesBackoffPause)
|
|
consecutiveTimeouts = 0
|
|
}
|
|
|
|
if failures >= cleanDatabasesMaxFailures {
|
|
fmt.Fprintf(os.Stderr, "\n✗ Aborting: %d consecutive failures suggest server is unhealthy.\n", failures)
|
|
fmt.Fprintf(os.Stderr, " Dropped %d/%d before stopping.\n", dropped, len(stale))
|
|
return dropped, SilentExit()
|
|
}
|
|
|
|
dropCtx, dropCancel := context.WithTimeout(ctx, cleanDatabasesDropTimeout)
|
|
//nolint:gosec // G201: identifier quoted+escaped via doltutil.QuoteIdentifierUnvalidated
|
|
_, err := conn.ExecContext(dropCtx, fmt.Sprintf("DROP DATABASE %s", doltutil.QuoteIdentifierUnvalidated(name)))
|
|
dropCancel()
|
|
if err != nil {
|
|
fmt.Fprintf(os.Stderr, " FAIL: %s: %v\n", name, err)
|
|
failures++
|
|
if isTimeoutError(err) {
|
|
consecutiveTimeouts++
|
|
}
|
|
} else {
|
|
fmt.Printf(" Dropped: %s\n", name)
|
|
dropped++
|
|
failures = 0
|
|
consecutiveTimeouts = 0
|
|
}
|
|
|
|
if (i+1)%cleanDatabasesBatchSize == 0 && i+1 < len(stale) {
|
|
fmt.Printf(" [%d/%d] pausing %s...\n", i+1, len(stale), cleanDatabasesBatchPause)
|
|
time.Sleep(cleanDatabasesBatchPause)
|
|
}
|
|
}
|
|
fmt.Printf("\nDropped %d/%d stale databases.\n", dropped, len(stale))
|
|
return dropped, nil
|
|
}
|