beads/cmd/bd/dolt_clean_databases.go
Vishnu J 29bd4eeacd
fix(doctor,doltserver): escape SHOW DATABASES identifiers at raw-interpolation sites (#6434) (#6638)
* fix(doctor,doltserver): escape SHOW DATABASES identifiers at raw-interpolation sites

Three call sites interpolated a SHOW DATABASES-derived name into SQL without
doubling backticks, so a database name containing a backtick could break out
of the identifier quoting:

- cmd/bd/doctor/dolt.go probeForCorrectDatabase (issues probe)
- internal/doltserver/doltserver.go flush routine (dolt_status probe and USE)

Extract each query into a small helper (issuesProbeQuery, doltStatusQuery,
useDatabaseStatement) that applies the same escape already used by
inspectServerMetadataDatabases, and unit-test them without a live Dolt server.

Fixes #6434

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(doctor): escape the bd doctor --fix probe via a shared doltutil helper

Maintainer review fixes for PR #6638 (review iterations 1 and 2).

The review's one major (Change Impact / Blast Radius) is that the PR
escapes three of four identical SHOW DATABASES interpolation sites.
probeForCorrectDoltDatabase (cmd/bd/doctor/fix/metadata.go) is a
near-line-for-line clone of the probeForCorrectDatabase this PR fixed --
same enumeration, same system-DB skip map, same testdb_/doctest_/
doctortest_ prefix filter, byte-identical query string -- and the two are
the check and fix halves of one GH#2160 wrong-database repair. So after
this PR "bd doctor" detects a backticked database and emits
"Fix: Run 'bd doctor --fix' to set dolt_database=X", while "bd doctor
--fix" reaches the still-raw twin, whose SELECT errors, yielding
correctDB == "" and a bare return nil: no fix, no error, no output. The
PR turned a consistent double failure into a repair tool that tells the
user to run a command that silently does nothing.

Take the review's preferred single remedy, which also closes the
Architectural Consistency minor: land one exported
doltutil.QuoteIdentifier and route every site through it.
internal/storage/doltutil was already imported by the affected packages
(cmd/bd/doctor, cmd/bd/doctor/fix, internal/doltserver), so only cmd/bd
gains an import edge.

- new doltutil.QuoteIdentifier, returning the backtick-quoted form so
  quoting and escaping cannot be applied separately; unit-tested for
  plain, empty, embedded/leading backtick, and backtick-only names
- route all five hand-inlined backtick-doubling copies through it: the
  three builders this PR added, the pre-existing DROP DATABASE in
  cmd/bd/dolt_clean_databases.go, and inspectServerMetadataDatabases.
  Non-test inline copies go from five to one -- the helper itself
- drop the surviving //nolint:gosec rationale "dbName from SHOW
  DATABASES, not user input" at fix/metadata.go, which documented as
  safe exactly the premise this PR exists to refute
- pin the call site, not only the builder: a sqlmock test asserts
  probeForCorrectDoltDatabase issues the escaped query. Reverting the
  escape fails it with probeForCorrectDoltDatabase() = "", the silent
  no-op described above, while the plain-name case stays green

Left raw deliberately, as both review lanes concluded: doltserver.go's
CREATE DATABASE takes the constant GlobalDatabaseName, and the raw USE in
internal/testutil takes test-controlled names.

Review iteration 2 found no blocker or major. Its one minor is that the
consolidation re-created the ambiguity it removed, one level up: the repo
already exported a QuoteIdentifier in internal/storage/domain/db with the
opposite contract -- allowlist-based, returns an error, rejects anything
outside validIdentifier -- whose docstring claimed to be "the one place
this repository's identifiers are quoted", a sentence the new helper
falsified while staking a competing near-universal claim of its own.

- rename doltutil.QuoteIdentifier to QuoteIdentifierUnvalidated, and have
  both docstrings state their input class and cross-reference the other:
  db.QuoteIdentifier for bd-minted names that must satisfy the allowlist
  before the database or table is addressed, the doltutil helper for
  server-reported names bd does not control. The escape-over-reject
  behavior does not change -- rejecting a SHOW DATABASES name would skip
  the candidate, which is the defect this PR exists to fix. The new name
  also keeps Quote, so callers do not re-wrap an already-quoted result
- scope the helper's "every identifier that is not a compile-time constant
  or already validated" rule to non-test code: internal/testutil still
  interpolates raw, on test-controlled names, and has no non-test importer
- pin dropStaleDatabases (cmd/bd/dolt_clean_databases.go), the one
  destructive statement in the set. versioncontrolops.DBConn is already a
  parameter there, so a recording fake needs no server and no sqlmock, and
  the test is tagless where the sibling file is cgo-gated. Reverting that
  call site to raw interpolation fails it with
  "DROP DATABASE `evil`; DROP TABLE x`" -- the injected statement escaping
  the identifier -- while the plain-name statement stays byte-identical

The remaining unpinned site, inspectServerMetadataDatabases, opens its own
*sql.DB inside openServerCatalogDB, so pinning it needs a production seam
rather than a test. It was already correctly escaped before this PR and is
unchanged by it; left as reviewer-noted follow-up, as is nolintlint, which
.golangci.yml still disables.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Julian Knutsen <julianknutsen@users.noreply.github.com>
2026-09-26 14:26:04 -07:00

153 lines
4.3 KiB
Go

package main
import (
"context"
"fmt"
"os"
"strings"
"time"
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/storage/versioncontrolops"
)
type cleanDatabasesOptions struct {
dryRun bool
purgeDropped bool
}
const (
cleanDatabasesListTimeout = 30 * time.Second
cleanDatabasesDropTimeout = 30 * time.Second
cleanDatabasesBatchSize = 5
cleanDatabasesBatchPause = 2 * time.Second
cleanDatabasesBackoffPause = 10 * time.Second
cleanDatabasesTimeoutTrip = 3
cleanDatabasesMaxFailures = 10
)
func listStaleDatabases(ctx context.Context, conn versioncontrolops.DBConn) ([]string, error) {
listCtx, cancel := context.WithTimeout(ctx, cleanDatabasesListTimeout)
defer cancel()
rows, err := conn.QueryContext(listCtx, "SHOW DATABASES")
if err != nil {
return nil, err
}
defer rows.Close()
var stale []string
for rows.Next() {
var dbName string
if err := rows.Scan(&dbName); err != nil {
return nil, err
}
for _, prefix := range staleDatabasePrefixes {
if strings.HasPrefix(dbName, prefix) {
stale = append(stale, dbName)
break
}
}
}
if err := rows.Err(); err != nil {
return nil, err
}
return stale, nil
}
func cleanDatabases(ctx context.Context, conn versioncontrolops.DBConn, opts cleanDatabasesOptions) error {
stale, err := listStaleDatabases(ctx, conn)
if err != nil {
return HandleError("listing databases: %v", err)
}
if len(stale) == 0 {
fmt.Println("No stale databases found.")
} else {
fmt.Printf("Found %d stale databases:\n", len(stale))
for _, name := range stale {
fmt.Printf(" %s\n", name)
}
}
if opts.dryRun {
if len(stale) > 0 {
fmt.Println("\n(dry run — no databases dropped)")
}
if opts.purgeDropped {
fmt.Println("(dry run — --purge-dropped ignored; no purge performed)")
}
return nil
}
dropped, err := dropStaleDatabases(ctx, conn, stale)
if err != nil {
return err
}
fmt.Println()
if shouldPurgeDroppedDatabases(opts.purgeDropped, dropped) {
if err := purgeDroppedDatabases(ctx, conn); err != nil {
fmt.Fprintf(os.Stderr, " WARN: PURGE_DROPPED_DATABASES failed: %v\n", err)
fmt.Fprintln(os.Stderr, " Try `dolt sql -q 'CALL DOLT_PURGE_DROPPED_DATABASES()'`.")
} else {
fmt.Println("Purged all dropped databases on this server (server-global, irreversible —")
fmt.Println("CALL DOLT_UNDROP is no longer available for any of them, not just this run's).")
}
} else {
fmt.Println("Dropped databases remain recoverable via `CALL DOLT_UNDROP(name)` until purged.")
fmt.Println("Pass --purge-dropped to permanently reclaim their disk. This purges ALL dropped")
fmt.Println("databases on the server (server-global), not just the ones from this run.")
}
return nil
}
func dropStaleDatabases(ctx context.Context, conn versioncontrolops.DBConn, stale []string) (int, error) {
if len(stale) == 0 {
return 0, nil
}
fmt.Println()
dropped := 0
failures := 0
consecutiveTimeouts := 0
for i, name := range stale {
if consecutiveTimeouts >= cleanDatabasesTimeoutTrip {
fmt.Fprintf(os.Stderr, " ⚠ %d consecutive timeouts — backing off %s\n",
consecutiveTimeouts, cleanDatabasesBackoffPause)
time.Sleep(cleanDatabasesBackoffPause)
consecutiveTimeouts = 0
}
if failures >= cleanDatabasesMaxFailures {
fmt.Fprintf(os.Stderr, "\n✗ Aborting: %d consecutive failures suggest server is unhealthy.\n", failures)
fmt.Fprintf(os.Stderr, " Dropped %d/%d before stopping.\n", dropped, len(stale))
return dropped, SilentExit()
}
dropCtx, dropCancel := context.WithTimeout(ctx, cleanDatabasesDropTimeout)
//nolint:gosec // G201: identifier quoted+escaped via doltutil.QuoteIdentifierUnvalidated
_, err := conn.ExecContext(dropCtx, fmt.Sprintf("DROP DATABASE %s", doltutil.QuoteIdentifierUnvalidated(name)))
dropCancel()
if err != nil {
fmt.Fprintf(os.Stderr, " FAIL: %s: %v\n", name, err)
failures++
if isTimeoutError(err) {
consecutiveTimeouts++
}
} else {
fmt.Printf(" Dropped: %s\n", name)
dropped++
failures = 0
consecutiveTimeouts = 0
}
if (i+1)%cleanDatabasesBatchSize == 0 && i+1 < len(stale) {
fmt.Printf(" [%d/%d] pausing %s...\n", i+1, len(stale), cleanDatabasesBatchPause)
time.Sleep(cleanDatabasesBatchPause)
}
}
fmt.Printf("\nDropped %d/%d stale databases.\n", dropped, len(stale))
return dropped, nil
}