* fix(lint): silence G122 on dolt cache cleanup remove
golangci-lint v2.11.4 (released recently) enabled a new gosec check:
G122: Filesystem operation in filepath.Walk/WalkDir callback uses
race-prone path; consider root-scoped APIs (e.g. os.Root) to prevent
symlink TOCTOU traversal (gosec)
This flags the `os.Remove(path)` inside cleanGitRemoteCacheGarbage.
Main went red on the Lint job with no code change to blame — it broke
on every PR that rebased past the golangci-lint bump.
The warning is not applicable here: the walk root is
`<beadsDir>/<database>/.dolt/git-remote-cache/`, owned by the user
running bd. An attacker who can symlink-swap a tmp_pack_*/tmp_idx_*
file mid-walk already has write access to the Dolt data directory,
which is a much bigger compromise than cache_cleanup.go doing anything
unsafe.
Add a targeted `#nosec G122` with justification rather than refactor
to `os.Root`-scoped APIs, which would be a larger change against a
file that exists to clean up a 102 GB leak (GH#3354) and shouldn't
take on additional complexity for a hypothetical threat. Matches the
project's existing pattern of per-site gosec exceptions with rationale.
Verification:
golangci-lint run --timeout=5m --build-tags=gms_pure_go ./...
# 0 issues.
* fix(dolt): update shared-server credential tests for new per-remote signatures
Two tests added by #3223 (f9ec9859, 2026-04-13) were not updated when
#3211 (73d72ec0, 2026-04-20) widened the credential-routing guards to
take per-remote parameters. The resulting signature drift landed on
main and broke the build on ubuntu-latest and macos-latest:
internal/storage/dolt/credentials_test.go:445:39: not enough
arguments in call to store.shouldUseCLIForCredentials
internal/storage/dolt/credentials_test.go:486:6: not enough
arguments in call to store.shouldUseCLIForCloudAuth
Fix:
- TestCredentialCLIRoutingSharedServerUsesSharedDoltRoot:
pass store.remote and store.mainRemoteCredentials() to match the
new shouldUseCLIForCredentials(ctx, remote, creds) signature,
matching the pattern already used by the sibling
TestCredentialCLIRoutingExternalServer/NoRemote tests.
- TestCloudAuthCLIRoutingSharedServerUsesSharedDoltRoot:
pass store.remote to the new shouldUseCLIForCloudAuth(remote)
signature, and change the dolt remote URL from https:// to az://.
The old global "any cloud env var triggers CLI routing" heuristic
was replaced by per-scheme matching (AZURE_STORAGE_* only triggers
for az:// remotes, etc. — see the positive/negative cases in
TestCloudAuthCLIRouting). The test sets AZURE_STORAGE_ACCOUNT, so
the remote must be az:// for the assertion to hold.
Verification: both tests pass locally (`go test -tags gms_pure_go
-run 'SharedServerUsesSharedDoltRoot' ./internal/storage/dolt/`).
94 lines
2.8 KiB
Go
94 lines
2.8 KiB
Go
//go:build cgo
|
|
|
|
package embeddeddolt
|
|
|
|
import (
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"sync"
|
|
"time"
|
|
)
|
|
|
|
// cleanGitRemoteCacheGarbage removes orphaned tmp_pack_* files from the
|
|
// Dolt git-remote-cache. These files are created by `git fetch` (invoked
|
|
// by Dolt's GitBlobstore) and should be renamed to final .pack/.idx files
|
|
// on success or deleted on failure. In practice, failed or interrupted
|
|
// fetches leave them behind indefinitely, and Dolt's built-in periodic
|
|
// git gc (maybeRunGC, gated to once per 24h) either never runs or cannot
|
|
// keep up with the accumulation rate.
|
|
//
|
|
// On a real machine with normal beads usage, this leak consumed 102 GB
|
|
// (412 files) in 7 days. See https://github.com/gastownhall/beads/issues/3354
|
|
//
|
|
// This function is safe to call concurrently and is rate-limited to avoid
|
|
// unnecessary filesystem walks on hot paths.
|
|
func (s *EmbeddedDoltStore) cleanGitRemoteCacheGarbage() {
|
|
if !cacheCleanupThrottle.shouldRun() {
|
|
return
|
|
}
|
|
|
|
cacheBase := filepath.Join(s.dataDir, s.database, ".dolt", "git-remote-cache")
|
|
if _, err := os.Stat(cacheBase); os.IsNotExist(err) {
|
|
return
|
|
}
|
|
|
|
cutoff := time.Now().Add(-tmpPackMinAge)
|
|
|
|
_ = filepath.WalkDir(cacheBase, func(path string, d os.DirEntry, err error) error {
|
|
if err != nil {
|
|
return nil // best-effort: skip unreadable entries
|
|
}
|
|
if d.IsDir() {
|
|
return nil
|
|
}
|
|
name := d.Name()
|
|
if !strings.HasPrefix(name, "tmp_pack_") && !strings.HasPrefix(name, "tmp_idx_") {
|
|
return nil
|
|
}
|
|
info, err := d.Info()
|
|
if err != nil {
|
|
return nil
|
|
}
|
|
if info.ModTime().Before(cutoff) {
|
|
// #nosec G122 -- path is under .dolt/git-remote-cache/ which is
|
|
// owned by the user running bd. A TOCTOU symlink swap would
|
|
// require write access to that directory; in that case the
|
|
// attacker already controls the Dolt data. The tmp_pack_/tmp_idx_
|
|
// prefix check further narrows the scope to files Dolt itself writes.
|
|
_ = os.Remove(path)
|
|
}
|
|
return nil
|
|
})
|
|
}
|
|
|
|
const (
|
|
// tmpPackMinAge is the minimum age before a tmp_pack file is considered
|
|
// garbage. Files younger than this may belong to an in-progress fetch.
|
|
tmpPackMinAge = 5 * time.Minute
|
|
|
|
// cacheCleanupInterval is how often cleanGitRemoteCacheGarbage actually
|
|
// walks the filesystem when called repeatedly.
|
|
cacheCleanupInterval = 10 * time.Minute
|
|
)
|
|
|
|
// throttle gates a function to run at most once per interval.
|
|
type throttle struct {
|
|
mu sync.Mutex
|
|
interval time.Duration
|
|
lastRun time.Time
|
|
}
|
|
|
|
func (t *throttle) shouldRun() bool {
|
|
t.mu.Lock()
|
|
defer t.mu.Unlock()
|
|
if time.Since(t.lastRun) < t.interval {
|
|
return false
|
|
}
|
|
t.lastRun = time.Now()
|
|
return true
|
|
}
|
|
|
|
// cacheCleanupThrottle is a package-level throttle shared across all
|
|
// EmbeddedDoltStore instances in the same process.
|
|
var cacheCleanupThrottle = &throttle{interval: cacheCleanupInterval}
|