kv: add 'bd kv list --prefix' — SQL-side key-prefix filter for scoped reads (wy-1rrwt8) (#6380)
* wy-1rrwt8: add 'bd kv list --prefix' — SQL-side key-prefix filter for scoped kv reads Downstream mail transports read the whole kv store (3.4MB, ~1988 mail.* envelopes on the wyvern fleet) for every inbox render. This adds a --prefix flag to 'bd kv list' that pushes the filter into SQL on stores that support it, via an optional domain.ConfigPrefixReader interface discovered by type assertion — no existing storage interface or test double changes shape, and every path falls back to GetAllConfig plus in-process filtering. - issueops: ConfigPrefixLikeQuery + ConfigPrefixPattern (LIKE with explicit ESCAPE '!' — the engine does not honor backslash as a default escape) and GetConfigByPrefixInTx. - domain: ConfigPrefixReader optional interface; use-case impl delegates to the repo fast path or filters GetAllConfig. - domain/db, dolt, embeddeddolt: SQL-side GetConfigByPrefix. - telemetry: instrumented passthrough preserving the capability. - cmd/bd: --prefix on kv list, both classic and proxied-server paths; kvPairsWithPrefix re-checks on every path so output is identical whichever path served the read. * build(bazel): list the PR's new test files in go_test srcs (make bazel-sync) kv_prefix_test.go (cmd/bd) and config_prefix_test.go (internal/storage/issueops) are new, untagged test files. Both packages' BUILD.bazel files were generated on main after the PR's commit was written, and the rebase onto main did not re-run gazelle, so neither go_test listed them: bd_test and issueops_test reported PASSED with "no tests to run" for TestKVPairsWithPrefix and TestConfigPrefixPattern, and bazel.yml's "BUILD files in sync" step fails on that drift in the required bazel-test lane. Generated by `make bazel-sync`, which rewrote only these two rows. Both files import only the standard library, so no deps change. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(kv): engage the --prefix SQL fast path through the decorator chain Maintainer review fixup for PR #6380. The SQL push-down this PR exists to deliver was unreachable on bd's default direct/embedded path. `bd kv list --prefix` asserted `store.(configPrefixReader)` on the outermost decorator, but wireStorageDecorators unconditionally interposes externaldeps.Store and (hooks on, the default) HookFiringStore, both of which embed the storage.DoltStorage *interface* — and GetConfigByPrefix is deliberately not declared there. The assertion could never succeed, so every direct-mode prefix read silently took the GetAllConfig full-table fallback. Output was identical, so nothing errored, logged, or failed a test, while the flag help and command Long text asserted the opposite. This carries the prior review lineage's fixup 9ca3caeb9, stranded when the PR was relaunched, plus what this review iteration found missing from it: - cmd/bd: discover the capability through storage.UnwrapStore, the repo's documented idiom for optional-interface assertions (~50 existing call sites), in configPrefixReaderFor. The direct-mode read itself is extracted as readKVListConfig so the regression test drives the command's own read: with only the helper under test, reverting the call site to the bare `store.(configPrefixReader)` assertion still left the whole suite green. - cmd/bd: chain-level regression test over wireStorageDecorators covering all four telemetry/hooks configurations, plus the negative case proving a store without the capability is still reported incapable. The stub counts full reads, so a fallback fails the test even though it returns the same rows. Verified non-vacuous: a bare assertion in the helper or in readKVListConfig, a disabled fast path, a dropped kv. prefix, or a method added only to the mirror interface each fail all four configurations; a peel that stops at InstrumentedStorage fails the telemetry-on negative case. - dolt, embeddeddolt, telemetry: compile-time pins against domain.ConfigPrefixReader, so method-set drift on any implementation is a build error instead of a silent reversion to full scans. The cmd/bd mirror's own pin checks one direction only (a method added just to the mirror still compiles); its comment now says so and names the test that covers the other direction. - domain/db: delegate GetConfigByPrefix to issueops.GetConfigByPrefixInTx like the sibling stores, instead of a second scan loop over the shared query constant. - telemetry: record the deliberate trade — peel-to-raw keeps the standard idiom and leaves the instrumented forwarder off the cmd/bd path. - bazel: add //internal/storage/domain to the telemetry_test deps (make bazel-sync); without it the telemetry pin's import fails strict deps. Co-authored-by: dog <dog@ghosttrack.com> Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: dog <dog@ghosttrack.com> Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
2f1e8be3ca
commit
c4305ff01e
16 changed files with 492 additions and 7 deletions
|
|
@ -854,6 +854,7 @@ go_test(
|
|||
"jira_test.go",
|
||||
"json_flag_shadow_test.go",
|
||||
"kv_embedded_test.go",
|
||||
"kv_prefix_test.go",
|
||||
"kv_proxied_integration_test.go",
|
||||
"kv_test.go",
|
||||
"label_embedded_test.go",
|
||||
|
|
|
|||
85
cmd/bd/kv.go
85
cmd/bd/kv.go
|
|
@ -1,6 +1,7 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
|
|
@ -9,6 +10,8 @@ import (
|
|||
"github.com/spf13/cobra"
|
||||
|
||||
"github.com/steveyegge/beads/internal/metrics"
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
"github.com/steveyegge/beads/internal/storage/domain"
|
||||
"github.com/steveyegge/beads/internal/storage/kvkeys"
|
||||
)
|
||||
|
||||
|
|
@ -110,6 +113,70 @@ func kvPairsFromConfig(allConfig map[string]string) map[string]string {
|
|||
return kvPairs
|
||||
}
|
||||
|
||||
// configPrefixReader mirrors domain.ConfigPrefixReader structurally: the
|
||||
// optional store fast path that reads only the keys under a prefix. Both
|
||||
// list paths discover it by assertion and fall back to the full read plus
|
||||
// kvPairsWithPrefix when the store predates it.
|
||||
type configPrefixReader interface {
|
||||
GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error)
|
||||
}
|
||||
|
||||
// The mirror above is structural, so this assignment is what ties it to the
|
||||
// domain interface the stores are pinned to. The check is one-way: it stops
|
||||
// compiling once the mirror no longer covers domain's method set, but a method
|
||||
// added only to the mirror still compiles — and would silently fail discovery
|
||||
// against the stores. TestKVPrefixReaderUnwrapsStorageDecorators covers that
|
||||
// direction: its stub implements only DoltStorage plus the domain method.
|
||||
var _ domain.ConfigPrefixReader = configPrefixReader(nil)
|
||||
|
||||
// configPrefixReaderFor discovers the optional prefix-read capability on a
|
||||
// store, peeling the decorator chain first.
|
||||
//
|
||||
// The peel is load-bearing, not defensive: cmd/bd's store is always decorated
|
||||
// (wireStorageDecorators interposes externaldeps.Store and, by default,
|
||||
// HookFiringStore), and those wrappers embed the storage.DoltStorage
|
||||
// *interface* — which deliberately does not declare GetConfigByPrefix. A bare
|
||||
// assertion on the outermost layer therefore can never succeed, and because
|
||||
// the fallback returns identical rows it would silently full-scan forever.
|
||||
// storage.UnwrapStore is the repo's documented idiom for reaching the concrete
|
||||
// store behind optional-interface assertions; every wrapper implements
|
||||
// Unwrap(), so it lands on the raw store in every chain configuration.
|
||||
func configPrefixReaderFor(s storage.DoltStorage) (configPrefixReader, bool) {
|
||||
pr, ok := storage.UnwrapStore(s).(configPrefixReader)
|
||||
return pr, ok
|
||||
}
|
||||
|
||||
// readKVListConfig is the direct-mode store read behind `bd kv list`: only the
|
||||
// rows under kvPrefix+userPrefix when the store offers the prefix fast path,
|
||||
// the whole config table otherwise. kvPairsWithPrefix narrows the result
|
||||
// either way, so the two arms differ only in cost — which is why
|
||||
// TestKVPrefixReaderUnwrapsStorageDecorators drives this function rather than
|
||||
// configPrefixReaderFor alone.
|
||||
func readKVListConfig(ctx context.Context, s storage.DoltStorage, userPrefix string) (map[string]string, error) {
|
||||
if pr, ok := configPrefixReaderFor(s); ok && userPrefix != "" {
|
||||
return pr.GetConfigByPrefix(ctx, kvPrefix+userPrefix)
|
||||
}
|
||||
return s.GetAllConfig(ctx)
|
||||
}
|
||||
|
||||
// kvPairsWithPrefix is kvPairsFromConfig narrowed to user keys starting with
|
||||
// userPrefix. It runs on every listing: after a SQL-side prefix read it is a
|
||||
// no-op re-check, and on the GetAllConfig fallback it IS the filter — so the
|
||||
// output is identical whichever path served the read.
|
||||
func kvPairsWithPrefix(allConfig map[string]string, userPrefix string) map[string]string {
|
||||
kvPairs := kvPairsFromConfig(allConfig)
|
||||
if userPrefix == "" {
|
||||
return kvPairs
|
||||
}
|
||||
out := make(map[string]string)
|
||||
for k, v := range kvPairs {
|
||||
if strings.HasPrefix(k, userPrefix) {
|
||||
out[k] = v
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// printKVListResult renders the `bd kv list` output. Shared by the classic
|
||||
// and proxied-server paths (a mail client parses the --json shape; keep it
|
||||
// byte-identical across modes).
|
||||
|
|
@ -292,15 +359,23 @@ Examples:
|
|||
},
|
||||
}
|
||||
|
||||
// kvListPrefix is the --prefix flag: list only keys starting with it.
|
||||
var kvListPrefix string
|
||||
|
||||
// kvListCmd lists all key-value pairs
|
||||
var kvListCmd = &cobra.Command{
|
||||
Use: "list",
|
||||
Short: "List all key-value pairs",
|
||||
Long: `List all key-value pairs in the beads key-value store.
|
||||
|
||||
With --prefix, list only the keys starting with that prefix. The filter is
|
||||
pushed into SQL on stores that support it, so a scoped read of a large kv
|
||||
store does not serialize the whole table.
|
||||
|
||||
Examples:
|
||||
bd kv list
|
||||
bd kv list --json`,
|
||||
bd kv list --json
|
||||
bd kv list --prefix mail.dog. --json`,
|
||||
SilenceUsage: true,
|
||||
SilenceErrors: true,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
|
|
@ -312,7 +387,7 @@ Examples:
|
|||
}()
|
||||
|
||||
if usesProxiedServer() {
|
||||
return runKVListProxiedServer(rootCtx)
|
||||
return runKVListProxiedServer(rootCtx, kvListPrefix)
|
||||
}
|
||||
|
||||
if err := ensureDirectMode("kv list requires direct database access"); err != nil {
|
||||
|
|
@ -320,16 +395,18 @@ Examples:
|
|||
}
|
||||
|
||||
ctx := rootCtx
|
||||
allConfig, err := store.GetAllConfig(ctx)
|
||||
allConfig, err := readKVListConfig(ctx, store, kvListPrefix)
|
||||
if err != nil {
|
||||
return HandleErrorRespectJSON("listing keys: %v", err)
|
||||
}
|
||||
|
||||
return printKVListResult(kvPairsFromConfig(allConfig))
|
||||
return printKVListResult(kvPairsWithPrefix(allConfig, kvListPrefix))
|
||||
},
|
||||
}
|
||||
|
||||
func init() {
|
||||
kvListCmd.Flags().StringVar(&kvListPrefix, "prefix", "", "list only keys starting with this prefix (filtered in SQL where supported)")
|
||||
|
||||
// Register all kv subcommands under kvCmd
|
||||
kvCmd.AddCommand(kvSetCmd)
|
||||
kvCmd.AddCommand(kvGetCmd)
|
||||
|
|
|
|||
57
cmd/bd/kv_prefix_test.go
Normal file
57
cmd/bd/kv_prefix_test.go
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// kvPairsWithPrefix is the shared post-filter on both `bd kv list` paths: a
|
||||
// no-op re-check after a SQL-side prefix read, and the whole filter on the
|
||||
// GetAllConfig fallback — so its semantics define the flag's output either way.
|
||||
func TestKVPairsWithPrefix(t *testing.T) {
|
||||
allConfig := map[string]string{
|
||||
"kv.mail.dog.m1": "a",
|
||||
"kv.mail.dog.m2": "b",
|
||||
"kv.mail.doggerel.m3": "c",
|
||||
"kv.mail.hare.m1": "d",
|
||||
"jira.url": "not-kv",
|
||||
}
|
||||
|
||||
t.Run("empty prefix lists every kv pair", func(t *testing.T) {
|
||||
got := kvPairsWithPrefix(allConfig, "")
|
||||
want := map[string]string{
|
||||
"mail.dog.m1": "a",
|
||||
"mail.dog.m2": "b",
|
||||
"mail.doggerel.m3": "c",
|
||||
"mail.hare.m1": "d",
|
||||
}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("prefix narrows to one box and strips kv.", func(t *testing.T) {
|
||||
got := kvPairsWithPrefix(allConfig, "mail.dog.")
|
||||
want := map[string]string{
|
||||
"mail.dog.m1": "a",
|
||||
"mail.dog.m2": "b",
|
||||
}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("prefix without trailing dot matches sibling boxes literally", func(t *testing.T) {
|
||||
got := kvPairsWithPrefix(allConfig, "mail.dog")
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("got %v, want mail.dog.* plus mail.doggerel.*", got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("no matches returns empty map", func(t *testing.T) {
|
||||
got := kvPairsWithPrefix(allConfig, "mail.owl.")
|
||||
if len(got) != 0 {
|
||||
t.Fatalf("got %v, want empty", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
@ -75,17 +75,21 @@ func runKVClearProxiedServer(ctx context.Context, key string) error {
|
|||
return printKVClearResult(key)
|
||||
}
|
||||
|
||||
func runKVListProxiedServer(ctx context.Context) error {
|
||||
func runKVListProxiedServer(ctx context.Context, prefix string) error {
|
||||
if uowProvider == nil {
|
||||
return HandleErrorRespectJSON("proxied-server UOW provider not initialized")
|
||||
}
|
||||
|
||||
allConfig, err := uow.RunTxRead(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) (map[string]string, error) {
|
||||
return uw.ConfigUseCase().GetAllConfig(ctx)
|
||||
cfg := uw.ConfigUseCase()
|
||||
if pr, ok := cfg.(configPrefixReader); ok && prefix != "" {
|
||||
return pr.GetConfigByPrefix(ctx, kvPrefix+prefix)
|
||||
}
|
||||
return cfg.GetAllConfig(ctx)
|
||||
})
|
||||
if err != nil {
|
||||
return HandleErrorRespectJSON("listing keys: %v", err)
|
||||
}
|
||||
|
||||
return printKVListResult(kvPairsFromConfig(allConfig))
|
||||
return printKVListResult(kvPairsWithPrefix(allConfig, prefix))
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ package main
|
|||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/hooks"
|
||||
|
|
@ -23,6 +24,28 @@ func (s *stubChainStore) ActiveDatabaseSize(context.Context) (int64, error) {
|
|||
return s.databaseSize, nil
|
||||
}
|
||||
|
||||
// stubPrefixChainStore is a concrete store that DOES implement the optional
|
||||
// prefix-read capability, standing in for DoltStore/EmbeddedDoltStore in the
|
||||
// discovery test below. It records the prefixes it was asked for, and counts
|
||||
// full reads, so the test can prove the call reached the raw store's fast path
|
||||
// rather than a fallback. The full read returns the same rows the prefix read
|
||||
// would, as the real stores do, so only those counters can tell the arms apart.
|
||||
type stubPrefixChainStore struct {
|
||||
stubChainStore
|
||||
prefixReads []string
|
||||
fullReads int
|
||||
}
|
||||
|
||||
func (s *stubPrefixChainStore) GetConfigByPrefix(_ context.Context, prefix string) (map[string]string, error) {
|
||||
s.prefixReads = append(s.prefixReads, prefix)
|
||||
return map[string]string{prefix + "one": "1"}, nil
|
||||
}
|
||||
|
||||
func (s *stubPrefixChainStore) GetAllConfig(context.Context) (map[string]string, error) {
|
||||
s.fullReads++
|
||||
return map[string]string{"kv.mail.one": "1"}, nil
|
||||
}
|
||||
|
||||
// clearTelemetryEnv is defined once for the package, in
|
||||
// command_telemetry_test.go; it unsets every BD_OTEL_* / OTEL_* variable
|
||||
// telemetry.Enabled or the SDK looks at, so each test starts from a known
|
||||
|
|
@ -109,6 +132,81 @@ func TestGCStoreSizeUnwrapsStorageDecorators(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
// TestKVPrefixReaderUnwrapsStorageDecorators pins the read behind
|
||||
// `bd kv list --prefix`. That flag reads only the rows under a prefix when the
|
||||
// store offers the optional GetConfigByPrefix capability, and falls back to a
|
||||
// full GetAllConfig scan when it does not — a fallback that returns identical
|
||||
// rows, so a discovery miss produces no error, no log, and no failing test
|
||||
// while costing exactly the full-table serialization the flag exists to avoid.
|
||||
// Every decorator wireStorageDecorators installs embeds the
|
||||
// storage.DoltStorage interface, which does not declare GetConfigByPrefix, so
|
||||
// discovery only works if it peels. Covers every chain configuration, and
|
||||
// drives readKVListConfig, the read the command performs, so a bare assertion
|
||||
// on that path fails here rather than only one inside the discovery helper.
|
||||
func TestKVPrefixReaderUnwrapsStorageDecorators(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
telemetryOn bool
|
||||
hooksDisabled bool
|
||||
}{
|
||||
{name: "TelemetryOff_HookOn"},
|
||||
{name: "TelemetryOn_HookOn", telemetryOn: true},
|
||||
{name: "TelemetryOff_HookDisabled", hooksDisabled: true},
|
||||
{name: "TelemetryOn_HookDisabled", telemetryOn: true, hooksDisabled: true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
clearTelemetryEnv(t)
|
||||
if tc.telemetryOn {
|
||||
t.Setenv("BD_OTEL_STDOUT", "true")
|
||||
}
|
||||
raw := &stubPrefixChainStore{}
|
||||
wrapped := wireStorageDecorators(raw, hooks.NewRunner("/nonexistent"), tc.hooksDisabled)
|
||||
|
||||
got, err := readKVListConfig(t.Context(), wrapped, "mail.")
|
||||
if err != nil {
|
||||
t.Fatalf("readKVListConfig: %v", err)
|
||||
}
|
||||
if raw.fullReads != 0 {
|
||||
t.Fatalf("prefix read fell back to %d full GetAllConfig scan(s) through the wired chain (%T); kv list --prefix would silently full-scan", raw.fullReads, wrapped)
|
||||
}
|
||||
if want := []string{"kv.mail."}; !reflect.DeepEqual(raw.prefixReads, want) {
|
||||
t.Errorf("raw store prefix reads = %v; want %v — the read never reached the concrete store", raw.prefixReads, want)
|
||||
}
|
||||
if want := (map[string]string{"kv.mail.one": "1"}); !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("readKVListConfig = %v; want %v", got, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestKVPrefixReaderAbsentCapabilityFallsBack keeps the test above honest:
|
||||
// discovery must report a store that lacks the capability as incapable, so the
|
||||
// caller takes the GetAllConfig fallback. The telemetry-on case also pins the
|
||||
// deliberate choice of peel-to-raw over peel-until-implements —
|
||||
// InstrumentedStorage does implement GetConfigByPrefix, and we peel past it,
|
||||
// trading the storage span on prefix reads for the repo's standard UnwrapStore
|
||||
// idiom. Flipping that trade should have to flip this assertion.
|
||||
func TestKVPrefixReaderAbsentCapabilityFallsBack(t *testing.T) {
|
||||
for _, telemetryOn := range []bool{false, true} {
|
||||
name := "TelemetryOff"
|
||||
if telemetryOn {
|
||||
name = "TelemetryOn"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
clearTelemetryEnv(t)
|
||||
if telemetryOn {
|
||||
t.Setenv("BD_OTEL_STDOUT", "true")
|
||||
}
|
||||
raw := &stubChainStore{}
|
||||
wrapped := wireStorageDecorators(raw, hooks.NewRunner("/nonexistent"), false)
|
||||
|
||||
if _, ok := configPrefixReaderFor(wrapped); ok {
|
||||
t.Errorf("store without GetConfigByPrefix reported as prefix-capable through %T", wrapped)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWireStorageDecorators_TelemetryOn_HookDisabled(t *testing.T) {
|
||||
clearTelemetryEnv(t)
|
||||
t.Setenv("BD_OTEL_STDOUT", "true")
|
||||
|
|
|
|||
|
|
@ -76,6 +76,23 @@ func (s *DoltStore) GetAllConfig(ctx context.Context) (map[string]string, error)
|
|||
return result, err
|
||||
}
|
||||
|
||||
// Callers discover the capability below by type assertion, which degrades
|
||||
// silently to a full scan when the method set drifts. This pin turns that
|
||||
// drift into a compile error instead.
|
||||
var _ domain.ConfigPrefixReader = (*DoltStore)(nil)
|
||||
|
||||
// GetConfigByPrefix retrieves the configuration values whose key starts with
|
||||
// prefix, filtered in SQL (the domain.ConfigPrefixReader optional fast path).
|
||||
func (s *DoltStore) GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error) {
|
||||
var result map[string]string
|
||||
err := s.withReadTx(ctx, func(tx *sql.Tx) error {
|
||||
var err error
|
||||
result, err = issueops.GetConfigByPrefixInTx(ctx, tx, prefix)
|
||||
return err
|
||||
})
|
||||
return result, err
|
||||
}
|
||||
|
||||
// DeleteConfig removes a configuration value
|
||||
func (s *DoltStore) DeleteConfig(ctx context.Context, key string) error {
|
||||
return s.withRetryTx(ctx, func(tx *sql.Tx) error {
|
||||
|
|
|
|||
|
|
@ -3,11 +3,22 @@ package domain
|
|||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/steveyegge/beads/internal/config"
|
||||
"github.com/steveyegge/beads/internal/types"
|
||||
)
|
||||
|
||||
// ConfigPrefixReader is the optional fast path for prefix-scoped config
|
||||
// reads (`bd kv list --prefix`): only the rows whose key starts with prefix
|
||||
// leave the database. It is deliberately NOT part of ConfigSQLRepository or
|
||||
// ConfigUseCase — existing implementations and test doubles keep compiling —
|
||||
// and callers discover it by type assertion, falling back to GetAllConfig
|
||||
// plus in-process filtering when the assertion misses.
|
||||
type ConfigPrefixReader interface {
|
||||
GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error)
|
||||
}
|
||||
|
||||
type ConfigSQLRepository interface {
|
||||
GetMetadata(ctx context.Context, key string) (string, error)
|
||||
SetMetadata(ctx context.Context, key, value string) error
|
||||
|
|
@ -229,6 +240,30 @@ func (u *configUseCaseImpl) GetAllConfig(ctx context.Context) (map[string]string
|
|||
return out, nil
|
||||
}
|
||||
|
||||
// GetConfigByPrefix implements ConfigPrefixReader on the use case, pushing
|
||||
// the prefix into SQL when the repository supports it and filtering the
|
||||
// GetAllConfig result in-process when it does not (same rows either way).
|
||||
func (u *configUseCaseImpl) GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error) {
|
||||
if pr, ok := u.cfgRepo.(ConfigPrefixReader); ok {
|
||||
out, err := pr.GetConfigByPrefix(ctx, prefix)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("GetConfigByPrefix: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
all, err := u.cfgRepo.GetAllConfig(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("GetConfigByPrefix: %w", err)
|
||||
}
|
||||
out := make(map[string]string)
|
||||
for k, v := range all {
|
||||
if strings.HasPrefix(k, prefix) {
|
||||
out[k] = v
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (u *configUseCaseImpl) SetMetadata(ctx context.Context, key, value string) error {
|
||||
if err := u.cfgRepo.SetMetadata(ctx, key, value); err != nil {
|
||||
return fmt.Errorf("SetMetadata: %w", err)
|
||||
|
|
|
|||
|
|
@ -126,6 +126,22 @@ func (r *configSQLRepositoryImpl) GetAllConfig(ctx context.Context) (map[string]
|
|||
return out, nil
|
||||
}
|
||||
|
||||
// GetConfigByPrefix retrieves only the config rows whose key starts with
|
||||
// prefix, pushing the filter into SQL (domain.ConfigPrefixReader — the
|
||||
// optional fast path `bd kv list --prefix` discovers by assertion).
|
||||
//
|
||||
// Like the dolt and embeddeddolt stores it delegates to issueops rather than
|
||||
// re-running the query itself (Runner satisfies issueops.DBTX): query and row
|
||||
// scan belong together, and they were already sharing ConfigPrefixLikeQuery
|
||||
// across two scan loops — the drift the shared helper exists to prevent.
|
||||
func (r *configSQLRepositoryImpl) GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error) {
|
||||
out, err := issueops.GetConfigByPrefixInTx(ctx, r.runner, prefix)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("db: GetConfigByPrefix %s: %w", prefix, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// GetCustomTypes resolves the workspace's custom issue types through
|
||||
// issueops.ComposeCustomTypes, the same rule the embedded and server-mode
|
||||
// stores use, so proxied `bd types` and proxied create/update validation
|
||||
|
|
|
|||
|
|
@ -41,6 +41,10 @@ func (s *testSuite) TestConfigSQLRepository() {
|
|||
s.Run("EmptyReturnsEmptyMap", s.configGetAllConfigEmpty)
|
||||
s.Run("ReturnsAllRows", s.configGetAllConfigAllRows)
|
||||
})
|
||||
s.Run("GetConfigByPrefix", func() {
|
||||
s.Run("ReturnsOnlyPrefixedRows", s.configGetConfigByPrefixOnlyPrefixed)
|
||||
s.Run("EscapesLikeMetacharacters", s.configGetConfigByPrefixEscapesLike)
|
||||
})
|
||||
s.Run("LocalMetadata", func() {
|
||||
s.Run("SetThenGetRoundTrips", s.configLocalMetadataRoundTrip)
|
||||
s.Run("SetOverwrites", s.configLocalMetadataOverwrite)
|
||||
|
|
@ -54,6 +58,7 @@ func (s *testSuite) TestConfigSQLRepository() {
|
|||
s.Run("DeleteConfigMissingKeyIsNoop", s.configUseCaseDeleteConfigMissing)
|
||||
s.Run("GetAllConfigEmpty", s.configUseCaseGetAllConfigEmpty)
|
||||
s.Run("GetAllConfigReturnsAllRows", s.configUseCaseGetAllConfigAllRows)
|
||||
s.Run("GetConfigByPrefixReturnsOnlyPrefixedRows", s.configUseCaseGetConfigByPrefixOnlyPrefixed)
|
||||
})
|
||||
s.Run("GetCustomTypes", func() {
|
||||
s.Run("MissingKeyReturnsNil", s.configGetCustomTypesMissing)
|
||||
|
|
@ -589,6 +594,47 @@ func (s *testSuite) configUseCaseDeleteConfigMissing() {
|
|||
s.Require().NoError(s.configUC().DeleteConfig(s.Ctx(), "no_such_key"))
|
||||
}
|
||||
|
||||
func (s *testSuite) configGetConfigByPrefixOnlyPrefixed() {
|
||||
_, err := s.Runner().ExecContext(s.Ctx(), "DELETE FROM config")
|
||||
s.Require().NoError(err)
|
||||
r := s.configRepo()
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.mail.dog.m1", "a"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.mail.dog.m2", "b"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.mail.doggerel.m3", "c"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.mail.hare.m1", "d"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "jira.url", "https://example.atlassian.net"))
|
||||
|
||||
pr, ok := interface{}(r).(domain.ConfigPrefixReader)
|
||||
s.Require().True(ok, "configSQLRepositoryImpl must implement domain.ConfigPrefixReader")
|
||||
got, err := pr.GetConfigByPrefix(s.Ctx(), "kv.mail.dog.")
|
||||
s.Require().NoError(err)
|
||||
s.Equal(map[string]string{
|
||||
"kv.mail.dog.m1": "a",
|
||||
"kv.mail.dog.m2": "b",
|
||||
}, got)
|
||||
}
|
||||
|
||||
func (s *testSuite) configGetConfigByPrefixEscapesLike() {
|
||||
_, err := s.Runner().ExecContext(s.Ctx(), "DELETE FROM config")
|
||||
s.Require().NoError(err)
|
||||
r := s.configRepo()
|
||||
// `_` and `%` are LIKE metacharacters: an unescaped prefix "kv.a_." would
|
||||
// also match "kv.axb"-shaped keys. The literal-prefix contract must hold.
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.a_.one", "underscore"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.ax.one", "letter"))
|
||||
s.Require().NoError(r.SetConfig(s.Ctx(), "kv.a%.one", "percent"))
|
||||
|
||||
pr, ok := interface{}(r).(domain.ConfigPrefixReader)
|
||||
s.Require().True(ok)
|
||||
got, err := pr.GetConfigByPrefix(s.Ctx(), "kv.a_.")
|
||||
s.Require().NoError(err)
|
||||
s.Equal(map[string]string{"kv.a_.one": "underscore"}, got)
|
||||
|
||||
got, err = pr.GetConfigByPrefix(s.Ctx(), "kv.a%.")
|
||||
s.Require().NoError(err)
|
||||
s.Equal(map[string]string{"kv.a%.one": "percent"}, got)
|
||||
}
|
||||
|
||||
func (s *testSuite) configUseCaseGetAllConfigEmpty() {
|
||||
_, err := s.Runner().ExecContext(s.Ctx(), "DELETE FROM config")
|
||||
s.Require().NoError(err)
|
||||
|
|
@ -612,6 +658,20 @@ func (s *testSuite) configUseCaseGetAllConfigAllRows() {
|
|||
}, got)
|
||||
}
|
||||
|
||||
func (s *testSuite) configUseCaseGetConfigByPrefixOnlyPrefixed() {
|
||||
_, err := s.Runner().ExecContext(s.Ctx(), "DELETE FROM config")
|
||||
s.Require().NoError(err)
|
||||
uc := s.configUC()
|
||||
s.Require().NoError(uc.SetConfig(s.Ctx(), "kv.mail.dog.m1", "a"))
|
||||
s.Require().NoError(uc.SetConfig(s.Ctx(), "kv.mail.hare.m1", "b"))
|
||||
|
||||
pr, ok := interface{}(uc).(domain.ConfigPrefixReader)
|
||||
s.Require().True(ok, "configUseCaseImpl must implement domain.ConfigPrefixReader")
|
||||
got, err := pr.GetConfigByPrefix(s.Ctx(), "kv.mail.dog.")
|
||||
s.Require().NoError(err)
|
||||
s.Equal(map[string]string{"kv.mail.dog.m1": "a"}, got)
|
||||
}
|
||||
|
||||
func (s *testSuite) configGetCustomTypesTablePrecedence() {
|
||||
_, err := s.Runner().ExecContext(s.Ctx(), "DELETE FROM custom_types")
|
||||
s.Require().NoError(err)
|
||||
|
|
|
|||
|
|
@ -43,6 +43,23 @@ func (s *EmbeddedDoltStore) GetAllConfig(ctx context.Context) (map[string]string
|
|||
return result, err
|
||||
}
|
||||
|
||||
// Callers discover the capability below by type assertion, which degrades
|
||||
// silently to a full scan when the method set drifts. This pin turns that
|
||||
// drift into a compile error instead.
|
||||
var _ domain.ConfigPrefixReader = (*EmbeddedDoltStore)(nil)
|
||||
|
||||
// GetConfigByPrefix retrieves the configuration values whose key starts with
|
||||
// prefix, filtered in SQL (the domain.ConfigPrefixReader optional fast path).
|
||||
func (s *EmbeddedDoltStore) GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error) {
|
||||
var result map[string]string
|
||||
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
|
||||
var err error
|
||||
result, err = issueops.GetConfigByPrefixInTx(ctx, tx, prefix)
|
||||
return err
|
||||
})
|
||||
return result, err
|
||||
}
|
||||
|
||||
func (s *EmbeddedDoltStore) GetMetadata(ctx context.Context, key string) (string, error) {
|
||||
var value string
|
||||
err := s.withConn(ctx, false, func(tx *sql.Tx) error {
|
||||
|
|
|
|||
|
|
@ -132,6 +132,7 @@ go_test(
|
|||
"blocked_state_template_test.go",
|
||||
"close_test.go",
|
||||
"config_helpers_test.go",
|
||||
"config_prefix_test.go",
|
||||
"counts_missing_table_test.go",
|
||||
"create_only_errors_test.go",
|
||||
"create_only_guard_test.go",
|
||||
|
|
|
|||
|
|
@ -53,6 +53,43 @@ func GetAllConfigInTx(ctx context.Context, tx DBTX) (map[string]string, error) {
|
|||
return result, rows.Err()
|
||||
}
|
||||
|
||||
// ConfigPrefixLikeQuery is the one SELECT for prefix-scoped config reads.
|
||||
// Its explicit ESCAPE character pairs with ConfigPrefixPattern's escaping
|
||||
// ('!' rather than backslash: the engine does not treat backslash as the
|
||||
// default LIKE escape, and '!' avoids literal-vs-placeholder ambiguity);
|
||||
// keeping query and pattern builder side by side is what stops them drifting.
|
||||
const ConfigPrefixLikeQuery = "SELECT `key`, value FROM config WHERE `key` LIKE ? ESCAPE '!'"
|
||||
|
||||
// ConfigPrefixPattern builds the LIKE pattern that matches config keys
|
||||
// starting with the given literal prefix, escaping LIKE metacharacters
|
||||
// (%, _) and the escape character itself so the prefix always matches
|
||||
// literally. Use only with ConfigPrefixLikeQuery.
|
||||
func ConfigPrefixPattern(prefix string) string {
|
||||
return strings.NewReplacer(`!`, `!!`, `%`, `!%`, `_`, `!_`).Replace(prefix) + "%"
|
||||
}
|
||||
|
||||
// GetConfigByPrefixInTx retrieves the configuration pairs whose key starts
|
||||
// with prefix, within an existing transaction. The filter runs in SQL so a
|
||||
// scoped read of a large config/kv store (e.g. one mailbox out of thousands
|
||||
// of kv mail envelopes) does not serialize the whole table.
|
||||
func GetConfigByPrefixInTx(ctx context.Context, tx DBTX, prefix string) (map[string]string, error) {
|
||||
rows, err := tx.QueryContext(ctx, ConfigPrefixLikeQuery, ConfigPrefixPattern(prefix))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get config by prefix %s: %w", prefix, err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
result := make(map[string]string)
|
||||
for rows.Next() {
|
||||
var k, v string
|
||||
if err := rows.Scan(&k, &v); err != nil {
|
||||
return nil, fmt.Errorf("get config by prefix %s: scan: %w", prefix, err)
|
||||
}
|
||||
result[k] = v
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
// SetMetadataInTx sets a metadata value within an existing transaction.
|
||||
func SetMetadataInTx(ctx context.Context, tx DBTX, key, value string) error {
|
||||
_, err := tx.ExecContext(ctx, "REPLACE INTO metadata (`key`, value) VALUES (?, ?)", key, value)
|
||||
|
|
|
|||
20
internal/storage/issueops/config_prefix_test.go
Normal file
20
internal/storage/issueops/config_prefix_test.go
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
package issueops
|
||||
|
||||
import "testing"
|
||||
|
||||
// ConfigPrefixPattern must escape every LIKE metacharacter so a prefix always
|
||||
// matches literally: an unescaped `_` in "kv.a_." would also match "kv.axb".
|
||||
func TestConfigPrefixPattern(t *testing.T) {
|
||||
cases := []struct{ in, want string }{
|
||||
{"kv.mail.dog.", "kv.mail.dog.%"},
|
||||
{"kv.a_.", "kv.a!_.%"},
|
||||
{"kv.a%.", "kv.a!%.%"},
|
||||
{"kv.a!.", "kv.a!!.%"},
|
||||
{"", "%"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ConfigPrefixPattern(c.in); got != c.want {
|
||||
t.Errorf("ConfigPrefixPattern(%q) = %q, want %q", c.in, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -72,6 +72,7 @@ go_test(
|
|||
embed = [":telemetry"],
|
||||
deps = [
|
||||
"//internal/storage",
|
||||
"//internal/storage/domain",
|
||||
"//internal/storage/externaldeps",
|
||||
"//issueops",
|
||||
"//memoryops",
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ package telemetry
|
|||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go.opentelemetry.io/otel/attribute"
|
||||
|
|
@ -596,6 +597,41 @@ func (s *InstrumentedStorage) GetAllConfig(ctx context.Context) (map[string]stri
|
|||
return v, err
|
||||
}
|
||||
|
||||
// GetConfigByPrefix forwards the domain.ConfigPrefixReader optional fast
|
||||
// path when the wrapped store provides it, and otherwise falls back to
|
||||
// instrumented GetAllConfig filtered in-process — so wrapping a store never
|
||||
// costs it the capability.
|
||||
//
|
||||
// cmd/bd discovers that capability with storage.UnwrapStore, the repo's
|
||||
// standard idiom, which peels all the way to the raw store: on that path the
|
||||
// inner method is called directly and this span does not fire. The forwarder
|
||||
// is kept deliberately rather than deleted — it preserves the capability for
|
||||
// any caller holding the instrumented store, and it is the layer a
|
||||
// peel-until-implements discovery would stop at if spans on prefix reads ever
|
||||
// justify that non-standard peel.
|
||||
func (s *InstrumentedStorage) GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error) {
|
||||
attrs := []attribute.KeyValue{attribute.String("bd.config.prefix", prefix)}
|
||||
if pr, ok := s.inner.(interface {
|
||||
GetConfigByPrefix(ctx context.Context, prefix string) (map[string]string, error)
|
||||
}); ok {
|
||||
ctx, span, t := s.op(ctx, "GetConfigByPrefix", attrs...)
|
||||
v, err := pr.GetConfigByPrefix(ctx, prefix)
|
||||
s.done(ctx, span, t, err, attrs...)
|
||||
return v, err
|
||||
}
|
||||
all, err := s.GetAllConfig(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make(map[string]string)
|
||||
for k, v := range all {
|
||||
if strings.HasPrefix(k, prefix) {
|
||||
out[k] = v
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *InstrumentedStorage) SetLocalMetadata(ctx context.Context, key, value string) error {
|
||||
attrs := []attribute.KeyValue{attribute.String("bd.local_metadata.key", key)}
|
||||
ctx, span, t := s.op(ctx, "SetLocalMetadata", attrs...)
|
||||
|
|
|
|||
|
|
@ -5,8 +5,16 @@ import (
|
|||
"testing"
|
||||
|
||||
"github.com/steveyegge/beads/internal/storage"
|
||||
"github.com/steveyegge/beads/internal/storage/domain"
|
||||
)
|
||||
|
||||
// InstrumentedStorage forwards the optional prefix-read capability, which
|
||||
// callers find by type assertion — so a drifting method set would degrade
|
||||
// them to full scans silently. Pinned from the test build rather than
|
||||
// storage.go to keep the telemetry package free of a production dependency on
|
||||
// storage/domain.
|
||||
var _ domain.ConfigPrefixReader = (*InstrumentedStorage)(nil)
|
||||
|
||||
// fakeDoltStore is a stub DoltStorage used to assert wrapper identity and
|
||||
// type chain. The embedded interface is nil; tests must not call any of its
|
||||
// methods (interface-promoted calls would panic).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue