* feat(storage): add CommitConfigOnly — scoped config-table commit (GH#4078)
Generic Commit() excludes the config table by design (GH#2455) and
CommitWithConfig stages everything via '-Am' — neither can commit an
intentional config write without sweeping concurrent operations' dirty
tables in server mode. CommitConfigOnly stages ONLY config (via the
existing doltAddAndCommit helper) and commits it.
- VersionControl interface method with GH#4078/GH#2455 rationale
- DoltStore: scoped DOLT_ADD('config') + DOLT_COMMIT
- EmbeddedDoltStore: aliases Commit — single-process, so '-Am' cannot
sweep concurrent work (same rationale as its CommitWithConfig alias)
- Tests against a real dolt-sql-server container: scoped commit lands,
concurrent dirty issues row survives uncommitted, and generic
Commit()'s config exclusion is pinned as a regression guard
(cherry picked from commit 61ba8f05c9)
Carry note: rebased onto b1bee123b. The two version_control.go conflicts
were additive (upstream added CommitMergeResolution/CommitAll in the same
spot); resolved by keeping both sides, no change to this commit's code.
* feat(storage): CommitConfigUserKVOnly — screen memory commits to kv.* rows (GH#4078)
Carry reconciliation for the #4383 review: CommitConfigOnly stages the whole
config table, so a memory write's commit in server mode could sweep a
concurrent writer's half-applied internal key (issue_prefix above all), the
row-level form of the GH#2455 hazard. The review asked for #4412's kv.*
screen at the remember/forget sites.
CommitConfigUserKVOnly applies that screen before the scoped commit: if any
dirty config row is outside kv.*, it commits nothing and names the keys,
pointing at `bd dolt commit` (CommitAll), the same remedy the pre-pull
refusal gives. User kv.* rows already dirty are committed with the memory,
as the pre-pull auto-commit would. The screen query moves out of
assertDirtyConfigUserKVOnly into dirtyInternalConfigKeys so both paths share
it; the pre-pull refusal text is unchanged. Embedded aliases Commit, as its
CommitConfigOnly does.
Also softens the CommitConfigOnly interface doc (review nit): embedded mode
commits the whole working set, not only config.
Tests (server container): user kv.* rows commit; a dirty internal key is
refused with HEAD and the working set untouched.
* fix(storage): stage config's lookup-table projections with CommitConfigOnly (GH#4078)
Carry reconciliation for a change on main since #4383 was written:
SetConfig now writes status.custom into custom_statuses and types.custom into
custom_types in the same transaction as the config row (SyncConfigTables),
and reads of those sets are table-first. CommitConfigOnly staged config
alone, so `bd config set status.custom ...` in server mode would commit the
row and leave its projection dirty: HEAD would carry the new string with the
old lookup set, the split the projection exists to prevent. The
transactional SetConfig already marks the synced table dirty for this reason.
CommitConfigOnly now stages config plus issueops.ConfigProjectionTables(),
a list kept next to SyncConfigTables. Clean tables stage nothing, and the
projections are written only from config, so no other kind of work is
swept. CommitConfigUserKVOnly keeps staging config alone: memory rows have
no projection.
Test (server container): status.custom and types.custom writes leave
config, custom_statuses and custom_types all clean after CommitConfigOnly.
* fix(cli): commit config writes in server mode — remember/forget/config set (GH#4078)
In server mode nothing ever commits config-table writes: maybeAutoCommit
skips SQL-server modes entirely, and the explicit flush commands call
Commit(), which excludes config (GH#2455) and silently no-ops on a
config-only working set. Every bd remember / bd config set then strands
the working set dirty and the next 'bd dolt pull' fails with 'cannot
merge with uncommitted changes' until a manual raw-SQL flush.
commitConfigWrite commits the config table immediately after an
intentional config write, scoped via CommitConfigOnly so concurrent
operations' dirty tables are never swept. Server modes only — embedded
'-Am' auto-commit and the unflagged-writes sweep already cover config.
This restores the v1.0.1 always-commit behavior from PR #3052 that the
unified-PostRun refactor regressed.
Wired sites: bd remember, bd forget, bd config set, bd config unset,
bd config set-many (one commit for the whole batch). Commit failures
surface loudly per each site's error convention.
(cherry picked from commit 64f77a1963)
Carry note: rebased onto b1bee123b. config set/unset now write through the
WorkspaceConfig role and remember/forget through the memoryops role, so the
conflict resolution re-anchors each commitConfigWrite call right after that
site's noteDirectConfigWrite/noteDirectMemoryWrite (set-many: after its
direct-route store.SetConfig loop) and uses each RunE's current error
convention (return HandleError / HandleErrorRespectJSON instead of os.Exit).
No change to commitConfigWrite or its tests in this commit.
* fix(cli): reconcile commitConfigWrite with current main (GH#4078)
Carry reconciliation on top of the cherry-picked commit, for two things
that changed on main since #4383 was written:
- Proxied route: proxied mode never opens a store, and its WorkspaceConfig
and memory roles commit each write inside their own unit of work
(noteDirectConfigWrite / noteDirectMemoryWrite say so). commitConfigWrite
now returns early on usesProxiedServer() instead of relying on the nil
store, and the proxied test asserts the no-op.
- dolt.auto-commit: bd-4wamg made batch/off real deferral in server mode,
and #5830 made `bd dolt commit` commit config (CommitAll). A config write
under batch/off now defers with every other server-mode write instead of
committing regardless; mode on (the default) commits as before. This
matches embedded mode, where the PersistentPostRun auto-commit honors the
same setting.
The doc comment drops the stale "-Am auto-commit and unflagged-writes
sweep" wording and says why rename-prefix and migrate are not wired.
* fix(cli): commit remember/forget through the kv.*-screened variant (GH#4078)
Carry reconciliation for the #4383 review: bd remember and bd forget write
user kv.* data, so their server-mode commit now goes through
CommitConfigUserKVOnly via a new commitMemoryWrite helper. A dirty internal
config key from another writer is refused with the keys named and the
`bd dolt commit` remedy, instead of being swept into the memory's commit.
bd config set/unset/set-many keep CommitConfigOnly: the operator is writing
config deliberately there, the same trust level as `bd dolt commit`.
commitConfigWrite and commitMemoryWrite share one guard path
(commitConfigTable), so both no-op on embedded, proxied, and batch/off.
* docs: changelog entries for the GH#4078 config-commit fixes
Server-mode config writes (bd remember/forget, config set/unset/set-many)
now commit immediately via the scoped CommitConfigOnly, and the explicit
'bd dolt commit'/'bd vc commit' are config-inclusive and truthful.
(cherry picked from commit c412b96b96)
Carry note: rebased onto b1bee123b. The hunk's old context now sits in a
released section, so it is moved under [Unreleased] > Fixed. The second
entry (bd dolt commit / bd vc commit) is dropped: that fix landed separately
as #5830 (CommitAll) and is already in the changelog.
* docs(changelog): update the GH#4078 entry for the carried fix
Rewords the cherry-picked entry for what the fix does on today's main:
batch/off defer config writes like every other server-mode write, the
commit stages config's lookup-table projections, remember/forget are
screened to kv.*, and #4412's pre-pull commit now absorbs dirty kv.* rows,
so the pull-side symptom is a refusal on internal keys rather than the
merge error. Drops the #2455 link, which does not resolve in this repo
(the number is a source-comment reference).
* test(cli): pin server-mode config writes leave no config dirt (GH#4078)
User-level regression test for the carried fix, on the direct SQL-server
route against the suite's Dolt server: bd remember, bd config set
(status.custom), bd config set-many (types.custom + status.custom), bd
config unset and bd forget must each leave config, custom_statuses and
custom_types clean in dolt_status. A final step dirties an internal config
key out of band and checks that bd remember refuses, names the key, and
leaves config dirty for bd dolt commit.
On b1bee123b (without the fix) it fails at its first step:
after bd remember --key gh4078 server-mode memory: config left
uncommitted in the working set (GH#4078)
With memory writes routed through the unscreened commit, it fails at the
last step: bd remember committed over a dirty internal config key.
* docs(cli): say why config writes skip the kv.* screen (GH#4078)
commitConfigWrite stages every dirty config row, while commitMemoryWrite
refuses internal keys. Record the reason in the code, not only in the
commit history: bd config set/unset/set-many are deliberate config
writes, the same trust level as bd dolt commit.
* build(bazel): list the GH#4078 test files in BUILD srcs
The rows the Bazel / test job's 'BUILD files in sync' check printed at ec27abd1d. Without them the Bazel tiers never run config_commit_write_test.go, config_commit_server_integration_test.go or commit_config_test.go, and the server-Dolt storage tier fails its shard check for the six storage tests.
* fix(storage): say the kv.* write is already saved when the screen refuses (GH#4078)
When CommitConfigUserKVOnly refuses, the memory row's SQL transaction has already committed, so exit 1 read as 'not stored'. The refusal now says the write is in the working set and that bd dolt commit publishes it with those keys.
* fix(httpclient): add the scoped config commits to the no-op commit family
The PR adds CommitConfigOnly and CommitConfigUserKVOnly to
storage.VersionControl. Since the PR's merge base, main added
internal/httpclient, whose Store asserts storage.DoltStorage, so the
tree merged with main did not build (*Store does not implement
storage.DoltStorage: missing method CommitConfigOnly).
Add both methods to httpclient's commit family as no-ops, for that
family's D3 reason: a write over this client is durable once the server
answers, so there is nothing to flush, and refusing would fail a command
whose write already landed. Regenerate unsupported_gen.go: only its
trailer count moves (201 -> 203 methods), because both methods are
hand-written rather than stubbed. Extend the two tests that pin the
family by name, TestCommitFamilyIsNoOp and
TestCommitFamilyIsNotOnTheUnsupportedMap, to cover both methods.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Eddie the Engineer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: sms <114885497+seanmartinsmith@users.noreply.github.com>
* fix(config): screen the staged set, warn on refused memory commits (GH#4078)
Review fixes for the adopted PR, iterations 1 and 2.
CommitConfigUserKVOnly screened the working set on one pooled
connection and staged config on another, so an internal key written in
between (issue_prefix above all) was committed with a memory: the
GH#2455 hazard the screen exists to refuse. The screen, DOLT_ADD and
DOLT_COMMIT now share one pinned connection (stageAndCommit), and the
screen runs twice: on the working set before DOLT_ADD, and on what
DOLT_ADD actually staged after it, because sessions on one branch share
the working set. A refusal after staging unstages config again with
DOLT_RESET, so the refused rows are not left staged for the next commit
on the branch. Two residuals are documented at the code: another commit
path's own DOLT_ADD of config landing between the second screen and
DOLT_COMMIT is absorbed, as for every scoped commit, and the table-wide
reset can unstage such a commit's config staging. The refusal is now a
*storage.ConfigCommitRefusedError carrying the keys.
The two config-table commits also stop widening their staging list
with the versioned-history tables: a config write mints no version
rows, and that list's issues table swept concurrent issue work into a
config or memory commit.
bd remember and bd forget exited 1 after their write had landed
whenever a non-kv.* config key was dirty, including keys bd's own
unwired writers leave, such as the issue_prefix of bd rename-prefix.
They now warn on stderr with the keys named and exit 0, as v1.0.1
(#3052) did on a failed memory commit. Under --json they add
committed: false, commit_error and dirty_config_keys, only when the
commit did not happen, so a committed write's payload keeps its shape.
The CHANGELOG entry now says so instead of claiming that the refusal
restores the v1.0.1 behavior.
bd config unset of a yaml-only secret key deleted the credential row an
older bd had stored, but never committed the delete, so HEAD, which
bd dolt push publishes, kept the credential while --json reported
database_row_removed. The delete is now committed like any other config
write. A failed commit is a warning, and database_commit_failed under
--json, because the delete and the YAML unset both landed.
A commit error the store marks storage.ErrCommitIndeterminate, such as
a connection lost during DOLT_COMMIT, may have landed, so neither
surface reports it as a definite non-commit any more. --json adds
commit_indeterminate (remember, forget) or
database_commit_indeterminate (the secret-row unset) beside
committed: false and database_commit_failed, which stay, so a consumer
that reads only those is still told to commit; the secret-row warning
and the note a failed YAML unset carries say the removal may not be
committed and point at bd dolt commit.
commitConfigWrite's doc now says why a failed commit fails the config
verbs while the memory verbs only warn, and names bd kv set/clear and
the kv.memory rows bd import writes as config writes it does not wire
yet. CommitWithConfig's interface doc drops bd config set, which on
the SQL-server route now commits through CommitConfigOnly, and points a
command committing only its own config write at the scoped commits.
Tests: a live-store test writes an internal key between the screen and
DOLT_ADD and requires a refusal that moves no HEAD and leaves nothing
staged; a sqlmock test pins the refusal SQL, including a failed
unstage; a versioned-history-on test requires a dirty issues table to
stay dirty through both scoped commits; cmd/bd fakes pin the
warn-and-succeed contract, the JSON fields, the secret-row commit, and
the indeterminate qualifiers in both payloads and both texts; and the
server E2E gains a secret-row step plus a bd rename-prefix step ahead
of bd remember and bd forget.
Pre-approval CI repair: tools/nogo/unvalidated_integration_tests.txt
now lists the PR's integration-tagged
cmd/bd/config_commit_server_integration_test.go. Since #7486 no nogo
lane analyzes integration-tagged test files, and
TestUnvalidatedIntegrationTestsFileIsExact requires the list to name
every one of them, so the new file failed //scripts:nogo_integration_scan_test
in the Bazel test and integration lanes.
Left for follow-ups, both pre-existing and not changed here: the
review's projection minor (config delete paths do not reconcile
custom_statuses/custom_types), and committing bd kv set/clear and
bd import's kv.memory rows on the SQL-server route.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: sms <114885497+seanmartinsmith@users.noreply.github.com>
---------
Co-authored-by: Aaron Lippold <lippold@gmail.com>
Co-authored-by: sms <114885497+seanmartinsmith@users.noreply.github.com>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com>
726 lines
28 KiB
Go
726 lines
28 KiB
Go
package main
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"os"
|
|
"sort"
|
|
"strings"
|
|
|
|
"github.com/spf13/cobra"
|
|
|
|
"github.com/steveyegge/beads/internal/config"
|
|
"github.com/steveyegge/beads/internal/memoryapi"
|
|
"github.com/steveyegge/beads/internal/metrics"
|
|
"github.com/steveyegge/beads/internal/storage"
|
|
"github.com/steveyegge/beads/internal/storage/kvkeys"
|
|
"github.com/steveyegge/beads/internal/storage/uow"
|
|
"github.com/steveyegge/beads/memoryops"
|
|
)
|
|
|
|
// openMemories hands back the persistent-memory role for whichever route this
|
|
// invocation is on, each through its OWN capability accessor — the store's for
|
|
// the direct route and the provider's for the proxied one.
|
|
//
|
|
// directRequirement is the message `ensureDirectMode` reports when a workspace
|
|
// is reachable by neither route. It is per-verb because the shipped text names
|
|
// the verb.
|
|
func openMemories(directRequirement string) (memoryops.Memories, error) {
|
|
if usesProxiedServer() {
|
|
return proxiedMemories()
|
|
}
|
|
if err := ensureDirectMode(directRequirement); err != nil {
|
|
return nil, err
|
|
}
|
|
return store.Memories()
|
|
}
|
|
|
|
// proxiedMemories hands back the guarded persistent-memory surface for this
|
|
// invocation's proxied-server provider, through the provider's OWN capability
|
|
// accessor — the same two-step proxiedWorkspaceConfig performs.
|
|
func proxiedMemories() (memoryops.Memories, error) {
|
|
if uowProvider == nil {
|
|
return nil, errors.New("proxied-server UOW provider not initialized")
|
|
}
|
|
return memoriesFromProvider(uowProvider)
|
|
}
|
|
|
|
// memoriesFromProvider is that accessor step for a provider the caller names.
|
|
//
|
|
// It takes the provider rather than reading the global one because `bd prime`
|
|
// opens a provider SCOPED to its read — prime is in noDbCommands, so the root
|
|
// pre-run opens nothing — and one spelling of "ask this provider for the memory
|
|
// surface" is the whole point of having an accessor at all.
|
|
func memoriesFromProvider(provider uow.UnitOfWorkProvider) (memoryops.Memories, error) {
|
|
src, ok := provider.(uow.MemoriesSource)
|
|
if !ok {
|
|
return nil, fmt.Errorf("proxied-server provider %T does not offer the persistent-memory surface", provider)
|
|
}
|
|
return src.Memories()
|
|
}
|
|
|
|
// noteDirectMemoryWrite marks the invocation as having written, which is what
|
|
// the auto-commit epilogue in main.go keys on.
|
|
//
|
|
// It is DIRECT-ROUTE ONLY, and both halves of that matter. A direct memory
|
|
// write lands in the Dolt working set and nothing else commits it, so a verb
|
|
// that forgets to call this stores a memory that exists until the process exits
|
|
// and then sits uncommitted — visible to the session that wrote it and to
|
|
// nothing after. A proxied write already committed inside the role's unit of
|
|
// work, so flagging it there would ask the epilogue to commit a second time on
|
|
// a route with nothing outstanding.
|
|
//
|
|
// The RunEs cannot make that distinction themselves: openMemories hides which
|
|
// route they are on, which is the point. So the guard lives here, once, the way
|
|
// noteDirectConfigWrite does for the settings plane.
|
|
func noteDirectMemoryWrite() {
|
|
if !usesProxiedServer() {
|
|
commandDidWrite.Store(true)
|
|
}
|
|
}
|
|
|
|
// memoryPrefix is prepended (after kvPrefix) to all memory keys.
|
|
const memoryPrefix = kvkeys.MemoryPrefix
|
|
|
|
// memoryKeyFlag allows explicit key override for bd remember.
|
|
var memoryKeyFlag string
|
|
|
|
// matchesKnownCommand reports whether insight is a single bare word that
|
|
// matches the name or an alias of a top-level bd command. It is used to catch
|
|
// `bd remember <subcommand>` mistakes before they become accidental memories.
|
|
// Multi-word insights (the normal case) always pass, since they contain
|
|
// whitespace and so cannot be a single command token.
|
|
func matchesKnownCommand(cmd *cobra.Command, insight string) (string, bool) {
|
|
word := strings.TrimSpace(insight)
|
|
if word == "" || strings.ContainsAny(word, " \t\r\n") {
|
|
return "", false
|
|
}
|
|
for _, c := range cmd.Root().Commands() {
|
|
if strings.EqualFold(c.Name(), word) {
|
|
return c.Name(), true
|
|
}
|
|
for _, alias := range c.Aliases {
|
|
if strings.EqualFold(alias, word) {
|
|
return c.Name(), true
|
|
}
|
|
}
|
|
}
|
|
return "", false
|
|
}
|
|
|
|
// memoryForceFlag is `bd remember --force`: write even when the corpus budget
|
|
// would be crossed. It only has meaning when memories.budget-chars is set.
|
|
var memoryForceFlag bool
|
|
|
|
// memoryConfigInt reads an integer config key (stubbable for tests), the way
|
|
// primeConfigInt does for the prime-side injection caps.
|
|
var memoryConfigInt = func(key string) int {
|
|
return config.GetInt(key)
|
|
}
|
|
|
|
// memoryCorpusBudget resolves the memories.budget-chars ceiling. 0 — the
|
|
// default — means OFF, and so does a negative value: the budget is opt-in, and
|
|
// a nonsense setting must not turn into a refusal the operator never asked for.
|
|
func memoryCorpusBudget() int {
|
|
budget := memoryConfigInt("memories.budget-chars")
|
|
if budget < 0 {
|
|
return 0
|
|
}
|
|
return budget
|
|
}
|
|
|
|
// memoryCorpusChars sums len(key)+len(content) over a corpus.
|
|
//
|
|
// The unit is BYTES (Go len), deliberately the same unit prime's
|
|
// --max-memory-chars cap counts in, so the two numbers are comparable. They are
|
|
// not the same MEASURE: prime counts each entry as it renders it — the key and
|
|
// value plus a few bytes of heading or bullet markup per memory, and the
|
|
// compact form truncates long values — so a corpus sitting exactly at a budget
|
|
// equal to prime.max-memory-chars can still have memories elided at injection.
|
|
// A budget meant to keep prime from eliding anything has to sit below that cap.
|
|
func memoryCorpusChars(memories map[string]string) int {
|
|
total := 0
|
|
for k, v := range memories {
|
|
total += len(k) + len(v)
|
|
}
|
|
return total
|
|
}
|
|
|
|
// projectedMemoryCorpusChars is that sum AFTER a write of content under key
|
|
// would apply. An OVERWRITE replaces the key's old content in the sum instead
|
|
// of adding to it — the delta, not the sum — which is what makes editing an
|
|
// existing memory possible at all once a corpus is near its ceiling.
|
|
func projectedMemoryCorpusChars(existing map[string]string, key, content string) int {
|
|
total := memoryCorpusChars(existing)
|
|
if old, ok := existing[key]; ok {
|
|
total -= len(key) + len(old)
|
|
}
|
|
return total + len(key) + len(content)
|
|
}
|
|
|
|
// memoryBudgetLine renders the ONE stderr line the budget check emits, and
|
|
// says whether the write must be refused.
|
|
//
|
|
// Three outcomes, in order of severity:
|
|
// - over budget, no --force: refuse, naming the override.
|
|
// - over budget with --force: write, but SAY SO — a forced crossing that
|
|
// printed nothing would make the budget invisible exactly when it matters.
|
|
// - at or above 80% of budget and within it: write, warn. That is the whole
|
|
// point of a budget over a tripwire: the warning arrives while there is
|
|
// still room to act on it.
|
|
//
|
|
// Below 80%, and whenever the budget is off, it returns "" and the command is
|
|
// silent — byte-identical to the pre-budget behavior.
|
|
//
|
|
// The percentage FLOORS within the budget and CEILINGS over it, and the
|
|
// rounding is not cosmetic: floor(201*100/200) is 100, and "100%" is exactly
|
|
// the at-budget reading this code deliberately ALLOWS — so a floored refusal
|
|
// would report the number of the case it is not. Ceiling above the line makes
|
|
// every over-budget line read 101% or more, and the split is on the VALUE, not
|
|
// on --force, so the same projection never reports two different percentages
|
|
// depending on a flag. Both forms are int64 arithmetic, so a large budget
|
|
// cannot overflow the comparison.
|
|
func memoryBudgetLine(projected, budget int, force bool) (line string, refuse bool) {
|
|
if budget <= 0 {
|
|
return "", false
|
|
}
|
|
pct := int(int64(projected) * 100 / int64(budget))
|
|
if projected > budget {
|
|
pct = int((int64(projected)*100 + int64(budget) - 1) / int64(budget))
|
|
}
|
|
switch {
|
|
case projected > budget && !force:
|
|
return fmt.Sprintf("bd remember: memory corpus would be %d chars, budget is %d (%d%%) — refused; use --force to override", projected, budget, pct), true
|
|
case projected > budget:
|
|
return fmt.Sprintf("bd remember: memory corpus at %d of %d chars (%d%%) — over budget, written anyway (--force)", projected, budget, pct), false
|
|
case int64(projected)*5 >= int64(budget)*4:
|
|
return fmt.Sprintf("bd remember: memory corpus at %d of %d chars (%d%%)", projected, budget, pct), false
|
|
}
|
|
return "", false
|
|
}
|
|
|
|
// rememberBudgetVerdict is the whole budget decision as ONE pure function:
|
|
// given the corpus as it stands, the write about to be made, the budget and
|
|
// --force, it answers with the line to print (empty = say nothing) and whether
|
|
// the write must be refused. The command around it only has to fetch the
|
|
// corpus and obey.
|
|
//
|
|
// Keeping it pure is what lets every branch — off, boundary, overwrite delta,
|
|
// force, warn — be pinned by a test that needs neither a database nor cgo.
|
|
func rememberBudgetVerdict(existing map[string]string, key, content string, budget int, force bool) (line string, refuse bool) {
|
|
if budget <= 0 {
|
|
return "", false
|
|
}
|
|
return memoryBudgetLine(projectedMemoryCorpusChars(existing, key, content), budget, force)
|
|
}
|
|
|
|
// rememberBareKeyPath implements the desire-path / footgun guard for
|
|
// `bd remember <bare-slug>` (no --key): a bare slug naming an EXISTING memory
|
|
// is recalled instead of stored; a bare slug naming nothing is refused. The
|
|
// caller only invokes it when memoryKeyFlag == "" and the insight round-trips
|
|
// through memoryapi.DeriveKey unchanged, having already read the key. As in
|
|
// printRecallResult, presence is found (row existence), never existing != "":
|
|
// a bare slug naming a memory stored as the empty string recalls it.
|
|
func rememberBareKeyPath(key, insight, existing string, found bool) error {
|
|
if found {
|
|
if jsonOutput {
|
|
return outputJSON(map[string]interface{}{
|
|
"key": key,
|
|
"value": existing,
|
|
"found": true,
|
|
"action": "recalled",
|
|
})
|
|
}
|
|
fmt.Fprintf(os.Stderr,
|
|
"(recalled %q -- a bare existing key READS. To overwrite: `bd remember \"<new content>\" --key %s`)\n",
|
|
key, key)
|
|
fmt.Printf("%s\n", existing)
|
|
return nil
|
|
}
|
|
return HandleErrorRespectJSON(
|
|
"no memory named %q to recall -- and refusing to store a bare key-like token as its own content. "+
|
|
"`bd remember` WRITES (its positional arg is CONTENT, not a key). "+
|
|
"To store it anyway: `bd remember %q --key %s`. To browse keys: `bd memories`",
|
|
key, insight, key)
|
|
}
|
|
|
|
// printRememberResult renders the `bd remember` success output. commitErr is
|
|
// the memory commit's failure, if any; the memory is stored either way.
|
|
func printRememberResult(verb, key, insight string, commitErr error) error {
|
|
if jsonOutput {
|
|
payload := map[string]interface{}{
|
|
"key": key,
|
|
"value": insight,
|
|
"action": strings.ToLower(verb),
|
|
}
|
|
memoryCommitFields(payload, commitErr)
|
|
return outputJSON(payload)
|
|
}
|
|
fmt.Printf("%s [%s]: %s\n", verb, key, truncateMemory(insight, 80))
|
|
return nil
|
|
}
|
|
|
|
// memoryCommitFields tells a `--json` consumer that the write landed but its
|
|
// Dolt commit did not, which the exit code no longer says (see the commit
|
|
// sites). The fields are added only then, so a committed write's payload keeps
|
|
// its shipped shape. dirty_config_keys names the internal keys the kv.* screen
|
|
// refused, which is what the user has to commit or revert. commit_indeterminate
|
|
// marks an error after which the store cannot say whether the commit landed
|
|
// (storage.ErrCommitIndeterminate, such as a connection lost during
|
|
// DOLT_COMMIT). committed stays false beside it, so a consumer that reads only
|
|
// committed never takes a memory that may be uncommitted for a committed one;
|
|
// `bd dolt commit` settles it either way.
|
|
func memoryCommitFields(payload map[string]interface{}, commitErr error) {
|
|
if commitErr == nil {
|
|
return
|
|
}
|
|
payload["committed"] = false
|
|
payload["commit_error"] = commitErr.Error()
|
|
if errors.Is(commitErr, storage.ErrCommitIndeterminate) {
|
|
payload["commit_indeterminate"] = true
|
|
}
|
|
var refused *storage.ConfigCommitRefusedError
|
|
if errors.As(commitErr, &refused) {
|
|
payload["dirty_config_keys"] = refused.Keys
|
|
}
|
|
}
|
|
|
|
// printMemoriesResult renders the `bd memories` output.
|
|
func printMemoriesResult(memories map[string]string, search string) error {
|
|
if jsonOutput {
|
|
return outputJSON(memories)
|
|
}
|
|
|
|
if len(memories) == 0 {
|
|
if search != "" {
|
|
fmt.Printf("No memories matching %q\n", search)
|
|
} else {
|
|
fmt.Println("No memories stored. Use 'bd remember \"insight\"' to add one.")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
keys := make([]string, 0, len(memories))
|
|
for k := range memories {
|
|
keys = append(keys, k)
|
|
}
|
|
sort.Strings(keys)
|
|
|
|
if search != "" {
|
|
fmt.Printf("Memories matching %q:\n\n", search)
|
|
} else {
|
|
fmt.Printf("Memories (%d):\n\n", len(memories))
|
|
}
|
|
for _, k := range keys {
|
|
v := memories[k]
|
|
fmt.Printf(" %s\n", k)
|
|
fmt.Printf(" %s\n\n", truncateMemory(v, 120))
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// printForgetNotFound renders the `bd forget` missing-key output (including
|
|
// the SilentExit contract).
|
|
func printForgetNotFound(key string) error {
|
|
if jsonOutput {
|
|
if jerr := outputJSON(map[string]string{
|
|
"key": key,
|
|
"found": "false",
|
|
}); jerr != nil {
|
|
return jerr
|
|
}
|
|
return SilentExit()
|
|
}
|
|
fmt.Fprintf(os.Stderr, "No memory with key %q\n", key)
|
|
return SilentExit()
|
|
}
|
|
|
|
// printForgetResult renders the `bd forget` success output. commitErr is as
|
|
// for printRememberResult.
|
|
func printForgetResult(key, existing string, commitErr error) error {
|
|
if jsonOutput {
|
|
payload := map[string]interface{}{
|
|
"key": key,
|
|
"deleted": "true",
|
|
}
|
|
memoryCommitFields(payload, commitErr)
|
|
return outputJSON(payload)
|
|
}
|
|
fmt.Printf("Forgot [%s]: %s\n", key, truncateMemory(existing, 80))
|
|
return nil
|
|
}
|
|
|
|
// printRecallResult renders the `bd recall` output (including the not-found
|
|
// SilentExit contract). Presence comes from found — ROW EXISTENCE, per the
|
|
// role's RecallResult.Found — never from value != "": a memory stored as the
|
|
// empty string is found and recalls as "", and only an absent key is a miss.
|
|
func printRecallResult(key, value string, found bool) error {
|
|
if jsonOutput {
|
|
if jerr := outputJSON(map[string]interface{}{
|
|
"key": key,
|
|
"value": value,
|
|
"found": found,
|
|
}); jerr != nil {
|
|
return jerr
|
|
}
|
|
if !found {
|
|
return SilentExit()
|
|
}
|
|
return nil
|
|
}
|
|
if !found {
|
|
fmt.Fprintf(os.Stderr, "No memory with key %q\n", key)
|
|
return SilentExit()
|
|
}
|
|
fmt.Printf("%s\n", value)
|
|
return nil
|
|
}
|
|
|
|
// rememberCmd stores a memory.
|
|
var rememberCmd = &cobra.Command{
|
|
Use: `remember "<insight>"`,
|
|
Short: "Store a persistent memory",
|
|
Long: `Store a memory that persists across sessions and account rotations.
|
|
|
|
Memories are injected at prime time (bd prime) so you have them
|
|
in every session without manual loading.
|
|
|
|
The positional arg is the memory CONTENT (the key is auto-generated from it
|
|
unless --key is given). As a convenience, if the arg is a bare key naming an
|
|
existing memory, it is RECALLED instead of stored (same as 'bd recall');
|
|
a bare key naming nothing is refused. Use --key to store slug-like content.
|
|
|
|
Corpus budget (opt-in, off by default): set the memories.budget-chars config
|
|
key to a byte ceiling for the whole memory corpus (sum of len(key)+len(content)).
|
|
A write that would cross it is refused, and --force overrides; a write that
|
|
lands at or above 80% of the budget warns. With the key unset nothing changes.
|
|
bd prime's --max-memory-chars also counts bytes, but of each memory as
|
|
injected, formatting included, so a budget equal to that cap does not
|
|
guarantee prime injects every memory; set the budget below it for that.
|
|
|
|
Examples:
|
|
bd remember "always run tests with -race flag"
|
|
bd remember "Dolt phantom DBs hide in three places" --key dolt-phantoms
|
|
bd remember "auth module uses JWT not sessions" --key auth-jwt
|
|
bd remember dolt-phantoms # bare existing key: reads it (= bd recall)`,
|
|
GroupID: "setup",
|
|
Args: cobra.ExactArgs(1),
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
CheckReadonly("remember") // also covers the migration freeze check (dc-6jaq)
|
|
|
|
evt := metrics.NewCommandEvent("remember")
|
|
defer func() {
|
|
if c := metrics.Global(); c != nil {
|
|
c.CloseEventAndAdd(evt)
|
|
}
|
|
}()
|
|
|
|
insight := args[0]
|
|
|
|
// Guard against a subcommand-like first argument being silently stored
|
|
// as memory content. `bd remember` is a leaf command, so a mistaken
|
|
// `bd remember recall` (or any bare bd command name) would otherwise
|
|
// store the word "recall" as a memory instead of doing what the user
|
|
// intended (GH#4401). A genuine insight is a phrase, so only a single
|
|
// bare word that matches a known command is treated as suspect, and an
|
|
// explicit --key signals deliberate intent and bypasses the guard.
|
|
//
|
|
// It stays at the FRONT DOOR and stays FIRST: it reads the cobra command
|
|
// tree, which no role can see, and it must answer before any storage is
|
|
// opened so that `bd remember list` in a directory with no workspace
|
|
// still says "looks like a command".
|
|
if memoryKeyFlag == "" {
|
|
if name, ok := matchesKnownCommand(cmd, insight); ok {
|
|
return HandleErrorWithHintRespectJSON(
|
|
fmt.Sprintf("%q looks like a command, not something to remember", insight),
|
|
fmt.Sprintf("Did you mean 'bd %s'? To store %q as a memory anyway, give it an explicit key: bd remember %q --key <key>", name, insight, insight),
|
|
)
|
|
}
|
|
}
|
|
|
|
memories, err := openMemories("remember requires direct database access")
|
|
if err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
|
|
// Desire path + footgun guard: `bd remember <x>` is a WRITE whose positional arg is
|
|
// the CONTENT, not a key -- but "remember X" reads as a getter in English, so agents
|
|
// routinely type `bd remember some-key` meaning "do you remember X?". The tell-tale of
|
|
// a mistyped read is content that round-trips through the key derivation unchanged (a
|
|
// bare slug); real prose insights never do. When that happens and no explicit --key was
|
|
// given:
|
|
// - the key EXISTS -> pave the desire path: recall it instead of writing
|
|
// - no such key -> refuse; storing a key-like token as its own content would
|
|
// create a junk memory that hides the mistake
|
|
// Passing --key states write intent and bypasses both branches.
|
|
//
|
|
// It stays ABOVE the role because it decides WHETHER TO WRITE AT ALL, and
|
|
// because it exists to disambiguate English: an HTTP POST is not ambiguous
|
|
// and must not inherit it. The read below is a plain Recall, so this whole
|
|
// branch touches nothing.
|
|
//
|
|
// `derived != ""` is load-bearing and is not decoration: DeriveKey("")
|
|
// is "", so without it every empty or unslugifiable insight would satisfy
|
|
// derived == insight and be routed into a "recall" of the empty key. The
|
|
// shipped code was saved from that by an empty-content check that ran
|
|
// first; that check is the role's now, so the condition has to say it.
|
|
derived := memoryapi.DeriveKey(insight)
|
|
if memoryKeyFlag == "" && derived != "" && derived == insight {
|
|
recalled, err := memories.Recall(rootCtx, memoryops.RecallRequest{Key: derived})
|
|
if err != nil {
|
|
return HandleErrorRespectJSON("recalling memory: %v", err)
|
|
}
|
|
return rememberBareKeyPath(derived, insight, recalled.Value, recalled.Found)
|
|
}
|
|
|
|
// Corpus budget (memories.budget-chars, OPT-IN, 0 = off). The
|
|
// wheelhouse-side tripwire only REPORTS the corpus size, so the corpus
|
|
// re-trips red every few days and nothing ever stops the write that
|
|
// crossed the line. This is the stop.
|
|
//
|
|
// It sits here, between the read-shaped branches above and the write
|
|
// below, because it is the last thing that can decide NOT to write —
|
|
// and above the role because a budget is a front-door policy, not a
|
|
// property of the memory plane (an HTTP POST must not inherit it).
|
|
//
|
|
// When the budget is off the whole block is skipped: no extra List, no
|
|
// output, nothing. Behavior is then byte-identical to the pre-budget
|
|
// command, which is what TestRememberBudgetVerdictOffIsSilent and the embedded
|
|
// budget_off_is_silent case pin.
|
|
//
|
|
// The key an overwrite is credited against is ResolveKey's, the one
|
|
// rule every Remember implementation lands a write under, so it cannot
|
|
// drift from the store's key. A write ResolveKey refuses (empty
|
|
// content, a whitespace-only --key, content that derives no key) skips
|
|
// the budget: Remember refuses it below with its own validation
|
|
// sentence, and a refused write crosses nothing.
|
|
budgetKey, budgetKeyErr := memoryapi.ResolveKey(memoryKeyFlag, insight)
|
|
var budgetLine string
|
|
if budget := memoryCorpusBudget(); budget > 0 && budgetKeyErr == nil {
|
|
// ADVISORY, not atomic: this List and the Remember below are
|
|
// separate transactions, so two concurrent writers can both measure
|
|
// a corpus inside the budget and both land, leaving it over. The
|
|
// budget is a guardrail against a corpus drifting past its ceiling
|
|
// one deliberate write at a time, not a hard limit — no locking.
|
|
corpus, err := memories.List(rootCtx, memoryops.ListRequest{})
|
|
if err != nil {
|
|
return HandleErrorRespectJSON("reading the memory corpus for the budget check: %v", err)
|
|
}
|
|
line, refuse := rememberBudgetVerdict(corpus.Memories, budgetKey, insight, budget, memoryForceFlag)
|
|
if refuse {
|
|
// The budget line IS the refusal sentence, so it prints as
|
|
// itself: routing it through HandleError would prefix it with
|
|
// "Error: " and reword text agents will match on.
|
|
//
|
|
// It is said ONCE, on exactly one stream, the way
|
|
// HandleErrorWithHintRespectJSON does it: --json gets only the
|
|
// machine-readable envelope on stdout (printing the prose to
|
|
// stderr as well would make the same refusal arrive twice),
|
|
// and every other route gets the bare sentence on stderr,
|
|
// where it can never corrupt structured output. Both routes
|
|
// carry the same hint, so a terminal user is offered the same
|
|
// ways out as a JSON consumer.
|
|
const hint = "raise memories.budget-chars, forget a memory, or pass --force"
|
|
if jsonOutput {
|
|
jsonStdoutError(line, hint)
|
|
} else {
|
|
fmt.Fprintln(os.Stderr, line)
|
|
fmt.Fprintf(os.Stderr, "Hint: %s\n", hint)
|
|
}
|
|
return SilentExit()
|
|
}
|
|
// Not a refusal: the write happens first and the line reports the
|
|
// corpus that now exists, so no line ever describes a write that
|
|
// did not land.
|
|
budgetLine = line
|
|
}
|
|
|
|
result, err := memories.Remember(rootCtx, memoryops.RememberRequest{Key: memoryKeyFlag, Content: insight})
|
|
if err != nil {
|
|
// The role's two refusals ARE this command's shipped sentences —
|
|
// "memory content cannot be empty" and "could not generate key from
|
|
// content; use --key to specify one" — so they print as themselves.
|
|
// Wrapping would reword output an agent may be matching on into
|
|
// "storing memory: validation failed: ..." to say the same thing.
|
|
if errors.Is(err, memoryops.ErrValidation) {
|
|
return HandleErrorRespectJSON("%s", strings.TrimPrefix(err.Error(), memoryops.ErrValidation.Error()+": "))
|
|
}
|
|
return HandleErrorRespectJSON("storing memory: %v", err)
|
|
}
|
|
noteDirectMemoryWrite()
|
|
// The memory is stored by now, so a commit the kv.* screen refuses (a
|
|
// dirty internal key such as `bd rename-prefix` leaves) or that fails
|
|
// is a warning, not the command's failure: exit 1 would report a write
|
|
// that landed as one that did not. v1.0.1 (#3052) warned and carried on
|
|
// the same way; --json gets the same news in memoryCommitFields.
|
|
commitErr := commitMemoryWrite(rootCtx, store, "remember")
|
|
if commitErr != nil {
|
|
WarnError("%v", commitErr)
|
|
}
|
|
|
|
if budgetLine != "" {
|
|
fmt.Fprintln(os.Stderr, budgetLine)
|
|
}
|
|
|
|
// Remembered versus Updated is Replaced, observed in the SAME
|
|
// transaction as the write. The shipped code read the row first and
|
|
// described a moment that had already passed.
|
|
verb := "Remembered"
|
|
if result.Replaced {
|
|
verb = "Updated"
|
|
}
|
|
return printRememberResult(verb, result.Key, result.Value, commitErr)
|
|
},
|
|
}
|
|
|
|
// memoriesCmd lists and searches memories.
|
|
var memoriesCmd = &cobra.Command{
|
|
Use: "memories [search]",
|
|
Short: "List or search persistent memories",
|
|
Long: `List all memories, or search by keyword.
|
|
|
|
Examples:
|
|
bd memories # list all memories
|
|
bd memories dolt # search for memories about dolt
|
|
bd memories "race flag" # search for a phrase`,
|
|
GroupID: "setup",
|
|
Args: cobra.MaximumNArgs(1),
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
evt := metrics.NewCommandEvent("memories")
|
|
defer func() {
|
|
if c := metrics.Global(); c != nil {
|
|
c.CloseEventAndAdd(evt)
|
|
}
|
|
}()
|
|
|
|
var search string
|
|
if len(args) > 0 {
|
|
search = args[0]
|
|
}
|
|
|
|
memories, err := openMemories("memories requires direct database access")
|
|
if err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
// The term goes to the role RAW. Case folding is List's, so the two
|
|
// routes cannot come to disagree about what matches — which is the
|
|
// whole reason the filter moved down.
|
|
result, err := memories.List(rootCtx, memoryops.ListRequest{Search: search})
|
|
if err != nil {
|
|
return HandleErrorRespectJSON("listing memories: %v", err)
|
|
}
|
|
|
|
// The ECHO, on the other hand, has always been lowercased: `bd memories
|
|
// FOO` prints `No memories matching "foo"`. It is a wart — a front door
|
|
// should say back what the user typed — but it is shipped output, and
|
|
// this commit is a convergence, not a change. Fixing it is a one-liner
|
|
// with its own test, like truncateMemory's rune splitting.
|
|
return printMemoriesResult(result.Memories, strings.ToLower(search))
|
|
},
|
|
}
|
|
|
|
// forgetCmd removes a memory.
|
|
var forgetCmd = &cobra.Command{
|
|
Use: "forget <key>",
|
|
Short: "Remove a persistent memory",
|
|
Long: `Remove a memory by its key.
|
|
|
|
Use 'bd memories' to see available keys.
|
|
|
|
Examples:
|
|
bd forget dolt-phantoms
|
|
bd forget auth-jwt`,
|
|
GroupID: "setup",
|
|
Args: cobra.ExactArgs(1),
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
CheckReadonly("forget")
|
|
|
|
evt := metrics.NewCommandEvent("forget")
|
|
defer func() {
|
|
if c := metrics.Global(); c != nil {
|
|
c.CloseEventAndAdd(evt)
|
|
}
|
|
}()
|
|
|
|
memories, err := openMemories("forget requires direct database access")
|
|
if err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
// No pre-read here, deliberately: the value printed below is the one
|
|
// the role's transaction actually deleted, not the one an earlier read
|
|
// happened to see.
|
|
result, err := memories.Forget(rootCtx, memoryops.ForgetRequest{Key: args[0]})
|
|
if err != nil {
|
|
return HandleErrorRespectJSON("forgetting memory: %v", err)
|
|
}
|
|
if !result.Found {
|
|
return printForgetNotFound(result.Key)
|
|
}
|
|
noteDirectMemoryWrite()
|
|
// As for remember: the delete landed, so a refused or failed commit
|
|
// warns and the command still succeeds.
|
|
commitErr := commitMemoryWrite(rootCtx, store, "forget")
|
|
if commitErr != nil {
|
|
WarnError("%v", commitErr)
|
|
}
|
|
|
|
return printForgetResult(result.Key, result.Value, commitErr)
|
|
},
|
|
}
|
|
|
|
// recallCmd retrieves a specific memory by key.
|
|
var recallCmd = &cobra.Command{
|
|
Use: "recall <key>",
|
|
Short: "Retrieve a specific memory",
|
|
Long: `Retrieve the full content of a memory by its key.
|
|
|
|
Examples:
|
|
bd recall dolt-phantoms
|
|
bd recall auth-jwt`,
|
|
GroupID: "setup",
|
|
Args: cobra.ExactArgs(1),
|
|
SilenceUsage: true,
|
|
SilenceErrors: true,
|
|
RunE: func(cmd *cobra.Command, args []string) error {
|
|
evt := metrics.NewCommandEvent("recall")
|
|
defer func() {
|
|
if c := metrics.Global(); c != nil {
|
|
c.CloseEventAndAdd(evt)
|
|
}
|
|
}()
|
|
|
|
memories, err := openMemories("recall requires direct database access")
|
|
if err != nil {
|
|
return HandleError("%v", err)
|
|
}
|
|
result, err := memories.Recall(rootCtx, memoryops.RecallRequest{Key: args[0]})
|
|
if err != nil {
|
|
return HandleErrorRespectJSON("recalling memory: %v", err)
|
|
}
|
|
|
|
return printRecallResult(result.Key, result.Value, result.Found)
|
|
},
|
|
}
|
|
|
|
// truncateMemory shortens a string to maxLen for display.
|
|
func truncateMemory(s string, maxLen int) string {
|
|
// Replace newlines with spaces for single-line display
|
|
s = strings.ReplaceAll(s, "\n", " ")
|
|
return truncate(s, maxLen)
|
|
}
|
|
|
|
func init() {
|
|
rememberCmd.Flags().StringVar(&memoryKeyFlag, "key", "", "Explicit key for the memory (auto-generated from content if not set). If a memory with this key already exists, it will be updated in place")
|
|
rememberCmd.Flags().BoolVar(&memoryForceFlag, "force", false, "Store even when the write would cross the memories.budget-chars corpus budget (no effect when the budget is unset)")
|
|
|
|
rootCmd.AddCommand(rememberCmd)
|
|
rootCmd.AddCommand(memoriesCmd)
|
|
rootCmd.AddCommand(forgetCmd)
|
|
rootCmd.AddCommand(recallCmd)
|
|
}
|