* fix(memory): preserve presence for empty values (#5963)
Recall and Forget now distinguish an existing empty memory row from an absent key across direct, embedded, and UOW storage paths.
Agent-Signature: codex-unknown-model-unknown-reasoning on behalf of sjarmak
* fix(memory): make bd recall report presence from row existence
printRecallResult (and the recallCmd caller) derived "found" from
value != "", so a memory stored as the empty string still printed
"No memory with key" and exited 1 even though the role's
RecallResult.Found was true and bd forget would delete the row.
Thread result.Found through instead: an empty-valued memory now
recalls cleanly (exit 0, empty value; JSON reports found: true), and
only an absent key is a miss.
Addresses maintainer review on #5964 (fixes #5963).
* test(httpapi): pin GET /v0/beads/memories/{key} as 200 for an empty value
TestGetMemoryAnswersAMissWithA404 pinned a RecallResult{Value: ""}
case (Found defaulting to its zero value, false) as a 404, a shape no
backend can produce since Found now comes from row existence rather
than value truthiness. Move that case into its own
TestGetMemoryAnswersAnEmptyValueWith200, which recalls Found: true
and asserts a 200 with an empty value. Also updates the handler's
doc comment, which still described the retired 404-on-empty
behavior.
Addresses maintainer review on #5964 (fixes #5963).
* docs(httpapi): fix openapi spec to say memory presence is row existence
The published spec still said an empty-valued memory is a 404 on
GET /v0/beads/memories/{key} and DELETE /v0/beads/memories/{key},
contradicting the role's RecallResult.Found (row existence). Rewrite
the operation descriptions, both 404 response bodies, Memory.value
and RememberedMemory.replaced to say presence is row existence
everywhere: an empty-string memory is a 200 (or is removed by
forget) with an empty value, matching what GET /v0/beads/memories
already enumerated. Regenerate types.gen.go via `make api-gen` so
the generated doc comments match.
Addresses maintainer review on #5964 (fixes #5963).
* docs(changelog): note the empty-valued memory presence fix
Add an Unreleased/Fixed entry for #5964 (fixes #5963): bd recall,
bd forget and GET /v0/beads/memories/{key} now treat an empty-valued
memory as present everywhere, instead of only on GET
/v0/beads/memories.
* fix(memory): make bd remember <bare-key> recall an empty-valued memory
rememberBareKeyPath decided presence from existing != "", so a bare slug
naming a memory stored as the empty string was refused as "no memory
named" instead of recalled. It now takes the role's Found, matching
printRecallResult, bd forget and the HTTP door.
* test(memory): pin empty-value presence through the real commands
TestPrintRecallResult and TestRememberBareKeyPath call the helpers directly,
so a RunE that went back to deciding presence from value != "" would leave
them green. TestEmbeddedMemory/empty_valued_memory_is_present empties a
memory row beneath bd and runs bd recall, bd remember <bare-key> and
bd forget against it, checking first that bd no longer sees the placeholder.
Two comments in problem.go and one in memories_test.go still said an
empty-valued memory is a 404. The CHANGELOG line now says a key with no row
is a miss, since a forgotten key was once stored.
* build(bazel): add the memoryops dep to the bd_test go_test (make bazel-sync)
memory_embedded_test.go now imports internal/storage/memoryops directly
(storagememoryops.StorageKey, to overwrite a memory row out of band), so
gazelle emits //internal/storage/memoryops on bd_test. cmd/bd/BUILD.bazel
landed on main after this PR branched, so the PR could not carry the dep;
without it, bazel.yml's "BUILD files in sync" step fails in the required
bazel-test lane. Generated by `make bazel-sync`.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(memory): maintainer fixup — adoption review nits
Maintainer fixup for the adopt-pr iteration-1 review of #5964. Both review
lanes approved and the quality scorecard approved at 981; these are its four
non-gating nits, each confined to lines this PR already adds.
- internal/httpapi: TestForgetMemoryOfAnEmptyValuedKeyIs200 pins the DELETE
door's empty-value 200 the way TestGetMemoryAnswersAnEmptyValueWith200
pins the read. Guarding the handler's 404 on Value == "" reddens exactly
this test and nothing else.
- CHANGELOG: the bullet names DELETE /v0/beads/memories/{key} beside GET,
whose answer changed the same way (404 with the row left standing -> 200
with the row removed).
- uow: the two new GetAllConfig presence probes in Recall and Forget say why
they read the whole table, by pointing at Remember's rationale.
- conformance: RunMemoriesEmptyRowIsPresentUntilForgotten runs its
no-history-on-retry leg last and SKIPs it with a reason when a fixture has
no CountHistory, as the MemoriesFixture doc promises, instead of passing
quietly. All three in-tree fixtures wire the hook, so every in-tree run
is unchanged (19/19 on dolt, embeddeddolt and uow, 0 SKIP).
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(httpclient): report an empty-valued memory as present (base-refresh repair)
Maintainer repair from refreshing #5964 onto main b630f96f5b. Main's #7247
added the HTTP client store (internal/httpclient) after this PR branched,
and its memory role and served conformance leg still treated a memory
stored as the empty string as absent:
- Recall set Found from Value != "".
- The RecallMemory doc said an empty-valued row was a 404.
- TestServedMemoriesRecallConflatesStoredEmptyWithAbsent called
conformance.RunMemoriesRecallConflatesStoredEmptyWithAbsent, which this
PR deletes.
The rebase applied cleanly, but the merged tree did not compile.
- internal/httpclient/memories.go: Recall reports Found true on any 200.
With this PR the wire returns a 404 only for a key with no row; an
empty-valued row comes back as a 200 with an empty value. Setting Found
from Value would hide that row again on the HTTP backend. Forget already
reported Found true on a 200.
- internal/httpclient/wire/writes.go: the RecallMemory doc now says the
same.
- internal/httpclient/served_memories_deps_test.go: the hm10 served case
now runs RunMemoriesEmptyRowIsPresentUntilForgotten, the contract case
this PR adds in place of the one it deletes.
With BEADS_TEST_EMBEDDED_DOLT=1 the served memory tests pass with 0 SKIP.
Reverting the Recall line makes hm10 fail with
`Recall("hm10-stored-empty") = value "" found false, want value "" found
true`.
Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: sjarmak <t@t.co>
Co-authored-by: Eddie the Engineer <adopt-pr@gascity.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
760 lines
29 KiB
Go
760 lines
29 KiB
Go
//go:build cgo
|
|
|
|
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"strings"
|
|
"sync"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/configfile"
|
|
"github.com/steveyegge/beads/internal/storage/embeddeddolt"
|
|
storagememoryops "github.com/steveyegge/beads/internal/storage/memoryops"
|
|
)
|
|
|
|
// countMemoryDoltCommits reads dolt_log for an embedded workspace. It is how
|
|
// the auto-commit epilogue is observed: a direct-route write that never flags
|
|
// it still answers `bd recall` perfectly well, because the row is in the
|
|
// working set — dolt_log is the only place the difference shows.
|
|
func countMemoryDoltCommits(t *testing.T, beadsDir string) int {
|
|
t.Helper()
|
|
cfg, _ := configfile.Load(beadsDir)
|
|
database := ""
|
|
if cfg != nil {
|
|
database = cfg.GetDoltDatabase()
|
|
}
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), database, "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
var count int
|
|
if err := db.QueryRowContext(t.Context(), "SELECT COUNT(*) FROM dolt_log").Scan(&count); err != nil {
|
|
t.Fatalf("query dolt_log: %v", err)
|
|
}
|
|
return count
|
|
}
|
|
|
|
// storeMemoryValueOutOfBand overwrites a memory row's value with SQL, beneath
|
|
// every bd code path. It is the only way a test gets an empty-valued memory:
|
|
// `bd remember` refuses empty content, so such a row always arrives out of band.
|
|
func storeMemoryValueOutOfBand(t *testing.T, beadsDir, key, value string) {
|
|
t.Helper()
|
|
cfg, _ := configfile.Load(beadsDir)
|
|
database := ""
|
|
if cfg != nil {
|
|
database = cfg.GetDoltDatabase()
|
|
}
|
|
db, cleanup, err := embeddeddolt.OpenSQL(t.Context(), filepath.Join(beadsDir, "embeddeddolt"), database, "main")
|
|
if err != nil {
|
|
t.Fatalf("OpenSQL: %v", err)
|
|
}
|
|
defer cleanup()
|
|
res, err := db.ExecContext(t.Context(), "UPDATE config SET value = ? WHERE `key` = ?", value, storagememoryops.StorageKey(key))
|
|
if err != nil {
|
|
t.Fatalf("update memory %q: %v", key, err)
|
|
}
|
|
if n, err := res.RowsAffected(); err != nil || n != 1 {
|
|
t.Fatalf("update memory %q: %d rows affected (err %v), want 1", key, n, err)
|
|
}
|
|
}
|
|
|
|
// bdRemember runs "bd remember" with the given args and returns stdout.
|
|
func bdRemember(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"remember"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
if err != nil {
|
|
t.Fatalf("bd remember %s failed: %v\nstdout:\n%s\nstderr:\n%s", strings.Join(args, " "), err, stdout.String(), stderr.String())
|
|
}
|
|
return stdout.String()
|
|
}
|
|
|
|
// bdRememberFail runs "bd remember" expecting failure.
|
|
func bdRememberFail(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"remember"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err == nil {
|
|
t.Fatalf("expected bd remember %s to fail, but succeeded:\n%s", strings.Join(args, " "), out)
|
|
}
|
|
return string(out)
|
|
}
|
|
|
|
// bdRecall runs "bd recall" with the given args and returns stdout.
|
|
func bdRecall(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"recall"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
if err != nil {
|
|
t.Fatalf("bd recall %s failed: %v\nstdout:\n%s\nstderr:\n%s", strings.Join(args, " "), err, stdout.String(), stderr.String())
|
|
}
|
|
return stdout.String()
|
|
}
|
|
|
|
// bdRecallFail runs "bd recall" expecting failure.
|
|
func bdRecallFail(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"recall"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err == nil {
|
|
t.Fatalf("expected bd recall %s to fail, but succeeded:\n%s", strings.Join(args, " "), out)
|
|
}
|
|
return string(out)
|
|
}
|
|
|
|
// bdMemories runs "bd memories" with the given args and returns stdout.
|
|
func bdMemories(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"memories"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
if err != nil {
|
|
t.Fatalf("bd memories %s failed: %v\nstdout:\n%s\nstderr:\n%s", strings.Join(args, " "), err, stdout.String(), stderr.String())
|
|
}
|
|
return stdout.String()
|
|
}
|
|
|
|
// bdForget runs "bd forget" with the given args and returns stdout.
|
|
func bdForget(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"forget"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
if err != nil {
|
|
t.Fatalf("bd forget %s failed: %v\nstdout:\n%s\nstderr:\n%s", strings.Join(args, " "), err, stdout.String(), stderr.String())
|
|
}
|
|
return stdout.String()
|
|
}
|
|
|
|
// bdForgetFail runs "bd forget" expecting failure.
|
|
func bdForgetFail(t *testing.T, bd, dir string, args ...string) string {
|
|
t.Helper()
|
|
fullArgs := append([]string{"forget"}, args...)
|
|
cmd := exec.Command(bd, fullArgs...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err == nil {
|
|
t.Fatalf("expected bd forget %s to fail, but succeeded:\n%s", strings.Join(args, " "), out)
|
|
}
|
|
return string(out)
|
|
}
|
|
|
|
func TestEmbeddedMemory(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "tm")
|
|
|
|
// ===== Remember and Recall =====
|
|
|
|
t.Run("remember_and_recall", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "always run tests with -race flag")
|
|
// Recall using auto-generated key — list memories to find it
|
|
out := bdMemories(t, bd, dir)
|
|
if !strings.Contains(out, "always run tests with -race flag") {
|
|
t.Errorf("expected memory in list: %s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("remember_with_key", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "auth module uses JWT not sessions", "--key", "auth-jwt")
|
|
out := bdRecall(t, bd, dir, "auth-jwt")
|
|
if !strings.Contains(out, "JWT") {
|
|
t.Errorf("expected 'JWT' in recall output: %s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("remember_overwrite", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "first version", "--key", "overwrite-test")
|
|
bdRemember(t, bd, dir, "second version", "--key", "overwrite-test")
|
|
out := bdRecall(t, bd, dir, "overwrite-test")
|
|
if !strings.Contains(out, "second version") {
|
|
t.Errorf("expected 'second version' after overwrite: %s", out)
|
|
}
|
|
})
|
|
|
|
// Desire path: `bd remember <existing-key>` (treating remember as a getter) is a READ --
|
|
// it recalls the memory instead of overwriting it with its own key as content.
|
|
t.Run("remember_bare_existing_key_recalls", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "the real insight worth keeping", "--key", "guard-key")
|
|
// Fat-finger a "read": positional arg is the existing bare key. Succeeds as a recall.
|
|
out := bdRemember(t, bd, dir, "guard-key")
|
|
if !strings.Contains(out, "the real insight worth keeping") {
|
|
t.Errorf("expected bare existing key to recall the memory content, got: %s", out)
|
|
}
|
|
// The original content must be intact, not clobbered to "guard-key".
|
|
recalled := bdRecall(t, bd, dir, "guard-key")
|
|
if !strings.Contains(recalled, "the real insight worth keeping") {
|
|
t.Errorf("recall-instead-of-write must preserve original content, got: %s", recalled)
|
|
}
|
|
})
|
|
|
|
t.Run("remember_guard_bypass_with_explicit_key", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "original", "--key", "bypass-key")
|
|
// Explicit --key signals deliberate intent and bypasses the guard.
|
|
bdRemember(t, bd, dir, "bypass-key", "--key", "bypass-key")
|
|
out := bdRecall(t, bd, dir, "bypass-key")
|
|
if !strings.Contains(out, "bypass-key") {
|
|
t.Errorf("explicit --key should bypass guard and overwrite, got: %s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("remember_bare_slug_without_memory_refuses", func(t *testing.T) {
|
|
// A bare slug naming NO existing memory is a mistyped read (or junk content) --
|
|
// refuse rather than silently creating a memory whose content is its own key.
|
|
out := bdRememberFail(t, bd, dir, "brand-new-slug-memory")
|
|
if !strings.Contains(out, "memories") {
|
|
t.Errorf("expected refusal to point at `bd memories`, got: %s", out)
|
|
}
|
|
// Nothing may have been stored.
|
|
bdRecallFail(t, bd, dir, "brand-new-slug-memory")
|
|
})
|
|
|
|
// The two refusals that sit either side of the desire path, and the reason
|
|
// the bare-slug test is `derived != "" && derived == insight` rather than
|
|
// the shipped `slugify(insight) == insight`: DeriveKey("") is "", so empty
|
|
// and underivable content satisfies derived == insight and would be routed
|
|
// into a "recall" of the empty key instead of being refused. The shipped
|
|
// code was saved from that by an empty-content check that ran BEFORE the
|
|
// branch; that check belongs to the role now, so the branch has to exclude
|
|
// the empty key itself.
|
|
t.Run("remember_refuses_content_no_key_derives_from", func(t *testing.T) {
|
|
for _, tc := range []struct{ insight, want string }{
|
|
{"", "memory content cannot be empty"},
|
|
{" ", "memory content cannot be empty"},
|
|
{"!!!", "could not generate key from content"},
|
|
} {
|
|
out := bdRememberFail(t, bd, dir, tc.insight)
|
|
if !strings.Contains(out, tc.want) {
|
|
t.Errorf("bd remember %q: expected %q, got: %s", tc.insight, tc.want, out)
|
|
}
|
|
if strings.Contains(out, "a bare existing key READS") || strings.Contains(out, "no memory named") {
|
|
t.Errorf("bd remember %q took the bare-slug desire path, got: %s", tc.insight, out)
|
|
}
|
|
}
|
|
})
|
|
|
|
t.Run("remember_new_slug_with_explicit_key_stores", func(t *testing.T) {
|
|
// --key states write intent: slug-like content stores fine.
|
|
bdRemember(t, bd, dir, "brand-new-slug-memory", "--key", "brand-new-slug-memory")
|
|
out := bdRecall(t, bd, dir, "brand-new-slug-memory")
|
|
if !strings.Contains(out, "brand-new-slug-memory") {
|
|
t.Errorf("expected --key to store slug content, got: %s", out)
|
|
}
|
|
})
|
|
|
|
// ===== Memories List =====
|
|
|
|
t.Run("memories_list", func(t *testing.T) {
|
|
out := bdMemories(t, bd, dir)
|
|
// Should show at least the memories we stored above
|
|
if !strings.Contains(out, "auth-jwt") {
|
|
t.Errorf("expected 'auth-jwt' in memories list: %s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("memories_search", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "dolt phantom DBs hide in three places", "--key", "dolt-phantoms")
|
|
out := bdMemories(t, bd, dir, "dolt")
|
|
if !strings.Contains(out, "dolt") {
|
|
t.Errorf("expected 'dolt' related memory in search: %s", out)
|
|
}
|
|
})
|
|
|
|
t.Run("memories_search_no_match", func(t *testing.T) {
|
|
out := bdMemories(t, bd, dir, "zzz_nonexistent_term_xyz")
|
|
// Should succeed but show no results or empty
|
|
_ = out
|
|
})
|
|
|
|
// ===== Forget =====
|
|
|
|
t.Run("forget", func(t *testing.T) {
|
|
bdRemember(t, bd, dir, "temporary memory to forget", "--key", "forget-me")
|
|
out := bdRecall(t, bd, dir, "forget-me")
|
|
if !strings.Contains(out, "temporary memory") {
|
|
t.Fatalf("expected memory before forget: %s", out)
|
|
}
|
|
bdForget(t, bd, dir, "forget-me")
|
|
// After forget, recall should fail
|
|
bdRecallFail(t, bd, dir, "forget-me")
|
|
})
|
|
|
|
// AN EMPTY-VALUED MEMORY IS PRESENT (#5963). These run the real commands, so
|
|
// they pin each RunE's wiring and not only its renderer: a call site that
|
|
// went back to deciding presence from value != "" fails here while
|
|
// TestPrintRecallResult and TestRememberBareKeyPath stay green. Its own
|
|
// workspace, because the row is emptied beneath bd.
|
|
t.Run("empty_valued_memory_is_present", func(t *testing.T) {
|
|
edir, ebeads, _ := bdInit(t, bd, "--prefix", "em")
|
|
bdRemember(t, bd, edir, "placeholder the test empties", "--key", "stored-empty")
|
|
storeMemoryValueOutOfBand(t, ebeads, "stored-empty", "")
|
|
|
|
// The placeholder check proves bd sees the emptied row. Without it, a
|
|
// write bd never read would leave this test green on a reverted fix.
|
|
if out := bdRecall(t, bd, edir, "stored-empty"); strings.Contains(out, "placeholder") {
|
|
t.Fatalf("bd recall still sees the placeholder, so the row was not emptied: %s", out)
|
|
}
|
|
// A bare key naming the empty-valued memory recalls it instead of refusing.
|
|
if out := bdRemember(t, bd, edir, "stored-empty"); strings.Contains(out, "placeholder") {
|
|
t.Fatalf("bd remember <bare-key> still sees the placeholder: %s", out)
|
|
}
|
|
bdForget(t, bd, edir, "stored-empty")
|
|
bdRecallFail(t, bd, edir, "stored-empty")
|
|
})
|
|
|
|
// THE AUTO-COMMIT EPILOGUE, which is the trap this convergence was most
|
|
// likely to spring. A direct-route write lands in the Dolt WORKING SET and
|
|
// nothing else commits it: durability is the epilogue in main.go, keyed on
|
|
// commandDidWrite, which noteDirectMemoryWrite is the only thing that sets
|
|
// for this plane. A converged RunE that forgets it produces memories that
|
|
// exist until the process exits and then sit uncommitted — and every other
|
|
// assertion in this file still passes, because a working-set row recalls
|
|
// and lists exactly like a committed one.
|
|
//
|
|
// Its own workspace, so the counts are deterministic (the pattern
|
|
// close_multiple_ids_single_dolt_commit uses).
|
|
t.Run("write_verbs_flag_the_auto_commit_epilogue", func(t *testing.T) {
|
|
edir, ebeads, _ := bdInit(t, bd, "--prefix", "ep")
|
|
|
|
// Warm-up write: a fresh workspace's first command may commit
|
|
// bookkeeping of its own, which is not what this is measuring.
|
|
bdRemember(t, bd, edir, "warm the workspace up", "--key", "epilogue-warmup")
|
|
|
|
before := countMemoryDoltCommits(t, ebeads)
|
|
bdRemember(t, bd, edir, "the epilogue is what makes a direct write durable", "--key", "epilogue-subject")
|
|
afterRemember := countMemoryDoltCommits(t, ebeads)
|
|
if got := afterRemember - before; got != 1 {
|
|
t.Errorf("bd remember: dolt_log %d -> %d (delta %d), want exactly one auto-commit", before, afterRemember, got)
|
|
}
|
|
|
|
bdForget(t, bd, edir, "epilogue-subject")
|
|
afterForget := countMemoryDoltCommits(t, ebeads)
|
|
if got := afterForget - afterRemember; got != 1 {
|
|
t.Errorf("bd forget: dolt_log %d -> %d (delta %d), want exactly one auto-commit", afterRemember, afterForget, got)
|
|
}
|
|
|
|
// A forget that found nothing deleted nothing, so it must leave no
|
|
// commit either — the other half of the contract, and the one a
|
|
// `noteDirectMemoryWrite()` placed before the not-found branch would
|
|
// break while every output assertion stayed green.
|
|
bdForgetFail(t, bd, edir, "epilogue-subject")
|
|
if got := countMemoryDoltCommits(t, ebeads); got != afterForget {
|
|
t.Errorf("bd forget of an absent key: dolt_log %d -> %d, want no commit", afterForget, got)
|
|
}
|
|
})
|
|
|
|
// ===== Error Cases =====
|
|
|
|
t.Run("recall_missing_key", func(t *testing.T) {
|
|
bdRecallFail(t, bd, dir, "nonexistent-key-xyz")
|
|
})
|
|
|
|
t.Run("forget_missing_key", func(t *testing.T) {
|
|
bdForgetFail(t, bd, dir, "nonexistent-key-xyz")
|
|
})
|
|
|
|
t.Run("remember_no_args", func(t *testing.T) {
|
|
bdRememberFail(t, bd, dir)
|
|
})
|
|
|
|
t.Run("recall_no_args", func(t *testing.T) {
|
|
bdRecallFail(t, bd, dir)
|
|
})
|
|
|
|
t.Run("forget_no_args", func(t *testing.T) {
|
|
bdForgetFail(t, bd, dir)
|
|
})
|
|
}
|
|
|
|
// TestEmbeddedMemoryConcurrent exercises memory operations concurrently.
|
|
func TestEmbeddedMemoryConcurrent(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "mx")
|
|
|
|
// Disable auto-export: this test exercises concurrent memory
|
|
// flock contention, not export behavior. With export.auto=true
|
|
// (the default since GH#2973), 8 concurrent writers also trigger
|
|
// post-write read paths that race with in-flight commits.
|
|
//
|
|
// The underlying race is not flock-level (flock already serializes
|
|
// bd subprocesses) but engine-shutdown-level: Dolt's working-set
|
|
// persistence can lag behind flock release, so the next subprocess
|
|
// sometimes commits with a stale view and overwrites a prior forget.
|
|
// See GH#3260 for the investigation (PR #3269 was the CI probe).
|
|
// Fix would require synchronous flush on engine close or a long-lived
|
|
// engine per store; both are substantial work. Until then, this
|
|
// workaround keeps the test deterministic.
|
|
disableAutoExport := exec.Command(bd, "config", "set", "export.auto", "false")
|
|
disableAutoExport.Dir = dir
|
|
disableAutoExport.Env = bdEnv(dir)
|
|
if out, err := disableAutoExport.CombinedOutput(); err != nil {
|
|
t.Fatalf("disable export.auto: %v\n%s", err, out)
|
|
}
|
|
|
|
const numWorkers = 8
|
|
|
|
type workerResult struct {
|
|
worker int
|
|
err error
|
|
}
|
|
|
|
results := make([]workerResult, numWorkers)
|
|
var wg sync.WaitGroup
|
|
wg.Add(numWorkers)
|
|
|
|
for w := 0; w < numWorkers; w++ {
|
|
go func(worker int) {
|
|
defer wg.Done()
|
|
r := workerResult{worker: worker}
|
|
|
|
// Each worker stores memories with unique keys
|
|
for i := 0; i < 3; i++ {
|
|
key := fmt.Sprintf("w%d-mem%d", worker, i)
|
|
content := fmt.Sprintf("worker %d memory %d content", worker, i)
|
|
|
|
cmd := exec.Command(bd, "remember", content, "--key", key)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
r.err = fmt.Errorf("remember %s: %v\n%s", key, err, out)
|
|
results[worker] = r
|
|
return
|
|
}
|
|
}
|
|
|
|
// Recall each memory
|
|
for i := 0; i < 3; i++ {
|
|
key := fmt.Sprintf("w%d-mem%d", worker, i)
|
|
expected := fmt.Sprintf("worker %d memory %d content", worker, i)
|
|
|
|
cmd := exec.Command(bd, "recall", key)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
r.err = fmt.Errorf("recall %s: %v\n%s", key, err, out)
|
|
results[worker] = r
|
|
return
|
|
}
|
|
if !strings.Contains(string(out), expected) {
|
|
r.err = fmt.Errorf("recall %s: expected %q, got %q", key, expected, string(out))
|
|
results[worker] = r
|
|
return
|
|
}
|
|
}
|
|
|
|
// Forget one memory
|
|
forgetKey := fmt.Sprintf("w%d-mem0", worker)
|
|
cmd := exec.Command(bd, "forget", forgetKey)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
r.err = fmt.Errorf("forget %s: %v\n%s", forgetKey, err, out)
|
|
results[worker] = r
|
|
return
|
|
}
|
|
|
|
// List memories
|
|
cmd = exec.Command(bd, "memories")
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
out, err = cmd.CombinedOutput()
|
|
if err != nil {
|
|
r.err = fmt.Errorf("memories: %v\n%s", err, out)
|
|
results[worker] = r
|
|
return
|
|
}
|
|
|
|
results[worker] = r
|
|
}(w)
|
|
}
|
|
wg.Wait()
|
|
|
|
for _, r := range results {
|
|
if r.err != nil && !strings.Contains(r.err.Error(), "one writer at a time") {
|
|
t.Errorf("worker %d failed: %v", r.worker, r.err)
|
|
}
|
|
}
|
|
|
|
// Verify memories only for workers that succeeded (err==nil).
|
|
// With exclusive flock, some workers may fail with "one writer at a time".
|
|
out := bdMemories(t, bd, dir)
|
|
var successCount int
|
|
for _, r := range results {
|
|
if r.err != nil {
|
|
continue
|
|
}
|
|
successCount++
|
|
w := r.worker
|
|
forgottenKey := fmt.Sprintf("w%d-mem0", w)
|
|
if strings.Contains(out, forgottenKey) {
|
|
t.Errorf("expected %s to be forgotten", forgottenKey)
|
|
}
|
|
for i := 1; i < 3; i++ {
|
|
key := fmt.Sprintf("w%d-mem%d", w, i)
|
|
if !strings.Contains(out, key) {
|
|
t.Errorf("expected %s to still exist in memories", key)
|
|
}
|
|
}
|
|
}
|
|
if successCount == 0 {
|
|
t.Fatal("expected at least 1 worker to succeed")
|
|
}
|
|
}
|
|
|
|
// bdRememberBuffers runs "bd remember" and hands back stdout, stderr and the
|
|
// error separately. The budget's lines are STDERR by contract — that is what
|
|
// keeps them out of --json output — so a helper that folds the two streams
|
|
// together cannot test them.
|
|
func bdRememberBuffers(t *testing.T, bd, dir string, args ...string) (string, string, error) {
|
|
t.Helper()
|
|
cmd := exec.Command(bd, append([]string{"remember"}, args...)...)
|
|
cmd.Dir = dir
|
|
cmd.Env = bdEnv(dir)
|
|
stdout, stderr, err := runCommandBuffers(t, cmd)
|
|
return stdout.String(), stderr.String(), err
|
|
}
|
|
|
|
// TestEmbeddedMemoryCorpusBudget exercises memories.budget-chars against a REAL
|
|
// store, end to end: the config key routed through `bd config set`, the corpus
|
|
// measured from rows that are actually there, the refusal's nonzero exit, and
|
|
// the fact that a refused write leaves NOTHING behind.
|
|
//
|
|
// The arithmetic below is exact and deliberate — every content length is chosen
|
|
// so the projected corpus lands on a named boundary — because a budget test
|
|
// that only asserts "some line appeared" would pass with an off-by-one budget.
|
|
// The pure branch coverage lives in memory_budget_pure_test.go; this is the
|
|
// wiring.
|
|
func TestEmbeddedMemoryCorpusBudget(t *testing.T) {
|
|
if os.Getenv("BEADS_TEST_EMBEDDED_DOLT") != "1" {
|
|
t.Skip("set BEADS_TEST_EMBEDDED_DOLT=1 to run embedded dolt integration tests")
|
|
}
|
|
t.Parallel()
|
|
|
|
bd := buildEmbeddedBD(t)
|
|
|
|
// ===== Budget OFF: byte-identical to the pre-budget command =====
|
|
t.Run("budget_off_is_silent", func(t *testing.T) {
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "mb")
|
|
stdout, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("x", 5000), "--key", "big")
|
|
if err != nil {
|
|
t.Fatalf("remember with no budget failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
if !strings.Contains(stdout, "Remembered [big]") {
|
|
t.Errorf("expected the shipped success line, got %q", stdout)
|
|
}
|
|
if strings.Contains(stderr, "memory corpus") {
|
|
t.Errorf("budget off must print no budget line, got stderr %q", stderr)
|
|
}
|
|
})
|
|
|
|
// ===== Budget ON =====
|
|
dir, _, _ := bdInit(t, bd, "--prefix", "mb")
|
|
|
|
// The arithmetic below assumes a fresh, empty corpus. Say so out loud, so
|
|
// a future `bd init` that seeds a memory fails HERE instead of failing as
|
|
// an inscrutable off-by-N in the assertions.
|
|
if out := bdMemories(t, bd, dir); !strings.Contains(out, "No memories stored") {
|
|
t.Fatalf("budget arithmetic needs an empty corpus after init, got:\n%s", out)
|
|
}
|
|
|
|
cfg := exec.Command(bd, "config", "set", "memories.budget-chars", "100")
|
|
cfg.Dir = dir
|
|
cfg.Env = bdEnv(dir)
|
|
if out, err := cfg.CombinedOutput(); err != nil {
|
|
t.Fatalf("bd config set memories.budget-chars failed: %v\n%s", err, out)
|
|
} else if strings.Contains(string(out), "not a recognized") || strings.Contains(string(out), "Unrecognized") {
|
|
t.Fatalf("memories.budget-chars must be a recognized config key, got:\n%s", out)
|
|
}
|
|
|
|
// 3 ("pad") + 90 = 93 of 100 -> writes, warns at 93%.
|
|
t.Run("warn_band_writes_and_warns", func(t *testing.T) {
|
|
_, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("a", 90), "--key", "pad")
|
|
if err != nil {
|
|
t.Fatalf("a write inside the budget must succeed: %v\nstderr:\n%s", err, stderr)
|
|
}
|
|
if want := "bd remember: memory corpus at 93 of 100 chars (93%)"; !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want the warn line %q", stderr, want)
|
|
}
|
|
if out := bdRecall(t, bd, dir, "pad"); !strings.Contains(out, strings.Repeat("a", 90)) {
|
|
t.Errorf("the warned write must still have landed, got %q", out)
|
|
}
|
|
})
|
|
|
|
// An OVERWRITE of the same size is a no-op on the corpus (delta, not sum).
|
|
// Were it summed, the projection would be 186 and this would be refused.
|
|
t.Run("overwrite_counts_delta_not_sum", func(t *testing.T) {
|
|
_, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("b", 90), "--key", "pad")
|
|
if err != nil {
|
|
t.Fatalf("a same-size overwrite must not be refused: %v\nstderr:\n%s", err, stderr)
|
|
}
|
|
if want := "bd remember: memory corpus at 93 of 100 chars (93%)"; !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want %q (an overwrite must not add to the sum)", stderr, want)
|
|
}
|
|
if out := bdRecall(t, bd, dir, "pad"); !strings.Contains(out, strings.Repeat("b", 90)) {
|
|
t.Errorf("overwrite did not land: %q", out)
|
|
}
|
|
})
|
|
|
|
// 93 + 2 ("ab") + 5 = exactly 100. Exactly AT the budget writes.
|
|
t.Run("exactly_at_budget_writes", func(t *testing.T) {
|
|
_, stderr, err := bdRememberBuffers(t, bd, dir, "ccccc", "--key", "ab")
|
|
if err != nil {
|
|
t.Fatalf("a write landing exactly on the budget must succeed: %v\nstderr:\n%s", err, stderr)
|
|
}
|
|
if want := "bd remember: memory corpus at 100 of 100 chars (100%)"; !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want %q", stderr, want)
|
|
}
|
|
})
|
|
|
|
// Overwriting "pad" with one more byte projects 101 — exactly ONE over.
|
|
t.Run("one_byte_over_refuses_and_writes_nothing", func(t *testing.T) {
|
|
stdout, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("d", 91), "--key", "pad")
|
|
if err == nil {
|
|
t.Fatalf("a write one byte over the budget must exit nonzero; stdout:\n%s\nstderr:\n%s", stdout, stderr)
|
|
}
|
|
want := "bd remember: memory corpus would be 101 chars, budget is 100 (101%) — refused; use --force to override"
|
|
if !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want %q", stderr, want)
|
|
}
|
|
// A terminal user is offered the same ways out as the --json envelope.
|
|
if hint := "Hint: raise memories.budget-chars, forget a memory, or pass --force"; !strings.Contains(stderr, hint) {
|
|
t.Errorf("stderr = %q, want the hint %q", stderr, hint)
|
|
}
|
|
if strings.Contains(stdout, "Remembered") || strings.Contains(stdout, "Updated") {
|
|
t.Errorf("a refused write must print no success line, got stdout %q", stdout)
|
|
}
|
|
if out := bdRecall(t, bd, dir, "pad"); !strings.Contains(out, strings.Repeat("b", 90)) {
|
|
t.Errorf("a refused write must leave the old content intact, got %q", out)
|
|
}
|
|
})
|
|
|
|
// The commonest call passes no --key, so the write lands under the key
|
|
// derived from the content, and that is the key the budget must measure:
|
|
// 100 + 22 ("derived-keys-count-too") + 24 (the content) projects 146.
|
|
t.Run("derived_key_write_is_budgeted", func(t *testing.T) {
|
|
stdout, stderr, err := bdRememberBuffers(t, bd, dir, "Derived keys count, too!")
|
|
var exitErr *exec.ExitError
|
|
if !errors.As(err, &exitErr) || exitErr.ExitCode() != 1 {
|
|
t.Fatalf("a write with no --key over the budget must exit 1, got %v; stdout:\n%s\nstderr:\n%s", err, stdout, stderr)
|
|
}
|
|
want := "bd remember: memory corpus would be 146 chars, budget is 100 (146%) — refused; use --force to override"
|
|
if !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want %q", stderr, want)
|
|
}
|
|
if strings.Contains(stdout, "Remembered") || strings.Contains(stdout, "Updated") {
|
|
t.Errorf("a refused write must print no success line, got stdout %q", stdout)
|
|
}
|
|
bdRecallFail(t, bd, dir, "derived-keys-count-too")
|
|
})
|
|
|
|
// A whitespace-only --key names nothing, so the role refuses the write with
|
|
// its own validation sentence. The budget must not answer first: on this
|
|
// full corpus it would call the write over budget and point at --force,
|
|
// and the --force retry would only then meet the validation error.
|
|
t.Run("blank_key_gets_the_validation_error_not_the_budget", func(t *testing.T) {
|
|
stdout, stderr, err := bdRememberBuffers(t, bd, dir, "some content", "--key", " ")
|
|
if err == nil {
|
|
t.Fatalf("a whitespace-only --key must be refused; stdout:\n%s\nstderr:\n%s", stdout, stderr)
|
|
}
|
|
if !strings.Contains(stderr, "memory key must not be empty") {
|
|
t.Errorf("stderr = %q, want the role's validation error", stderr)
|
|
}
|
|
if strings.Contains(stderr, "memory corpus") {
|
|
t.Errorf("a write the role refuses must not be measured against the budget, got stderr %q", stderr)
|
|
}
|
|
})
|
|
|
|
// The same refusal under --json is said ONCE, and only on stdout: the
|
|
// machine-readable envelope, nothing else. Printing the prose to stderr as
|
|
// well would deliver one refusal twice to a caller reading both streams.
|
|
t.Run("json_refusal_is_said_once", func(t *testing.T) {
|
|
stdout, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("d", 91), "--key", "pad", "--json")
|
|
if err == nil {
|
|
t.Fatalf("a --json write one byte over the budget must exit nonzero; stdout:\n%s\nstderr:\n%s", stdout, stderr)
|
|
}
|
|
|
|
// Exactly ONE JSON document on stdout — no second envelope, no prose
|
|
// before or after it.
|
|
dec := json.NewDecoder(strings.NewReader(stdout))
|
|
var doc map[string]interface{}
|
|
if decErr := dec.Decode(&doc); decErr != nil {
|
|
t.Fatalf("stdout is not a JSON document (%v):\n%s", decErr, stdout)
|
|
}
|
|
if decErr := dec.Decode(new(json.RawMessage)); !errors.Is(decErr, io.EOF) {
|
|
t.Errorf("stdout carries more than one JSON document (second Decode = %v):\n%s", decErr, stdout)
|
|
}
|
|
|
|
// The envelope carries the refusal sentence, under either the enveloped
|
|
// or the flat error shape.
|
|
inner := doc
|
|
if data, ok := doc["data"].(map[string]interface{}); ok {
|
|
inner = data
|
|
}
|
|
msg, _ := inner["error"].(string)
|
|
want := "bd remember: memory corpus would be 101 chars, budget is 100 (101%) — refused; use --force to override"
|
|
if msg != want {
|
|
t.Errorf("JSON error = %q, want %q (full stdout:\n%s)", msg, want, stdout)
|
|
}
|
|
|
|
// ...and stderr says nothing, so the refusal arrived exactly once.
|
|
if strings.Contains(stderr, "memory corpus") || strings.Contains(stderr, "Hint:") {
|
|
t.Errorf("--json refusal must not also print the prose line to stderr, got stderr %q", stderr)
|
|
}
|
|
|
|
// A refused --json write still leaves the old content intact.
|
|
if out := bdRecall(t, bd, dir, "pad"); !strings.Contains(out, strings.Repeat("b", 90)) {
|
|
t.Errorf("a refused --json write must leave the old content intact, got %q", out)
|
|
}
|
|
})
|
|
|
|
// The same write with --force lands, and still says so.
|
|
t.Run("force_writes_over_budget", func(t *testing.T) {
|
|
_, stderr, err := bdRememberBuffers(t, bd, dir, strings.Repeat("d", 91), "--key", "pad", "--force")
|
|
if err != nil {
|
|
t.Fatalf("--force must write over the budget: %v\nstderr:\n%s", err, stderr)
|
|
}
|
|
if want := "bd remember: memory corpus at 101 of 100 chars (101%) — over budget, written anyway (--force)"; !strings.Contains(stderr, want) {
|
|
t.Errorf("stderr = %q, want %q", stderr, want)
|
|
}
|
|
if out := bdRecall(t, bd, dir, "pad"); !strings.Contains(out, strings.Repeat("d", 91)) {
|
|
t.Errorf("--force did not land the write, got %q", out)
|
|
}
|
|
})
|
|
}
|