fix(config): stop bd config set truncating config.yaml at a 64 KiB line (#7670)
* fix(config): stop bd config set truncating config.yaml at a 64 KiB line updateYamlKey's flat-key path reads config.yaml through a default bufio.Scanner and never checked scanner.Err(). A line of bufio.MaxScanTokenSize (65,536) bytes or more makes Scan return false partway through the document, so the function returned the lines read so far with a nil error and setYamlConfigAtPath wrote them over the file, dropping the long line and everything after it. Measured at5c9c02ab77: a 65,550-byte document came back as the 8-byte `k1: "zz"` when the key was present, and as 16 bytes when it was absent (the new key appended onto the truncated prefix). Check scanner.Err() after the loop and refuse with an error that wraps bufio.ErrTooLong. It is returned before setYamlConfigAtPath's os.WriteFile, so `bd config set` and `set-many` now fail and leave the file as it was. The guard returns only on a scan error, so a document whose lines all fit gets the same rewrite as before. Unset (commentOutYamlKey) already splits instead of scanning, and dotted keys rewrite through yaml.Node, so neither needs it. TestUpdateYamlKeyRejectsOversizeLine fails at the base (nil error, 8- and 16-byte results) and passes with the guard. Adds a CHANGELOG Fixed entry. AC: AC-1 Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(config): pin that a line just under the scanner limit still rewrites The oversize-line refusal must fire only at the bufio.Scanner cliff. The default scanner buffer holds bufio.MaxScanTokenSize (65,536) bytes, so a line of 65,535 bytes still fits with its newline, and updateYamlKey has to give a document with such a line the same rewrite a short document gets: the key replaced in place when present, appended when absent. Adds both rows to TestUpdateYamlKeyRejectsOversizeLine. They pass at the base and with the AC-1 guard, which returns only on a scan error. Two over-eager variants of the guard pass the existing yaml tests and the AC-1 rows but fail these rows: capping the scanner buffer at MaxScanTokenSize-1, and a pre-scan length check that refuses a line of MaxScanTokenSize-1 bytes. AC: AC-2 Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(config): pin that a refused bd config set leaves config.yaml unchanged setYamlConfigAtPath reads config.yaml, rewrites it through updateYamlKey, and writes the result back. The AC-1 guard makes updateYamlKey refuse a document with a line of bufio.MaxScanTokenSize bytes or more, and setYamlConfigAtPath returns that error before its os.WriteFile, so the file is never touched. No test covered that order. TestSetYamlConfigAtPathOversizeLineLeavesFileUntouched writes a config.yaml with a 65,538-byte comment line, sets no-db, and checks that the error wraps bufio.ErrTooLong and that the file still holds all 65,563 of its bytes. At5c9c02ab77it fails both checks: a nil error, and the file cut to the 11-byte `no-db: true`. It passes with the AC-1 guard and needs no code change of its own. A variant that writes the file before returning the error fails the byte check (0 bytes left), and one that swallows the error fails the error check. AC: AC-3 Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): say set-many keeps keys it wrote before a refused one The entry said `bd config set` and `set-many` both fail and leave config.yaml as it was. That holds for `set`, and for a `set-many` whose first config.yaml key is the refused one. But `set-many` writes its config.yaml pairs one at a time, in argument order, and returns on the first error with no rollback, so a key listed before the refused undotted key (a dotted `sync.remote`, say) is already on disk when the command exits 1. Measured at0d3aa63761with a 70,000-byte comment line in config.yaml: `bd config set-many sync.remote=git@example.com:o/r.git actor=bob` exits 1 and adds the `sync:` block (70,039 -> 70,083 bytes); `set-many actor=bob sync.remote=...` and `bd config set actor bob` exit 1 and leave the file byte-identical. de330c3e7f's message makes the same claim; this commit does not rewrite it. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Julian Knutsen Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Eddie the Engineer <julianknutsen@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Eddie the Engineer <338826946+gascity-eddie[bot]@users.noreply.github.com>
This commit is contained in:
parent
4c48829c07
commit
994f5b82dd
3 changed files with 96 additions and 0 deletions
|
|
@ -826,6 +826,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
those blank lines with the section, and setup appends a section after one
|
||||
blank line, so removing and re-adding it toggles between the same two files.
|
||||
A re-added section goes at the end of the file.
|
||||
- **`bd config set` no longer truncates a `config.yaml` that has a line of
|
||||
64 KiB or more.** Setting an undotted key (such as `actor` or `no-db`)
|
||||
rewrites the file line by line, and a line of 65,536 bytes or more ended
|
||||
that read early without an error, so the command reported success and wrote
|
||||
back only the lines before the long one, plus the key it set. Dotted keys
|
||||
and `bd config unset` were not affected. Setting an undotted key in such a
|
||||
file now fails with `bufio.Scanner: token too long`. `bd config set` leaves
|
||||
the file as it was. `set-many` stops at that key, but it has already written
|
||||
any `config.yaml` keys listed before it, such as a dotted `sync.remote`.
|
||||
- **`bd show --as-of` exits 1 when it reads no issue.** A ref the Dolt AS OF
|
||||
read does not take (an instant, a versioned-history address), a ref that
|
||||
does not exist, or an issue that did not exist at the ref was reported on
|
||||
|
|
|
|||
|
|
@ -913,6 +913,11 @@ func updateYamlKey(content, key, value string) (string, error) {
|
|||
result = append(result, line)
|
||||
}
|
||||
}
|
||||
// A line of bufio.MaxScanTokenSize bytes or more ends the scan early;
|
||||
// returning the lines read so far would truncate config.yaml on write.
|
||||
if err := scanner.Err(); err != nil {
|
||||
return "", fmt.Errorf("cannot set %q: scanning config content: %w", key, err)
|
||||
}
|
||||
|
||||
if !found {
|
||||
// Key not found - append at end
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
package config
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
|
|
@ -247,6 +249,86 @@ func walkMap(m map[string]interface{}, parts []string) interface{} {
|
|||
return cur
|
||||
}
|
||||
|
||||
// A line of bufio.MaxScanTokenSize bytes or more stops the flat-key path's
|
||||
// line scanner partway through the document. updateYamlKey used to return the
|
||||
// lines read up to that point with a nil error, which setYamlConfigAtPath then
|
||||
// wrote over config.yaml, dropping the long line and everything after it.
|
||||
// A line one byte shorter still fits, and must get the same rewrite a short
|
||||
// document gets rather than the refusal.
|
||||
func TestUpdateYamlKeyRejectsOversizeLine(t *testing.T) {
|
||||
doc := func(line string) string { return "k1: v1\n" + line + "\nk2: v2" }
|
||||
atLimit := strings.Repeat("x", bufio.MaxScanTokenSize)
|
||||
underLimit := strings.Repeat("x", bufio.MaxScanTokenSize-1)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
content string
|
||||
key string
|
||||
want string
|
||||
wantErr bool
|
||||
}{
|
||||
{name: "line of MaxScanTokenSize bytes, key present", content: doc(atLimit), key: "k1", wantErr: true},
|
||||
{name: "line of MaxScanTokenSize bytes, key absent", content: doc(atLimit), key: "k3", wantErr: true},
|
||||
{
|
||||
name: "line of MaxScanTokenSize-1 bytes, key present",
|
||||
content: doc(underLimit),
|
||||
key: "k1",
|
||||
want: "k1: \"zz\"\n" + underLimit + "\nk2: v2",
|
||||
},
|
||||
{
|
||||
name: "line of MaxScanTokenSize-1 bytes, key absent",
|
||||
content: doc(underLimit),
|
||||
key: "k3",
|
||||
want: doc(underLimit) + "\n\nk3: \"zz\"",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := updateYamlKey(tt.content, tt.key, "zz")
|
||||
if tt.wantErr {
|
||||
if !errors.Is(err, bufio.ErrTooLong) {
|
||||
t.Errorf("updateYamlKey() error = %v, want %v", err, bufio.ErrTooLong)
|
||||
}
|
||||
if got != "" {
|
||||
t.Errorf("updateYamlKey() returned %d bytes of the %d-byte document, want none", len(got), len(tt.content))
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("updateYamlKey() error = %v", err)
|
||||
}
|
||||
if got != tt.want {
|
||||
t.Errorf("updateYamlKey() returned %d bytes, want the %d-byte rewrite", len(got), len(tt.want))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// setYamlConfigAtPath is the read, rewrite, write step behind bd config set for
|
||||
// an undotted key. When updateYamlKey refuses a document with an oversize line,
|
||||
// the error has to come back before the write, so config.yaml keeps every byte
|
||||
// it had.
|
||||
func TestSetYamlConfigAtPathOversizeLineLeavesFileUntouched(t *testing.T) {
|
||||
configPath := filepath.Join(t.TempDir(), "config.yaml")
|
||||
before := "no-db: false\n# " + strings.Repeat("x", bufio.MaxScanTokenSize) + "\njson: true\n"
|
||||
if err := os.WriteFile(configPath, []byte(before), 0o600); err != nil {
|
||||
t.Fatalf("failed to write config.yaml: %v", err)
|
||||
}
|
||||
|
||||
if err := setYamlConfigAtPath(configPath, "no-db", "true"); !errors.Is(err, bufio.ErrTooLong) {
|
||||
t.Errorf("setYamlConfigAtPath() error = %v, want %v", err, bufio.ErrTooLong)
|
||||
}
|
||||
|
||||
after, err := os.ReadFile(configPath)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to read config.yaml: %v", err)
|
||||
}
|
||||
if string(after) != before {
|
||||
t.Errorf("config.yaml is %d bytes after the refused set, want the original %d bytes unchanged", len(after), len(before))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatYamlValue(t *testing.T) {
|
||||
tests := []struct {
|
||||
value string
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue