`bd query "type=bug OR label=urgent"` has been answering with a prefix of its
answer and calling it complete. The expression language has OR, NOT and
parentheses, which no storage filter expresses, so both routes fetched a WINDOW
of candidate rows — max(3*limit, 100) — and applied the expression to those in
memory. A match past row 100 was absent from the page and unreported by
has-more. The same six lines of arithmetic were written twice, once per route,
and both copies were wrong in the same way.
THE ROLE. issueops.Querier, reached through store.Querier() and the provider's
own accessor. One method over a request whose predicate is a SENTENCE rather
than a set of fields, which is what makes it a role instead of a mode of
Reader: a ListRequest is a conjunction — every field narrows — and no
arrangement of its twelve callers' fields expresses a disjunction. It is
equally not a raw query passthrough; the language names no table, no join and
no column outside a closed vocabulary, which is why a contract can promise
anything about it at all.
THE FIX, which Q3 approved as user-visible. A predicate query's filter now
carries NO row limit: the predicate sees every candidate row, the page is the
first Limit MATCHES of the complete matching set, and has-more is exact. The
decision has one home, internal/workapi.BuildQueryPlan, which both bodies
execute. The cost is stated rather than hidden — a broad OR over a large
workspace now reads every row its base filters admit — and CHANGELOG.md says so
in the same words.
WHAT PINS THE FIX, at three levels, because the contract alone cannot see it.
TestBuildQueryPlanLeavesAPredicateQueryUNBOUNDED asserts the arithmetic in
microseconds. TestAPredicateQueryFindsAMatchBeyondTheOldWindow seeds 101 rows at
ONE backend and reads back the match at row 101 — verified to discriminate:
with the old window temporarily reinstated it returns zero rows. The portable
contract asserts the PROMISE the window broke (a page is a prefix, has-more is
exact) and, confirmed by the same experiment, does NOT discriminate the old
window at its small scale; the contract's header says so rather than implying
coverage it does not have.
THE CONTRACT. backend/conformance/querier_contract.go holds nine cases, each
citing the leaf doc by line, run at all three backends with ZERO parks. TWO
independent bodies plus an engine check: dolt and embeddeddolt share
internal/workapi/storequerier, and the unit-of-work provider is genuinely
separate — it is the only leg on which the Offset case takes its HONORED branch,
since that seam renders OFFSET where the store body refuses uniformly.
TWO SMALLER USER-VISIBLE CHANGES ride along, both filed in AMBIGUITIES.md
(Q22, Q23) for the owner's batch. `--offset` with an OR/NOT expression is no
longer refused: the offset skips MATCHES, which is only meaningful now that
every match is seen, so an error became an answer. `--limit -1` is now
ErrValidation instead of an undocumented second spelling of unlimited. The
`--offset` + `--sort` refusal survives and moved INWARD — the role makes it, for
every caller, because the order is applied to the rows the query bounded and
each page of an offset walk would be sorted for itself — so its wording changed
with its address, as did the wording of a malformed expression on both routes.
A DEFECT FOUND AND NOT FIXED, recorded as A-qry-1 rather than papered over: an
UNLIMITED request carrying an Offset renders SQL the Dolt engine answers with a
recovered panic (makeslice: cap out of range, out of topRowsIter). It predates
this role — `bd query --offset N --limit 0 --proxied-server` built the same
filter — and it is a storage-seam bug, so the contract case asks its question at
a bounded limit and says why at the case site.
THE HTTP OPERATION, spec first (per bd-jtp1h's A2 scope decision):
GET /v0/beads/issues:query, capability `issues.query`, answering a new QueryPage
{items, has_more}. It publishes no cursor and no `offset`: a predicate query's
matching set is assembled outside the database, so there is no keyset position
to encode, and an offset is honored by one database source and refused by the
other, which is not a difference to put on a wire. QueryPage carries no allOf
and its Items are pinned in pinning.go beside ReadyPage's and IssuesPage's — the
edit wave 2 missed once (bd-8ri3m). Adding a second sort enum made oapi-codegen
disambiguate the FIRST one, renaming the generated ListReadyWorkParamsSort
constants (Hybrid -> ListReadyWorkParamsSortHybrid); nothing outside the
generated file named them. httpapi.Config's roles source grows a required eighth
member, and a Config complete under the previous release is now refused at
Listen rather than nil-dereferencing on the first issues:query request.
WHAT ELSE CONVERGED. Both output modes read ONE query — the counted one --json
always used — so a predicate can no longer be applied to two different row sets
depending on the flag (A-qry-1's neighbour, Q24). The direct route prints the
truncation hint for the first time, not by new choice but because it finally has
an honest has-more to print.
Incidental: internal/storage/role_accessor_decorator_test.go's four
near-identical "the four read roles" paragraphs — one appended by each wave-2
slice — are collapsed into the one accurate paragraph the table needs. Same
defect as bd-8ri3m, comments only.
Verified: build, vet, gofmt clean; golangci-lint 0 issues; make api-check green;
TestQuerierContract 9/9 at dolt, embeddeddolt and uow;
TestAPredicateQueryFindsAMatchBeyondTheOldWindow; the internal/httpapi package
suite; internal/workapi, internal/storage, internal/telemetry, internal/query
and the root package; TestEmbeddedQuery under BEADS_TEST_EMBEDDED_DOLT=1 and
TestProxiedServerQuery under BEADS_TEST_PROXIED_SERVER=1; TestProxiedLookup*.
The protocol corpus drives no `bd query` invocation and is untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
210 lines
8.1 KiB
Go
210 lines
8.1 KiB
Go
package workapi
|
|
|
|
import (
|
|
"errors"
|
|
"slices"
|
|
"testing"
|
|
|
|
"github.com/steveyegge/beads/internal/types"
|
|
"github.com/steveyegge/beads/issueops"
|
|
)
|
|
|
|
func queryRequest(expression string, limit int) issueops.QueryRequest {
|
|
return issueops.QueryRequest{Expression: expression, Limit: &limit}
|
|
}
|
|
|
|
// TestBuildQueryPlanLeavesAPredicateQueryUNBOUNDED is the unit-level pin on the
|
|
// defect this role shipped to fix, and it is here rather than only in the
|
|
// conformance contract because it is the ARITHMETIC that was wrong, not the
|
|
// backend: both front doors set the row limit to max(3*Limit, 100) and filtered
|
|
// what came back, so an OR query over a workspace with more than a hundred
|
|
// candidate rows returned a prefix of its answer and reported it as whole.
|
|
//
|
|
// A predicate query's filter must carry NO row limit. There is no value it
|
|
// could carry that is safe: the predicate rejects an unknown fraction of the
|
|
// rows the database returns, so any bound on the fetch is a bound on the
|
|
// matches with no relationship to the page the caller asked for.
|
|
func TestBuildQueryPlanLeavesAPredicateQueryUNBOUNDED(t *testing.T) {
|
|
for _, expression := range []string{
|
|
"type=bug OR type=feature",
|
|
"NOT priority>2",
|
|
"(status=open OR status=blocked) AND priority<2",
|
|
"type=bug OR (label=urgent AND assignee=none)",
|
|
} {
|
|
t.Run(expression, func(t *testing.T) {
|
|
plan, err := BuildQueryPlan(queryRequest(expression, 5))
|
|
if err != nil {
|
|
t.Fatalf("BuildQueryPlan(%q): %v", expression, err)
|
|
}
|
|
if !plan.RequiresPredicate() {
|
|
t.Fatalf("%q was planned without a predicate; this case no longer covers the shape it exists for", expression)
|
|
}
|
|
if plan.Filter.Limit != 0 {
|
|
t.Errorf("predicate query's filter carries Limit=%d; a bounded fetch drops matches the predicate never saw",
|
|
plan.Filter.Limit)
|
|
}
|
|
if plan.Limit != 5 {
|
|
t.Errorf("plan.Limit = %d, want the 5 the caller asked to receive", plan.Limit)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestBuildQueryPlanPushesTheLimitDownWhenTheFilterIsExact is the other half:
|
|
// an expression the storage filter expresses exactly is bounded by the
|
|
// database, because there is no predicate to hide rows from it.
|
|
func TestBuildQueryPlanPushesTheLimitDownWhenTheFilterIsExact(t *testing.T) {
|
|
for _, expression := range []string{
|
|
"status=open",
|
|
"status=open AND priority<=2",
|
|
"NOT status=closed",
|
|
"label=frontend OR label=backend",
|
|
} {
|
|
t.Run(expression, func(t *testing.T) {
|
|
plan, err := BuildQueryPlan(queryRequest(expression, 7))
|
|
if err != nil {
|
|
t.Fatalf("BuildQueryPlan(%q): %v", expression, err)
|
|
}
|
|
if plan.RequiresPredicate() {
|
|
t.Fatalf("%q was planned WITH a predicate; this case no longer covers the shape it exists for", expression)
|
|
}
|
|
if plan.Filter.Limit != 7 {
|
|
t.Errorf("filter.Limit = %d, want the page limit pushed into the query", plan.Filter.Limit)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestBuildQueryPlanLimitDefaulting pins the pointer's three states, which is
|
|
// what lets one constant serve both surfaces.
|
|
func TestBuildQueryPlanLimitDefaulting(t *testing.T) {
|
|
plan, err := BuildQueryPlan(issueops.QueryRequest{Expression: "status=open"})
|
|
if err != nil {
|
|
t.Fatalf("BuildQueryPlan: %v", err)
|
|
}
|
|
if plan.Limit != DefaultQueryLimit {
|
|
t.Errorf("unset Limit = %d, want the shared default %d", plan.Limit, DefaultQueryLimit)
|
|
}
|
|
|
|
plan, err = BuildQueryPlan(queryRequest("status=open", 0))
|
|
if err != nil {
|
|
t.Fatalf("BuildQueryPlan: %v", err)
|
|
}
|
|
if plan.Limit != 0 || plan.Filter.Limit != 0 {
|
|
t.Errorf("explicit 0 = plan %d / filter %d, want unlimited on both", plan.Limit, plan.Filter.Limit)
|
|
}
|
|
|
|
if _, err := BuildQueryPlan(queryRequest("status=open", -1)); !errors.Is(err, issueops.ErrValidation) {
|
|
t.Errorf("negative Limit error = %v, want ErrValidation rather than a second spelling of unlimited", err)
|
|
}
|
|
}
|
|
|
|
// TestBuildQueryPlanExcludesClosedUnlessTheExpressionSaysOtherwise pins the
|
|
// conditional default (issueops/querier.go:29-41): an expression with an
|
|
// opinion about status keeps it, and only one without gets the exclusion.
|
|
func TestBuildQueryPlanExcludesClosedUnlessTheExpressionSaysOtherwise(t *testing.T) {
|
|
for _, test := range []struct {
|
|
expression string
|
|
includeClosed bool
|
|
wantExcluded bool
|
|
}{
|
|
{"priority=1", false, true},
|
|
{"priority=1", true, false},
|
|
{"status=closed", false, false},
|
|
{"NOT status=open", false, false},
|
|
{"type=bug OR status=closed", false, false},
|
|
{"type=bug OR type=feature", false, true},
|
|
} {
|
|
t.Run(test.expression, func(t *testing.T) {
|
|
plan, err := BuildQueryPlan(issueops.QueryRequest{
|
|
Expression: test.expression, IncludeClosed: test.includeClosed,
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("BuildQueryPlan(%q): %v", test.expression, err)
|
|
}
|
|
excluded := slices.Contains(plan.Filter.ExcludeStatus, types.StatusClosed)
|
|
if excluded != test.wantExcluded {
|
|
t.Errorf("closed excluded = %v, want %v (ExcludeStatus=%v, Status=%v)",
|
|
excluded, test.wantExcluded, plan.Filter.ExcludeStatus, plan.Filter.Status)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestBuildQueryPlanRefusals pins every deterministic refusal, each of which a
|
|
// front door would otherwise have to make for itself — which is how the two
|
|
// routes came to disagree about what an offset means.
|
|
func TestBuildQueryPlanRefusals(t *testing.T) {
|
|
for _, test := range []struct {
|
|
name string
|
|
req issueops.QueryRequest
|
|
}{
|
|
{"blank expression", issueops.QueryRequest{}},
|
|
{"whitespace expression", issueops.QueryRequest{Expression: " "}},
|
|
{"unparseable expression", issueops.QueryRequest{Expression: "===invalid==="}},
|
|
{"unknown field", issueops.QueryRequest{Expression: "nosuchfield=1"}},
|
|
{"negative offset", issueops.QueryRequest{Expression: "status=open", Offset: -1}},
|
|
{"offset with a display order", issueops.QueryRequest{Expression: "status=open", Offset: 1, SortBy: "priority"}},
|
|
} {
|
|
t.Run(test.name, func(t *testing.T) {
|
|
if _, err := BuildQueryPlan(test.req); !errors.Is(err, issueops.ErrValidation) {
|
|
t.Errorf("error = %v, want ErrValidation", err)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestAQueryExpressionRefusalNamesItselfForTheWire pins the message prefix
|
|
// internal/httpapi matches to turn an unparseable expression into a 400 on `q`
|
|
// rather than a 500. It is the same bounded prose dependency the filter
|
|
// builders already carry, written down here so a reword fails beside the
|
|
// sentence it rewords.
|
|
func TestAQueryExpressionRefusalNamesItselfForTheWire(t *testing.T) {
|
|
_, err := BuildQueryPlan(issueops.QueryRequest{Expression: "===invalid==="})
|
|
if err == nil {
|
|
t.Fatal("BuildQueryPlan accepted an unparseable expression")
|
|
}
|
|
const prefix = "invalid query expression"
|
|
if got := err.Error(); len(got) < len(prefix) || got[:len(prefix)] != prefix {
|
|
t.Errorf("error = %q, want it to start with %q (internal/httpapi maps that prefix to param `q`)", got, prefix)
|
|
}
|
|
}
|
|
|
|
// TestApplyQueryPredicateDropsRowsWithNoIssue pins the one thing the shared
|
|
// filter does beyond calling the predicate: a row carrying no issue is dropped
|
|
// rather than handed to a function that reads issue fields.
|
|
func TestApplyQueryPredicateDropsRowsWithNoIssue(t *testing.T) {
|
|
rows := []*types.IssueWithCounts{
|
|
{Issue: &types.Issue{ID: "bd-1"}},
|
|
nil,
|
|
{},
|
|
{Issue: &types.Issue{ID: "bd-2"}},
|
|
}
|
|
got := ApplyQueryPredicate(rows, func(i *types.Issue) bool { return i.ID != "bd-1" })
|
|
if len(got) != 1 || got[0].Issue.ID != "bd-2" {
|
|
t.Fatalf("kept %d rows, want only bd-2", len(got))
|
|
}
|
|
if same := ApplyQueryPredicate(rows, nil); len(same) != len(rows) {
|
|
t.Errorf("a nil predicate filtered %d rows down to %d; it must be the identity", len(rows), len(same))
|
|
}
|
|
}
|
|
|
|
// TestSkipRows pins the Offset half of a predicate query: it skips MATCHES, and
|
|
// skipping past the end is an empty page rather than a nil one.
|
|
func TestSkipRows(t *testing.T) {
|
|
rows := []*types.IssueWithCounts{
|
|
{Issue: &types.Issue{ID: "bd-1"}},
|
|
{Issue: &types.Issue{ID: "bd-2"}},
|
|
{Issue: &types.Issue{ID: "bd-3"}},
|
|
}
|
|
if got := SkipRows(rows, 0); len(got) != 3 {
|
|
t.Errorf("SkipRows(0) returned %d rows, want all three", len(got))
|
|
}
|
|
if got := SkipRows(rows, 2); len(got) != 1 || got[0].Issue.ID != "bd-3" {
|
|
t.Errorf("SkipRows(2) = %v, want bd-3 alone", got)
|
|
}
|
|
got := SkipRows(rows, 9)
|
|
if got == nil || len(got) != 0 {
|
|
t.Errorf("SkipRows past the end = %v, want an empty non-nil page", got)
|
|
}
|
|
}
|