feat(reclaim): scope filters --label/--label-any/--exclude-label/--assignee/--id (wy-jpd3.3)

bd reclaim was global-only: on a federated deployment one machine's reaper
can revert a stale-looking unit that is alive on the machine that granted its
lease. Add scope filters mirroring the claim-side label surface
(bd ready --claim) so a supervisor can point its reaper at exactly the
partition it claims from.

- types.ReclaimFilter (IDs/Assignees/Labels/LabelsAny/ExcludeLabels) + IsEmpty.
- sqlbuild.BuildReclaimScopeClauses/ReclaimScopeSQL: same SQL shapes the
  claim/ready path emits, alias-qualified for the reclaim JOIN. Fails closed
  (1=0) on a degenerate all-empty field so a scoped sweep can never silently
  widen to global.
- issueops.ReclaimExpiredLeasesInTx threads the filter into the snapshot SELECT
  only; the per-row revert re-checks by id from the already-scoped snapshot.
- Threaded ReclaimExpiredLeases through the Store/domain interfaces and every
  backend (dolt, embedded dolt, sql repo, use case) and both CLI paths
  (direct + proxied server). A zero ReclaimFilter reproduces the exact prior
  global behavior.
- CLI hard-errors a scope flag supplied with no usable value (e.g. --label
  "$LANE" with LANE unset) instead of degrading to a global sweep; --json now
  reports a "scoped" boolean.
- Cross-backend conformance coverage (testReclaimScoped) + sqlbuild and
  flag-parsing unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Steve Yegge 2026-07-24 05:31:23 -07:00
commit c38582b7e6
20 changed files with 582 additions and 36 deletions

View file

@ -7,8 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **`bd reclaim` scope filters** (wy-jpd3.3). `bd reclaim` gained
`--label` / `--label-any` / `--exclude-label` / `--assignee` / `--id`,
mirroring the claim-side label surface (`bd ready --claim`), so a supervisor
can scope its reaper to exactly the partition it claims from. Filters
AND-combine and never widen the stale set. This matters on a federated
deployment: each replica's view of another machine's liveness is stale by up
to one sync interval, so an unscoped reaper on one machine can revert a unit
that is alive on the machine that granted its lease — point each supervisor's
reclaim at its own claim partition and that cannot happen. A scope flag
supplied with no usable value (e.g. `--label "$LANE"` with `LANE` unset) is a
hard error rather than a silent degrade into a global sweep, and the builder
fails closed (`1=0`) for programmatic callers. `bd reclaim --json` now reports
a `scoped` boolean so a reclaim log distinguishes a scoped sweep from a global
one.
### Changed
- **`beads.BulkIssueStore.ReclaimExpiredLeases` gained a `types.ReclaimFilter`
parameter** (wy-jpd3.3), threaded through the domain use-case/repository
interfaces and every backend. Callers that reclaim globally pass the zero
`types.ReclaimFilter{}` (unchanged behavior); any external type that
*implements* the interface must update its signature to compile.
- **Public `beads.Storage` interface gained two required methods**
([#4911](https://github.com/gastownhall/beads/pull/4911)). `UpdateIssueChecked`
(an optional `ExpectedVersion` compare-and-swap on updates) and `MergeMetadata`

View file

@ -2,8 +2,10 @@ package main
import (
"fmt"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/steveyegge/beads/internal/metrics"
"github.com/steveyegge/beads/internal/storage/issueops"
@ -29,10 +31,24 @@ least this long ago are reclaimed, so a worker briefly paused (GC, clock skew)
is not robbed of live work. Run it from a supervisor on a timer with a window
of roughly 2× the claim TTL.
By default reclaim is global: every stale lease in the database is fair game.
The scope filters below narrow it, using the same label surface claiming is
scoped by (--label / --label-any / --exclude-label), plus --assignee and --id.
Scoping matters on a federated deployment: each replica's view of the other
machine's liveness is stale by up to one sync interval, so an unscoped reaper on
one machine can revert a unit that is alive on the machine that granted its
lease. Point each supervisor's reclaim at its own claim partition and that
cannot happen. Filters AND-combine and never widen the set: a reclaimed lease
must still be stale.
Examples:
bd reclaim # default grace window (2× the lease TTL)
bd reclaim --older-than 10m # reclaim leases expired >10m ago
bd reclaim --older-than 0s # reclaim every currently-expired lease`,
bd reclaim --older-than 0s # reclaim every currently-expired lease
bd reclaim --label lane-a # only this machine's claim partition
bd reclaim --label-any lane-a,lane-b --exclude-label pinned
bd reclaim --assignee zelda --assignee epona # only these workers' leases
bd reclaim --id wy-abc --id wy-def # exactly these issues`,
Args: cobra.NoArgs,
SilenceUsage: true,
SilenceErrors: true,
@ -49,14 +65,19 @@ Examples:
return HandleErrorRespectJSON("--older-than must not be negative")
}
filter, err := reclaimFilterFromFlags(cmd)
if err != nil {
return HandleErrorRespectJSON("%v", err)
}
CheckReadonly("reclaim")
if usesProxiedServer() {
return runReclaimProxiedServer(rootCtx, olderThan)
return runReclaimProxiedServer(rootCtx, olderThan, filter)
}
ctx := rootCtx
reclaimed, err := store.ReclaimExpiredLeases(ctx, olderThan, actor)
reclaimed, err := store.ReclaimExpiredLeases(ctx, olderThan, filter, actor)
if err != nil {
return HandleErrorRespectJSON("reclaim: %v", err)
}
@ -72,18 +93,25 @@ Examples:
return HandleErrorRespectJSON("failed to commit: %v", err)
}
return renderReclaim(reclaimed)
return renderReclaim(reclaimed, !filter.IsEmpty())
},
}
func renderReclaim(reclaimed []types.ReclaimedLease) error {
func renderReclaim(reclaimed []types.ReclaimedLease, scoped bool) error {
if jsonOutput {
return outputJSON(map[string]interface{}{
"reclaimed": reclaimed,
"count": len(reclaimed),
// Whether any scope filter was in effect, so a supervisor auditing
// its own reclaim log can tell a scoped sweep from a global one.
"scoped": scoped,
})
}
if len(reclaimed) == 0 {
if scoped {
fmt.Printf("%s No stale leases to reclaim in the filtered scope\n", ui.RenderPass("✓"))
return nil
}
fmt.Printf("%s No stale leases to reclaim\n", ui.RenderPass("✓"))
return nil
}
@ -98,8 +126,70 @@ func renderReclaim(reclaimed []types.ReclaimedLease) error {
return nil
}
// registerReclaimScopeFlags declares the scope filters — the same label surface
// as the claim side (bd ready --claim), so a fleet can reclaim exactly the
// partition it claims from. Split out of init() so tests can exercise the real
// flag set on a fresh command instead of a hand-copied one.
func registerReclaimScopeFlags(fs *pflag.FlagSet) {
fs.StringSlice("id", nil, "Only reclaim these issue IDs (repeatable)")
fs.StringSliceP("assignee", "a", nil, "Only reclaim leases held by these assignees (repeatable)")
fs.StringSliceP("label", "l", nil, "Only reclaim issues with ALL these labels (AND). Can combine with --label-any")
fs.StringSlice("label-any", nil, "Only reclaim issues with AT LEAST ONE of these labels (OR). Can combine with --label")
fs.StringSlice("exclude-label", nil, "Never reclaim issues carrying ANY of these labels")
}
// reclaimFilterFromFlags maps the scope flags onto a types.ReclaimFilter,
// rejecting a flag that was SUPPLIED but carries no usable value.
//
// The hard error is the point. `bd reclaim --label "$LANE"` with LANE unset
// parses to an empty slice, which would otherwise be indistinguishable from
// "no --label at all" — i.e. a supervisor's scoped sweep silently degrading
// into a global one that reaps every stale lease in a federated database. A
// scope flag that resolves to nothing is operator error, not a wildcard.
func reclaimFilterFromFlags(cmd *cobra.Command) (types.ReclaimFilter, error) {
get := func(name string) ([]string, error) {
values, err := cmd.Flags().GetStringSlice(name)
if err != nil {
return nil, fmt.Errorf("--%s: %w", name, err)
}
if !cmd.Flags().Changed(name) {
return nil, nil
}
var kept []string
for _, v := range values {
if strings.TrimSpace(v) != "" {
kept = append(kept, v)
}
}
if len(kept) == 0 {
return nil, fmt.Errorf("--%s was given no usable value (an empty scope flag would reclaim everything; drop the flag to sweep globally)", name)
}
return kept, nil
}
var filter types.ReclaimFilter
var err error
if filter.IDs, err = get("id"); err != nil {
return types.ReclaimFilter{}, err
}
if filter.Assignees, err = get("assignee"); err != nil {
return types.ReclaimFilter{}, err
}
if filter.Labels, err = get("label"); err != nil {
return types.ReclaimFilter{}, err
}
if filter.LabelsAny, err = get("label-any"); err != nil {
return types.ReclaimFilter{}, err
}
if filter.ExcludeLabels, err = get("exclude-label"); err != nil {
return types.ReclaimFilter{}, err
}
return filter, nil
}
func init() {
reclaimCmd.Flags().Duration("older-than", 2*issueops.DefaultLeaseTTL,
"Only reclaim leases that expired at least this long ago (grace window)")
registerReclaimScopeFlags(reclaimCmd.Flags())
rootCmd.AddCommand(reclaimCmd)
}

82
cmd/bd/reclaim_test.go Normal file
View file

@ -0,0 +1,82 @@
package main
import (
"testing"
"github.com/spf13/cobra"
)
// newReclaimTestCmd builds a command with the real reclaim scope flag set so the
// parser under test is exactly the one production uses (registerReclaimScopeFlags
// is shared with init()).
func newReclaimTestCmd() *cobra.Command {
cmd := &cobra.Command{Use: "reclaim", RunE: func(*cobra.Command, []string) error { return nil }}
registerReclaimScopeFlags(cmd.Flags())
return cmd
}
func TestReclaimFilterFromFlagsUnset(t *testing.T) {
cmd := newReclaimTestCmd()
if err := cmd.ParseFlags(nil); err != nil {
t.Fatal(err)
}
filter, err := reclaimFilterFromFlags(cmd)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !filter.IsEmpty() {
t.Fatalf("no flags => filter %+v, want empty (global sweep)", filter)
}
}
func TestReclaimFilterFromFlagsPopulated(t *testing.T) {
cmd := newReclaimTestCmd()
args := []string{
"--id", "wy-a", "--id", "wy-b",
"--assignee", "zelda",
"--label", "lane-a",
"--label-any", "x,y",
"--exclude-label", "pinned",
}
if err := cmd.ParseFlags(args); err != nil {
t.Fatal(err)
}
filter, err := reclaimFilterFromFlags(cmd)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if filter.IsEmpty() {
t.Fatal("populated flags => empty filter")
}
if len(filter.IDs) != 2 || filter.IDs[0] != "wy-a" || filter.IDs[1] != "wy-b" {
t.Errorf("IDs = %v", filter.IDs)
}
if len(filter.Assignees) != 1 || filter.Assignees[0] != "zelda" {
t.Errorf("Assignees = %v", filter.Assignees)
}
if len(filter.Labels) != 1 || filter.Labels[0] != "lane-a" {
t.Errorf("Labels = %v", filter.Labels)
}
if len(filter.LabelsAny) != 2 {
t.Errorf("LabelsAny = %v", filter.LabelsAny)
}
if len(filter.ExcludeLabels) != 1 || filter.ExcludeLabels[0] != "pinned" {
t.Errorf("ExcludeLabels = %v", filter.ExcludeLabels)
}
}
// A scope flag that parses to no usable value is operator error: it must be a
// hard error, never a silent degrade into a global sweep.
func TestReclaimFilterFromFlagsRejectsEmptyValue(t *testing.T) {
for _, flag := range []string{"--label", "--label-any", "--assignee", "--id", "--exclude-label"} {
t.Run(flag, func(t *testing.T) {
cmd := newReclaimTestCmd()
if err := cmd.ParseFlags([]string{flag, ""}); err != nil {
t.Fatal(err)
}
if _, err := reclaimFilterFromFlags(cmd); err == nil {
t.Fatalf("%s '' => nil error, want a hard error", flag)
}
})
}
}

View file

@ -90,13 +90,13 @@ func runUnclaimProxiedServer(ctx context.Context, args []string, reason string,
return nil
}
func runReclaimProxiedServer(ctx context.Context, olderThan time.Duration) error {
func runReclaimProxiedServer(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter) error {
if uowProvider == nil {
return HandleError("proxied-server UOW provider not initialized")
}
reclaimed, err := uow.RunTxResult(ctx, uowProvider, func(ctx context.Context, uw uow.UnitOfWork) ([]types.ReclaimedLease, string, error) {
out, rerr := uw.IssueUseCase().ReclaimExpiredLeases(ctx, olderThan, actor)
out, rerr := uw.IssueUseCase().ReclaimExpiredLeases(ctx, olderThan, filter, actor)
if rerr != nil {
return nil, "", rerr
}
@ -117,5 +117,5 @@ func runReclaimProxiedServer(ctx context.Context, olderThan time.Duration) error
commandDidWrite.Store(true)
}
return renderReclaim(reclaimed)
return renderReclaim(reclaimed, !filter.IsEmpty())
}

View file

@ -21,8 +21,10 @@ type BulkIssueStore interface {
HeartbeatIssue(ctx context.Context, id, actor string) error
// ReclaimExpiredLeases reverts in_progress issues whose lease expired more
// than olderThan ago back to ready (clearing the assignee), recovering work
// stranded by dead workers. Returns the issues it reclaimed.
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error)
// stranded by dead workers. filter scopes which stale leases are eligible
// (the zero ReclaimFilter reclaims globally, the historical behavior).
// Returns the issues it reclaimed.
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error)
PromoteFromEphemeral(ctx context.Context, id string, actor string) error
GetNextChildID(ctx context.Context, parentID string) (string, error)
}

View file

@ -217,7 +217,7 @@ func testReclaimExpiredLease(t *testing.T, f Factory) {
// cutoff (now - olderThan) into the future, so even the fresh lease counts as
// expired. This exercises the revert path deterministically with no wall-clock
// wait; ReclaimSkipsFreshLease (olderThan 0) pins the other side of the cutoff.
reclaimed, err := s.ReclaimExpiredLeases(ctx(), -time.Hour, "reaper")
reclaimed, err := s.ReclaimExpiredLeases(ctx(), -time.Hour, types.ReclaimFilter{}, "reaper")
must(t, err)
found := false
for _, r := range reclaimed {
@ -254,7 +254,7 @@ func testReclaimSkipsFreshLease(t *testing.T, f Factory) {
must(t, s.CreateIssue(ctx(), withDefaults(&types.Issue{ID: "rc-2", Title: "T"}), "a"))
must(t, s.ClaimIssue(ctx(), "rc-2", "liveworker")) // fresh default-TTL lease
reclaimed, err := s.ReclaimExpiredLeases(ctx(), 0, "reaper")
reclaimed, err := s.ReclaimExpiredLeases(ctx(), 0, types.ReclaimFilter{}, "reaper")
must(t, err)
for _, r := range reclaimed {
if r.ID == "rc-2" {

View file

@ -155,6 +155,7 @@ func RunAll(t *testing.T, factory Factory) {
t.Run("HeartbeatWisp", func(t *testing.T) { testHeartbeatWisp(t, factory) })
t.Run("ReclaimExpiredLease", func(t *testing.T) { testReclaimExpiredLease(t, factory) })
t.Run("ReclaimSkipsFreshLease", func(t *testing.T) { testReclaimSkipsFreshLease(t, factory) })
t.Run("ReclaimScoped", func(t *testing.T) { testReclaimScoped(t, factory) })
t.Run("UnclaimIfAssigneeMatch", func(t *testing.T) { testUnclaimIfAssigneeMatch(t, factory) })
t.Run("UnclaimIfAssigneeStale", func(t *testing.T) { testUnclaimIfAssigneeStale(t, factory) })

View file

@ -0,0 +1,111 @@
package conformance
import (
"sort"
"testing"
"time"
"github.com/steveyegge/beads/internal/types"
)
// testReclaimScoped: bd reclaim honors the scope filters (--label / --label-any /
// --exclude-label / --assignee / --id), reverting ONLY the stale leases the
// filter selects and leaving every other stale lease in_progress. The filters
// never widen the set — an unmatched scope reclaims nothing even though a global
// reclaim would grab everything — which is the property a federated deployment
// partitions reclaim on. Runs identically on Dolt and SQLite because both route
// through issueops.ReclaimExpiredLeasesInTx + sqlbuild.ReclaimScopeSQL.
//
// The whole test runs one reaper cutoff (-time.Hour), so every fresh lease is
// already "expired": scoping, not staleness, decides what each call reclaims.
func testReclaimScoped(t *testing.T, f Factory) {
s := f(t)
// Four claimed, all-stale issues across two lanes and three workers.
must(t, s.CreateIssue(ctx(), withDefaults(&types.Issue{ID: "rs-a1", Title: "lane-a"}), "a"))
must(t, s.CreateIssue(ctx(), withDefaults(&types.Issue{ID: "rs-a2", Title: "lane-a opus"}), "a"))
must(t, s.CreateIssue(ctx(), withDefaults(&types.Issue{ID: "rs-b1", Title: "lane-b"}), "a"))
must(t, s.CreateIssue(ctx(), withDefaults(&types.Issue{ID: "rs-c1", Title: "no lane"}), "a"))
must(t, s.AddLabel(ctx(), "rs-a1", "lane-a", "a"))
must(t, s.AddLabel(ctx(), "rs-a2", "lane-a", "a"))
must(t, s.AddLabel(ctx(), "rs-a2", "tier:opus", "a"))
must(t, s.AddLabel(ctx(), "rs-b1", "lane-b", "a"))
must(t, s.ClaimIssue(ctx(), "rs-a1", "wa"))
must(t, s.ClaimIssue(ctx(), "rs-a2", "wa"))
must(t, s.ClaimIssue(ctx(), "rs-b1", "wb"))
must(t, s.ClaimIssue(ctx(), "rs-c1", "wc"))
// reclaim runs a scoped reaper and returns the sorted set of reclaimed IDs.
reclaim := func(filter types.ReclaimFilter) []string {
t.Helper()
got, err := s.ReclaimExpiredLeases(ctx(), -time.Hour, filter, "reaper")
must(t, err)
ids := make([]string, 0, len(got))
for _, r := range got {
ids = append(ids, r.ID)
}
sort.Strings(ids)
return ids
}
inProgress := func(id string) bool {
t.Helper()
got, err := s.GetIssue(ctx(), id)
must(t, err)
return got.Status == types.StatusInProgress
}
eq := func(got, want []string) bool {
if len(got) != len(want) {
return false
}
for i := range got {
if got[i] != want[i] {
return false
}
}
return true
}
// --label (AND-set): only lane-b's stale lease is reverted; the lane-a and
// unlabeled claims are untouched even though a global reaper would take them.
if got := reclaim(types.ReclaimFilter{Labels: []string{"lane-b"}}); !eq(got, []string{"rs-b1"}) {
t.Fatalf("reclaim(--label lane-b) = %v, want [rs-b1]", got)
}
if !inProgress("rs-a1") || !inProgress("rs-a2") || !inProgress("rs-c1") {
t.Fatalf("label-scoped reclaim disturbed out-of-scope claims")
}
// --assignee: only worker wc's stale lease.
if got := reclaim(types.ReclaimFilter{Assignees: []string{"wc"}}); !eq(got, []string{"rs-c1"}) {
t.Fatalf("reclaim(--assignee wc) = %v, want [rs-c1]", got)
}
if !inProgress("rs-a1") || !inProgress("rs-a2") {
t.Fatalf("assignee-scoped reclaim disturbed out-of-scope claims")
}
// --id: exactly the named issue, not its lane-mate rs-a2.
if got := reclaim(types.ReclaimFilter{IDs: []string{"rs-a1"}}); !eq(got, []string{"rs-a1"}) {
t.Fatalf("reclaim(--id rs-a1) = %v, want [rs-a1]", got)
}
if !inProgress("rs-a2") {
t.Fatalf("id-scoped reclaim disturbed rs-a2")
}
// --exclude-label: rs-a2 is the only stale lease left, and it carries the
// excluded label, so the reaper reverts nothing (fail-closed on no match).
if got := reclaim(types.ReclaimFilter{ExcludeLabels: []string{"tier:opus"}}); len(got) != 0 {
t.Fatalf("reclaim(--exclude-label tier:opus) = %v, want []", got)
}
if !inProgress("rs-a2") {
t.Fatalf("exclude-label reclaim wrongly reverted the excluded rs-a2")
}
// --label-any: OR-set matches rs-a2 via lane-a; the previous owner is reported.
got, err := s.ReclaimExpiredLeases(ctx(), -time.Hour, types.ReclaimFilter{LabelsAny: []string{"lane-a", "lane-z"}}, "reaper")
must(t, err)
if len(got) != 1 || got[0].ID != "rs-a2" || got[0].PreviousOwner != "wa" {
t.Fatalf("reclaim(--label-any lane-a,lane-z) = %+v, want [rs-a2 held by wa]", got)
}
if inProgress("rs-a2") {
t.Fatalf("rs-a2 should be reverted to open after its lane was reclaimed")
}
}

View file

@ -381,12 +381,12 @@ func (s *DoltStore) HeartbeatIssue(ctx context.Context, id, actor string) error
// olderThan ago back to ready, recovering work stranded by dead workers. The
// reclaim rewrites row_lock so it conflicts with any racing heartbeat/close on
// the same row; withRetryTx replays the loser. Returns the reclaimed issues.
func (s *DoltStore) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error) {
func (s *DoltStore) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error) {
cutoff := time.Now().UTC().Add(-olderThan)
var reclaimed []types.ReclaimedLease
err := s.withRetryTx(ctx, func(tx *sql.Tx) error {
var err error
reclaimed, err = issueops.ReclaimExpiredLeasesInTx(ctx, tx, cutoff, actor)
reclaimed, err = issueops.ReclaimExpiredLeasesInTx(ctx, tx, cutoff, filter, actor)
if err != nil {
return err
}

View file

@ -9,6 +9,7 @@ import (
"github.com/steveyegge/beads/internal/storage/doltutil"
"github.com/steveyegge/beads/internal/storage/issueops"
"github.com/steveyegge/beads/internal/storage/schema"
"github.com/steveyegge/beads/internal/types"
)
// preLeaseMoveVersion is the last schema version where leases were columns on
@ -144,7 +145,7 @@ func TestMigration0055MovesLiveLeases(t *testing.T) {
}
// The dead worker's claim is reclaimable through the normal path.
reclaimed, err := issueops.ReclaimExpiredLeasesInTx(ctx, conn, time.Now().UTC(), "reaper")
reclaimed, err := issueops.ReclaimExpiredLeasesInTx(ctx, conn, time.Now().UTC(), types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim after migration: %v", err)
}

View file

@ -283,7 +283,7 @@ func TestBareUpdateClaimDoesNotArmLease(t *testing.T) {
// Even a reclaim sweep with its cutoff pushed into the future (which would
// reap ANY leased issue) must leave the interactive claim alone: durable
// until the actor releases it or a human reclaims.
reclaimed, err := store.ReclaimExpiredLeases(ctx, -time.Hour, "reaper")
reclaimed, err := store.ReclaimExpiredLeases(ctx, -time.Hour, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim sweep: %v", err)
}
@ -348,7 +348,7 @@ func TestReclaimRevertsExpiredOnly(t *testing.T) {
time.Sleep(1500 * time.Millisecond) // let dead's lease expire
// Grace window larger than how long the lease has been expired: nothing yet.
reclaimed, err := store.ReclaimExpiredLeases(ctx, time.Hour, "reaper")
reclaimed, err := store.ReclaimExpiredLeases(ctx, time.Hour, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim with big grace: %v", err)
}
@ -357,7 +357,7 @@ func TestReclaimRevertsExpiredOnly(t *testing.T) {
}
// Zero grace: the dead lease (expired) is reclaimed, the live one is not.
reclaimed, err = store.ReclaimExpiredLeases(ctx, 0, "reaper")
reclaimed, err = store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim grace=0: %v", err)
}
@ -627,7 +627,7 @@ func TestConcurrentHeartbeatReclaimClose(t *testing.T) {
return
default:
}
if _, err := store.ReclaimExpiredLeases(raceCtx, 0, "reaper"); err != nil {
if _, err := store.ReclaimExpiredLeases(raceCtx, 0, types.ReclaimFilter{}, "reaper"); err != nil {
if raceCtx.Err() == nil {
reaperErrs.Add(1)
}
@ -684,7 +684,7 @@ func TestConcurrentHeartbeatReclaimClose(t *testing.T) {
// dead workers' issues reach their terminal open state regardless of reaper
// timing.
time.Sleep(ttl + 1500*time.Millisecond)
if _, err := store.ReclaimExpiredLeases(ctx, 0, "final-reaper"); err != nil {
if _, err := store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "final-reaper"); err != nil {
t.Fatalf("final reclaim sweep: %v", err)
}
@ -782,7 +782,7 @@ func TestStartedAtStampedOnceAcrossClaims(t *testing.T) {
}
deadStart := dead.startedAt.Time
time.Sleep(expiryMargin)
reclaimed, err := store.ReclaimExpiredLeases(ctx, 0, "reaper")
reclaimed, err := store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim: %v", err)
}
@ -826,7 +826,7 @@ func TestHeartbeatRevivesExpiredLease(t *testing.T) {
t.Errorf("heartbeat did not revive lease: %v", revived.leaseExpires)
}
reclaimed, err := store.ReclaimExpiredLeases(ctx, 0, "reaper")
reclaimed, err := store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("reclaim grace=0: %v", err)
}
@ -868,7 +868,7 @@ func TestConcurrentReapersAreBenign(t *testing.T) {
wg.Add(1)
go func(actor string) {
defer wg.Done()
got, err := store.ReclaimExpiredLeases(ctx, 0, actor)
got, err := store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, actor)
mu.Lock()
defer mu.Unlock()
if err != nil {

View file

@ -1008,9 +1008,9 @@ func (r *issueSQLRepositoryImpl) UnclaimIssue(ctx context.Context, id, actor str
return nil
}
func (r *issueSQLRepositoryImpl) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error) {
func (r *issueSQLRepositoryImpl) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error) {
cutoff := time.Now().UTC().Add(-olderThan)
out, err := issueops.ReclaimExpiredLeasesInTx(ctx, r.runner, cutoff, actor)
out, err := issueops.ReclaimExpiredLeasesInTx(ctx, r.runner, cutoff, filter, actor)
if err != nil {
return nil, fmt.Errorf("db: IssueSQLRepository.ReclaimExpiredLeases: %w", err)
}

View file

@ -745,7 +745,7 @@ func (s *testSuite) issueClaimStampsLease() {
tx, err := s.db.BeginTx(s.Ctx(), nil)
s.Require().NoError(err)
defer func() { _ = tx.Rollback() }()
reclaimed, err := issueops.ReclaimExpiredLeasesInTx(s.Ctx(), tx, time.Now().Add(time.Hour), "reaper")
reclaimed, err := issueops.ReclaimExpiredLeasesInTx(s.Ctx(), tx, time.Now().Add(time.Hour), types.ReclaimFilter{}, "reaper")
s.Require().NoError(err)
s.Require().NoError(tx.Commit())
var owner string

View file

@ -70,7 +70,7 @@ type IssueSQLRepository interface {
GetStaleIssues(ctx context.Context, filter types.StaleFilter) ([]*types.Issue, error)
GetEpicsEligibleForClosure(ctx context.Context) ([]*types.EpicStatus, error)
UnclaimIssue(ctx context.Context, id, actor string, force bool) error
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error)
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error)
}
type CloseRowParams struct {
@ -240,7 +240,7 @@ type IssueUseCase interface {
GetStaleIssues(ctx context.Context, filter types.StaleFilter) ([]*types.Issue, error)
GetEpicsEligibleForClosure(ctx context.Context) ([]*types.EpicStatus, error)
Unclaim(ctx context.Context, id, actor string, force bool) error
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error)
ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error)
CreateIssue(ctx context.Context, params CreateIssueParams, actor string) (CreateIssueResult, error)
CreateIssues(ctx context.Context, params []CreateIssueParams, actor string) (CreateIssuesResult, error)
@ -1598,8 +1598,8 @@ func (u *issueUseCaseImpl) Unclaim(ctx context.Context, id, actor string, force
return nil
}
func (u *issueUseCaseImpl) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error) {
out, err := u.issueRepo.ReclaimExpiredLeases(ctx, olderThan, actor)
func (u *issueUseCaseImpl) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error) {
out, err := u.issueRepo.ReclaimExpiredLeases(ctx, olderThan, filter, actor)
if err != nil {
return nil, fmt.Errorf("ReclaimExpiredLeases: %w", err)
}

View file

@ -120,12 +120,12 @@ func (s *EmbeddedDoltStore) HeartbeatIssue(ctx context.Context, id, actor string
// ReclaimExpiredLeases reverts in_progress issues whose lease expired more than
// olderThan ago back to ready, recovering work stranded by dead workers.
func (s *EmbeddedDoltStore) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, actor string) ([]types.ReclaimedLease, error) {
func (s *EmbeddedDoltStore) ReclaimExpiredLeases(ctx context.Context, olderThan time.Duration, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error) {
cutoff := time.Now().UTC().Add(-olderThan)
var reclaimed []types.ReclaimedLease
err := s.withConn(ctx, true, func(tx *sql.Tx) error {
var err error
reclaimed, err = issueops.ReclaimExpiredLeasesInTx(ctx, tx, cutoff, actor)
reclaimed, err = issueops.ReclaimExpiredLeasesInTx(ctx, tx, cutoff, filter, actor)
return err
})
return reclaimed, err

View file

@ -58,7 +58,7 @@ func TestLeaseLifecycleEmbedded(t *testing.T) {
// Let the lease expire, then reclaim it.
time.Sleep(2500 * time.Millisecond)
reclaimed, err := te.store.ReclaimExpiredLeases(ctx, 0, "reaper")
reclaimed, err := te.store.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("ReclaimExpiredLeases: %v", err)
}
@ -159,7 +159,7 @@ func TestReclaimExpiredLeaseSurvivesRestartEmbedded(t *testing.T) {
time.Sleep(wait)
}
reclaimed, err := store2.ReclaimExpiredLeases(ctx, 0, "reaper")
reclaimed, err := store2.ReclaimExpiredLeases(ctx, 0, types.ReclaimFilter{}, "reaper")
if err != nil {
t.Fatalf("ReclaimExpiredLeases after restart: %v", err)
}

View file

@ -8,6 +8,7 @@ import (
"time"
"github.com/steveyegge/beads/internal/storage"
"github.com/steveyegge/beads/internal/storage/sqlbuild"
"github.com/steveyegge/beads/internal/types"
)
@ -288,18 +289,27 @@ func HeartbeatIssueInTx(ctx context.Context, tx DBTX, id, actor string) error {
// are never leased work. Returns the issues it reverted (id + the owner it took
// the lease from) so the caller can log/emit recovery events. The caller owns
// Dolt versioning.
func ReclaimExpiredLeasesInTx(ctx context.Context, tx DBTX, cutoff time.Time, actor string) ([]types.ReclaimedLease, error) {
//
// filter narrows which stale leases are eligible (see types.ReclaimFilter); the
// zero filter keeps the historical global behavior. Scoping is applied to the
// snapshot SELECT only — the per-row DELETE/UPDATE re-checks staleness by id,
// and the ids it re-checks all came from the already-scoped snapshot, so a
// label removed mid-reclaim cannot smuggle an out-of-scope issue into the
// revert.
func ReclaimExpiredLeasesInTx(ctx context.Context, tx DBTX, cutoff time.Time, filter types.ReclaimFilter, actor string) ([]types.ReclaimedLease, error) {
// Snapshot the stale set first so we can report exactly which issues we
// reverted and record per-issue recovery events. The DELETE below repeats
// the expiry predicate, so an issue that a concurrent heartbeat rescued
// between the SELECT and the DELETE is simply skipped (0 rows) — it never
// appears as reclaimed.
scopeSQL, scopeArgs := sqlbuild.ReclaimScopeSQL(filter, sqlbuild.IssuesFilterTables, "i")
args := append([]any{cutoff}, scopeArgs...)
rows, err := tx.QueryContext(ctx, `
SELECT l.issue_id, COALESCE(i.assignee, '') FROM leases l
JOIN issues i ON i.id = l.issue_id
WHERE i.status = 'in_progress'
AND l.lease_expires_at < ?
`, cutoff)
`+scopeSQL, args...)
if err != nil {
return nil, fmt.Errorf("scan for stale leases: %w", err)
}

View file

@ -0,0 +1,111 @@
package sqlbuild
import (
"fmt"
"strings"
"github.com/steveyegge/beads/internal/types"
)
// BuildReclaimScopeClauses renders the WHERE fragments that scope a reclaim to
// a subset of the stale-lease set. It returns only the scoping predicates: the
// staleness predicate itself (status/lease_expires_at) belongs to the reclaim
// query and is not this builder's business.
//
// alias is the SQL alias the reclaim query binds the issues table to (e.g. "i"
// in `... FROM leases l JOIN issues i ON i.id = l.issue_id`); the id/assignee
// column references are qualified with it so the clauses compose with the
// leases JOIN without column ambiguity. Pass "" for an unaliased issues table.
//
// The label clauses are deliberately the same SQL shapes the claim/ready path
// emits (AND-set per-label subquery, OR-set IN-list, exclusion NOT IN), so a
// fleet that partitions its claims by label can partition its reclaims
// identically and get the same matching semantics. All fields AND-combine; the
// zero filter yields no clauses at all.
//
// FAIL-CLOSED on a degenerate field: a field that carries entries which are ALL
// empty strings renders "1=0" (match nothing) rather than dropping out. Dropping
// out is how `bd reclaim --label "$LANE"` with LANE unset would silently become
// a GLOBAL sweep — the precise accident scoping exists to prevent. Reclaiming
// nothing is recoverable; reclaiming the whole federation is not. (The CLI
// rejects such a flag outright; this is the belt to that suspenders, covering
// programmatic callers.)
func BuildReclaimScopeClauses(filter types.ReclaimFilter, tables FilterTables, alias string) ([]string, []any) {
var clauses []string
var args []any
// col qualifies a column of the issues table with the query's alias.
col := func(name string) string {
if alias == "" {
return name
}
return alias + "." + name
}
// degenerate reports a field that was populated but has no usable values.
degenerate := func(raw, compacted []string) bool { return len(raw) > 0 && len(compacted) == 0 }
ids := CompactNonEmptyStrings(filter.IDs)
switch {
case degenerate(filter.IDs, ids):
clauses = append(clauses, "1=0")
case len(ids) > 0:
ph, idArgs := InPlaceholders(ids)
clauses = append(clauses, fmt.Sprintf("%s IN (%s)", col("id"), ph))
args = append(args, idArgs...)
}
assignees := CompactNonEmptyStrings(filter.Assignees)
switch {
case degenerate(filter.Assignees, assignees):
clauses = append(clauses, "1=0")
case len(assignees) > 0:
ph, aArgs := InPlaceholders(assignees)
clauses = append(clauses, fmt.Sprintf("%s IN (%s)", col("assignee"), ph))
args = append(args, aArgs...)
}
labels := CompactNonEmptyStrings(filter.Labels)
if degenerate(filter.Labels, labels) {
clauses = append(clauses, "1=0")
}
for _, label := range labels {
clauses = append(clauses, fmt.Sprintf("%s IN (SELECT issue_id FROM %s WHERE label = ?)", col("id"), tables.Labels))
args = append(args, label)
}
any := CompactNonEmptyStrings(filter.LabelsAny)
switch {
case degenerate(filter.LabelsAny, any):
clauses = append(clauses, "1=0")
case len(any) > 0:
ph, lArgs := InPlaceholders(any)
clauses = append(clauses, fmt.Sprintf("%s IN (SELECT issue_id FROM %s WHERE label IN (%s))", col("id"), tables.Labels, ph))
args = append(args, lArgs...)
}
// An all-empty exclusion set excludes nothing, which cannot widen the set
// beyond what the other clauses allow — but a degenerate ExcludeLabels ALONE
// would otherwise leave the sweep global, so it fails closed too.
excl := CompactNonEmptyStrings(filter.ExcludeLabels)
switch {
case degenerate(filter.ExcludeLabels, excl):
clauses = append(clauses, "1=0")
case len(excl) > 0:
ph, lArgs := InPlaceholders(excl)
clauses = append(clauses, fmt.Sprintf("%s NOT IN (SELECT issue_id FROM %s WHERE label IN (%s))", col("id"), tables.Labels, ph))
args = append(args, lArgs...)
}
return clauses, args
}
// ReclaimScopeSQL is BuildReclaimScopeClauses rendered as a suffix ready to be
// appended to an existing WHERE clause (leading " AND ", or empty when the
// filter constrains nothing).
func ReclaimScopeSQL(filter types.ReclaimFilter, tables FilterTables, alias string) (string, []any) {
clauses, args := BuildReclaimScopeClauses(filter, tables, alias)
if len(clauses) == 0 {
return "", nil
}
return " AND " + strings.Join(clauses, " AND "), args
}

View file

@ -0,0 +1,90 @@
package sqlbuild
import (
"strings"
"testing"
"github.com/steveyegge/beads/internal/types"
)
func TestBuildReclaimScopeClausesEmpty(t *testing.T) {
clauses, args := BuildReclaimScopeClauses(types.ReclaimFilter{}, IssuesFilterTables, "i")
if len(clauses) != 0 || len(args) != 0 {
t.Fatalf("empty filter yielded clauses=%v args=%v, want none", clauses, args)
}
suffix, sargs := ReclaimScopeSQL(types.ReclaimFilter{}, IssuesFilterTables, "i")
if suffix != "" || sargs != nil {
t.Fatalf("empty filter suffix = %q args=%v, want empty", suffix, sargs)
}
}
func TestBuildReclaimScopeClausesShapeAndAlias(t *testing.T) {
filter := types.ReclaimFilter{
IDs: []string{"wy-a", "wy-b"},
Assignees: []string{"zelda"},
Labels: []string{"lane-a", "tier:opus"},
LabelsAny: []string{"x", "y"},
ExcludeLabels: []string{"pinned"},
}
clauses, args := BuildReclaimScopeClauses(filter, IssuesFilterTables, "i")
joined := strings.Join(clauses, " AND ")
wantContains := []string{
"i.id IN (?,?)", // IDs
"i.assignee IN (?)", // assignees
"i.id IN (SELECT issue_id FROM labels WHERE label = ?)", // per-label AND (x2)
"i.id IN (SELECT issue_id FROM labels WHERE label IN (?,?))", // label-any OR
"i.id NOT IN (SELECT issue_id FROM labels WHERE label IN (?))", // exclude
}
for _, w := range wantContains {
if !strings.Contains(joined, w) {
t.Errorf("clauses missing %q\n got: %s", w, joined)
}
}
// Two AND-labels => two per-label subqueries.
if got := strings.Count(joined, "WHERE label = ?"); got != 2 {
t.Errorf("per-label AND subquery count = %d, want 2", got)
}
// Args order: ids, assignee, each AND-label, label-any set, exclude set.
want := []any{"wy-a", "wy-b", "zelda", "lane-a", "tier:opus", "x", "y", "pinned"}
if len(args) != len(want) {
t.Fatalf("args = %v, want %v", args, want)
}
for i := range want {
if args[i] != want[i] {
t.Fatalf("arg[%d] = %v, want %v (full %v)", i, args[i], want[i], args)
}
}
}
func TestBuildReclaimScopeClausesNoAlias(t *testing.T) {
clauses, _ := BuildReclaimScopeClauses(types.ReclaimFilter{IDs: []string{"wy-a"}}, IssuesFilterTables, "")
if len(clauses) != 1 || clauses[0] != "id IN (?)" {
t.Fatalf("no-alias id clause = %v, want [\"id IN (?)\"]", clauses)
}
}
// A scope field supplied with only-empty values must FAIL CLOSED to "1=0"
// (match nothing), never drop out into a global sweep. This is the belt to the
// CLI's suspenders, protecting programmatic callers.
func TestBuildReclaimScopeClausesFailsClosedOnDegenerate(t *testing.T) {
cases := map[string]types.ReclaimFilter{
"ids": {IDs: []string{"", ""}},
"assignees": {Assignees: []string{""}},
"labels": {Labels: []string{""}},
"labelsAny": {LabelsAny: []string{""}},
"exclude": {ExcludeLabels: []string{""}},
}
for name, f := range cases {
t.Run(name, func(t *testing.T) {
clauses, args := BuildReclaimScopeClauses(f, IssuesFilterTables, "i")
if len(args) != 0 {
t.Fatalf("degenerate %s produced args %v, want none", name, args)
}
joined := strings.Join(clauses, " AND ")
if !strings.Contains(joined, "1=0") {
t.Fatalf("degenerate %s = %q, want a 1=0 fail-closed clause", name, joined)
}
})
}
}

View file

@ -1447,6 +1447,31 @@ type ReclaimedLease struct {
PreviousOwner string `json:"previous_owner"`
}
// ReclaimFilter scopes which stale-lease issues bd reclaim may revert. The
// zero value reclaims every stale lease (the historical global behavior);
// every populated field narrows the set further (fields AND-combine).
//
// Scoping matters on a federated deployment: each replica's view of another
// machine's liveness is stale by up to one sync interval, so an unscoped
// reaper can revert a unit that is very much alive on the machine that granted
// its lease. Partitioning reclaim by the same label surface the claim side is
// partitioned by (--label/--label-any/--exclude-label, plus --assignee/--id)
// turns after-the-fact revert auditing into prevention.
type ReclaimFilter struct {
IDs []string // Only these issue IDs are eligible
Assignees []string // Only leases held by one of these owners
Labels []string // AND semantics: issue must have ALL these labels
LabelsAny []string // OR semantics: issue must have AT LEAST ONE of these labels
ExcludeLabels []string // Exclusion: issue must NOT have ANY of these labels
}
// IsEmpty reports whether the filter constrains nothing, i.e. reclaim runs in
// its global, unscoped form.
func (f ReclaimFilter) IsEmpty() bool {
return len(f.IDs) == 0 && len(f.Assignees) == 0 &&
len(f.Labels) == 0 && len(f.LabelsAny) == 0 && len(f.ExcludeLabels) == 0
}
// WorkFilter is used to filter ready work queries
type WorkFilter struct {
Status Status