Commit graph

22 commits

Author SHA1 Message Date
Jozef Izso
42064c46fe
Sync Linear milestones as local epics (#3528) 2026-05-05 07:45:30 -07:00
Kevin Glynn
ac15e099e3
feat(linear): adopt issueBatchCreate/issueBatchUpdate for 50x efficiency (#3654)
* feat(linear): adopt issueBatchCreate/issueBatchUpdate for 50x efficiency

Implements tracker.BatchPushTracker interface on linear.Tracker.
Adds BatchCreateIssues and BatchUpdateIssues methods that chunk
inputs into groups of 50 (Linear's per-call limit). Graceful
fallback to per-issue mutations on batch failure.

* fix(linear): address BatchPush correctness blockers from maphew review

Three correctness issues flagged in PR #3654 review:

1. Skip semantics: BatchPush now fetches each existing issue before
   updating and compares with PushFieldsEqual (title, description,
   priority, status). Unchanged issues are added to result.Skipped
   instead of being re-pushed unconditionally. forceIDs bypasses the
   check, matching the single-issue path in engine.go:doPush.

2. Batch-create result mapping: results are now matched by title rather
   than array index. Linear's issueBatchCreate API does not guarantee
   response order matches input order; index-based mapping silently
   associated wrong ExternalRefs with issues. Unmatched inputs are
   recorded in result.Errors with the title for debugging.

3. Multi-team state cache: build a StateCache per configured team at the
   start of BatchPush. Updates to issues belonging to non-primary teams
   resolve workflow state IDs against that team's cache, not the primary
   team's. The primary cache is used only for creates (which always target
   the primary team).

Add four regression tests covering: unchanged-issue skip, forceIDs bypass,
title-based result mapping with reversed API response, and per-team state ID
selection in a two-team setup.

* fix(linear): prevent duplicate issues from batch-create correctness gaps

Two correctness fixes addressing maphew's re-review blockers on PR #3654:

1. Title-based result correlation is unsafe for duplicate titles.
   BatchPush now detects duplicate titles before batching. Issues with
   duplicate titles are routed through single-create with idempotency
   markers (CreateIssueIdempotent) instead of the batch mutation where
   title-based correlation would silently lose one duplicate.

2. Full-chunk fallback after ambiguous batch failure risks duplicates.
   BatchCreateIssues no longer falls back to creating the entire chunk
   when the batch mutation returns success=false or an API error. Instead
   it searches for each issue's idempotency marker to discover which were
   partially created, returns only confirmed issues, and errors on the
   rest. If the recovery search itself fails, the error propagates rather
   than risking duplicates.

Supporting infrastructure ported from feat/linear-idempotency-markers:
- idempotency.go: GenerateIdempotencyMarker, AppendIdempotencyMarker
- FindIssueByDescriptionContains, CreateIssueIdempotent on Client

* fix(linear): gofmt client.go and tracker_test.go

Fix Check formatting CI failure — two files had formatting drift
from the merge conflict resolution.
2026-05-03 16:28:13 -07:00
maphew
425ae20683 Merge main into OAuth client credentials branch 2026-05-02 19:14:05 -07:00
matt wilkie
a9e437a0a8
Merge pull request #3655 from kevglynn/feat/linear-retry-after-adaptive-backoff
feat(linear): parse Retry-After headers and add rate-limit circuit breaker
2026-05-02 19:05:11 -07:00
maphew
f118303741 Merge main into OAuth client credentials branch 2026-05-02 18:40:54 -07:00
kev
3e804cd39f fix(linear): address maintainer feedback on PR #3655
- Format types.go (struct field alignment was off by one space)
- parseRetryAfter: use http.ParseTime for HTTP-date form (RFC 1123 /
  RFC 850 / ANSI C) instead of time.Parse(time.RFC1123) — the latter
  requires an exact timezone abbreviation while http.ParseTime accepts
  the canonical GMT form mandated by RFC 9110 §10.2.3
- Add MaxRetryAfterDelay (300s) cap on server-provided Retry-After
  delays; log a warning to stderr and clamp when the hint exceeds the cap
- Wire linear.rate_limit_floor / LINEAR_RATE_LIMIT_FLOOR through both
  Tracker.Init() and getLinearClient() so the config key is functional
  via `bd config set linear.rate_limit_floor <n>` and the env var
- ErrRateLimitExhausted now implements RateLimitExhausted() bool;
  engine.go uses an interface check (no import cycle) to abort the
  push loop immediately on quota exhaustion instead of cascading the
  error across every remaining issue
- Tests: extend TestParseRetryAfter with HTTP-date future/past cases;
  add TestExecute_RetryAfterCapApplied to confirm the cap path is taken
2026-05-02 19:00:25 -04:00
kev
b8e7562f0f fix(linear): address maintainer blockers on idempotency PR
Fix 1 (double description formatting): Tracker.CreateIssue was calling
BuildLinearDescription(issue) to format the description, but the sync
engine's FormatDescription hook already calls BuildLinearDescription
before invoking CreateIssue. For issues with AcceptanceCriteria/Design/
Notes, this caused those sections to be appended twice in the Linear
description. Fix: use issue.Description directly in CreateIssue, trusting
the caller to have pre-formatted it.

Fix 2 (retry race in CreateIssueIdempotent): The previous implementation
called CreateIssue (which goes through Execute's retry loop) after the
initial marker search. If issueCreate reached Linear but the HTTP response
was lost, Execute could retry the mutation and create a duplicate issue
with the same idempotency marker. Fix: add createIssueSingleAttempt that
runs the mutation exactly once. After any failure, re-search for the
marker before returning the error so callers can safely retry
CreateIssueIdempotent without creating duplicates.

Also documents the concurrent multi-source limitation (Linear provides no
server-side uniqueness enforcement; the dedup window is bounded by search
index propagation).

Adds regression tests:
- TestCreateIssueNoDoubleFormatDescription: verifies AC/Design sections
  appear exactly once in the description sent to Linear
- TestCreateIssueIdempotentRecoverAfterAmbiguousFailure: verifies that an
  ambiguous create failure (5xx response) followed by a recovery search
  returns the already-created issue with deduped=true
2026-05-02 18:59:33 -04:00
kev
097ed1cf16 feat(linear): add OAuth client-credentials support
Add OAuthTokenManager for machine-to-machine authentication using
Linear's client_credentials grant. This enables CI workers to use a
service identity (actor=application) instead of personal API keys.

Key changes:
- New OAuthTokenManager with lazy acquisition, caching, thread-safe
  refresh (5-min buffer before expiry), and Invalidate() for 401 retry
- AuthMode enum (APIKey vs OAuth) on Client with appropriate header
  format: API keys use "Authorization: <key>", OAuth uses "Bearer"
- NewOAuthClient() constructor and WithOAuth() option
- 401 auto-retry: invalidate token and re-acquire on first 401
- Auth precedence: LINEAR_OAUTH_CLIENT_ID/SECRET env vars >
  LINEAR_API_KEY env var > bd config
- Mark oauth_client_secret as secret key (uses IsSecretKey from PR-1)
- 12 new tests covering acquisition, caching, refresh, invalidation,
  error handling, thread safety, and header format

Validated against Linear sandbox: client_credentials grant works with
scope=read,write, actor=application gives bot identity, 30-day TTL.

Refs: gastownhall/beads integration charter, PLAN.md §6 PR-3
2026-05-02 09:35:08 -04:00
kev
0ec4a99ea0 feat(linear): add idempotency markers to prevent duplicate issue creation
Embeds <!-- bd-idempotency: hash --> in Linear descriptions on create.
Pre-create dedup query uses description contains filter (validated
against Linear API). Hash uses immutable fields only (beadID +
creatorEmail + createdAtNano). Prevents duplicates under sync
interruption or multi-source create races.
2026-05-02 01:00:09 -04:00
kev
a845bc4eba feat(linear): parse Retry-After headers and add rate-limit circuit breaker
Server-hint-aware backoff: prefer Retry-After delay over computed
exponential backoff when present. Circuit breaker pauses sync when
remaining quota drops below linear.rate_limit_floor (default 100).
Addresses thundering-herd retry amplification pattern (cf. #3623).
2026-05-02 00:42:06 -04:00
Harry
2cd88e9426 fix: address PR review feedback
- Jira: reset body reader at top of retry loop (not just in retriable
  branch) so POST/PUT retries after network errors don't send empty bodies
- GitLab: add useServerDelay pattern matching other clients; skip jitter
  for server-mandated Retry-After delays
- Sanitize: replace custom decodeRune with stdlib utf8.DecodeRuneInString
- Ready --explain: log warnings instead of silently swallowing errors from
  GetDependencyCounts, GetDependencyRecordsForIssues, DetectCycles,
  GetIssuesByIDs
- Linear: fix pagination off-by-one; check page limit before processing
  instead of after, preventing MaxPages+1 fetches
2026-03-30 09:15:55 -07:00
Harry
860ef5969b lint: suppress gosec G404 for retry jitter (math/rand is fine for backoff)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-30 09:15:53 -07:00
Harry
733b8fab89 Sanitize external tracker content for terminal display and add response size limits
- Add ui.SanitizeForTerminal() to strip ANSI escape sequences, OSC sequences,
  and C0/C1 control characters from external content before terminal display.
  Preserves printable UTF-8, newlines, and tabs.
- Apply sanitization in tracker engine dry-run output (3 call sites)
- Add MaxResponseSize (50MB) and io.LimitReader to Linear client to prevent
  OOM from malformed API responses (was the only tracker without this guard)
- Add comprehensive test suite for sanitization (17 test cases including
  ANSI injection, screen clearing, OSC title change, control characters)

Fixes: bd-u9u

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-30 09:12:26 -07:00
Harry
c5e033f93e Add pagination guards to Linear and context checks to Jira client
- Linear FetchIssues/FetchIssuesSince: add MaxPages=1000 guard to prevent
  infinite loops from malformed API responses with hasNextPage=true
- Linear: add ctx.Done() checks at start of each pagination iteration
- Jira SearchIssues: add ctx.Done() check at start of each pagination iteration
- Add MaxPages constant to internal/linear/types.go

Fixes: bd-6fy

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-30 09:11:20 -07:00
Harry
00b78da749 Add retry logic to Jira client and jitter to all tracker HTTP clients
- Add retry with exponential backoff to Jira doRequest (was single-attempt)
- Add MaxRetries, RetryDelay, MaxResponseSize constants to jira/types.go
- Add response body size limit via io.LimitReader to Jira client
- Handle permanent errors (400/401/403/404) without retry
- Parse Retry-After header for rate-limited responses
- Add jitter (0-50% of delay) to backoff in all 5 tracker clients:
  GitHub, GitLab, Linear, ADO, and Jira
- Guard rand.Int64N against zero delay to prevent panics
- Add retry tests: 429, 503, no-retry-on-400, max-retries-exceeded

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-30 09:11:19 -07:00
lydia
0b95c75d68 fix(linear): correct Project sync issues from PR #2022
- Progress field: int → float64 (Linear returns 0.0-1.0)
- CreateProject: teamId → teamIds (projects span multiple teams)
- ProjectToEpic: map completedAt → ClosedAt, set CloseReason for canceled
- ProjectToEpic: canonicalize ExternalRef URL, remove raw Linear UUID as ID
- MapEpicToProjectState: map StatusInProgress → "started" for correct round-trip

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 12:24:19 -08:00
HuyNguyenDinh
e17d73ae9d feat(linear): add Project sync support
- Add Project type and related GraphQL response types
- Add FetchProjects, CreateProject, UpdateProject methods
- Add ProjectToEpic mapping function
- Resolve conflicts with upstream/main
2026-02-23 10:47:29 +07:00
jasper
619f34a441 chore: add explanatory comments to silently ignored errors across codebase
Code health review found 30+ instances of error suppression using blank
identifier without explanation. Added comments to 82 files explaining WHY
each error is safely ignored (best-effort cleanup, advisory operations,
safe defaults, etc.) to improve code maintainability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Executed-By: beads/polecats/jasper
Rig: beads
Role: polecats
2026-02-14 17:16:43 -08:00
beads/crew/lizzy
de567cda79 chore: unexport 14 dead symbols across 8 packages
Unexport package-internal symbols that have no external callers:
- validation: ValidatePrefix, ValidatePrefixWithAllowed, ForUpdate, ForClose, ForDelete, ForReopen
- debug: PrintNormal, PrintlnNormal
- linear: IssuesQuery
- timeparsing: IsCompactDuration, ParseNaturalLanguage
- utils: FindMoleculesJSONLInDir, ParseIssueID
- testutil/fixtures: XLargeFromJSONL
- storage/sqlite: Snapshot type

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:24:22 -08:00
Steve Yegge
0ed349b3ed
fix(linear): use project_id when creating issues via sync --push (GH#973) (#1012)
The linear.project_id config was being read and used for filtering
when pulling issues from Linear, but was not being passed when
creating new issues via --push. Now CreateIssue includes projectId
in the mutation input when configured.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 23:46:00 -08:00
Paddo
5254ade346
feat(linear): add project_id filter for sync (#938)
Add project_id filter for Linear sync

When linear.project_id is configured, bd linear sync will only fetch issues
belonging to that project instead of all team issues.

Closes #937
2026-01-07 20:45:00 -08:00
Justin Williams
df5ceb5d82
feat: Linear Integration (#655)
* Add Linear integration CLI with sync and status commands

- Add `bd linear sync` for bidirectional issue sync with Linear
- Add `bd linear status` to show configuration and sync state
- Stub pull/push functions pending GraphQL client (bd-b6b.2)

* Implement Linear GraphQL client with full sync support

- Add LinearClient with auth, fetch, create, update methods
- Implement pull/push operations with Beads type mapping
- Clean up redundant comments and remove unused code

* Add configurable data mapping and dependency sync for Linear

- Add LinearMappingConfig with configurable priority/state/label/relation maps
- Import parent-child and issue relations as Beads dependencies
- Support custom workflow states via linear.state_map.* config

* Add incremental sync support for Linear integration

- Add FetchIssuesSince() method using updatedAt filter in GraphQL
- Check linear.last_sync config to enable incremental pulls
- Track sync mode (incremental vs full) in LinearPullStats

* feat(linear): implement push updates for existing Linear issues

Add FetchIssueByIdentifier method to retrieve single issues by identifier
(e.g., "TEAM-123") for timestamp comparison during push.

Update doPushToLinear to:
- Fetch Linear issue to get internal ID and UpdatedAt timestamp
- Compare timestamps: only update if local is newer
- Build update payload with title, description, priority, and state
- Call UpdateIssue for issues where local has newer changes

Closes bd-b6b.5

* Implement Linear conflict resolution strategies

- Add true conflict detection by fetching Linear timestamps via API
- Implement --prefer-linear resolution (re-import from Linear)
- Implement timestamp-based resolution (newer wins as default)
- Fix linter issues: handle resp.Body.Close() and remove unused error return

* Add Linear integration tests and documentation

- Add comprehensive unit tests for Linear mapping (priority, state, labels, relations)
- Update docs/CONFIG.md with Linear configuration reference
- Add examples/linear-workflow guide for bidirectional sync
- Remove AI section header comments from tests

* Fix Linear GraphQL filter construction and improve test coverage

- Refactor filter handling to combine team ID into main filter object
- Add test for duplicate issue relation mapping
- Add HTTP round-trip helper for testing request payload validation

* Refactor Linear queries to use shared constant and add UUID validation

- Extract linearIssuesQuery to deduplicate FetchIssues/FetchIssuesSince
- Add linearMaxPageSize constant and UUID validation with regex
- Expand test coverage for new functionality

* Refactor Linear integration into internal/linear package

- Extract types, client, and mapping logic from cmd/bd/linear.go
- Create internal/linear/ package for better code organization
- Update tests to work with new package structure

* Add linear teams command to list available teams

- Add FetchTeams GraphQL query to Linear client
- Refactor config reading to support daemon mode
- Add tests for teams listing functionality

* Refactor Linear config to use getLinearConfig helper

- Consolidate config/env var lookup using getLinearConfig function
- Add LINEAR_TEAM_ID environment variable support
- Update error messages to include env var configuration options

* Add hash ID generation and improve Linear conflict detection

- Add configurable hash ID mode for Linear imports (matches bd/Jira)
- Improve conflict detection with content hash comparison
- Enhance conflict resolution with skip/force update tracking

* Fix test for updated doPushToLinear signature

- Add missing skipUpdateIDs parameter to test call
2025-12-19 17:58:24 -08:00