* feat(github): sync epic/child and blocks relationships to GitHub bd github sync only ever pushed issue content (title/body/labels/state); Epic-child links and beads "blocks" dependencies never reached GitHub's relationships panel. Adds a GitHub relationship-sync pass, mirroring the existing GitLab dependency-link push, using GitHub's sub-issues and issue dependencies REST APIs: - internal/github/client.go: AddSubIssue/ListSubIssues and AddBlockedBy/ListBlockedBy against GitHub's sub_issues and dependencies/blocked_by endpoints. - internal/github/links.go: DependencyLink + LinkResolver, converting beads epic/child parent-child dependencies into GitHub sub-issues and beads "blocks" dependencies into GitHub blocked_by relationships. Idempotent (lists current relationships before creating) and additive (stale remote relationships are left untouched). - cmd/bd/github.go, cmd/bd/sync_push_pull.go: wires the push pass into both `bd github sync` and `bd github push`. Verified against a live GitHub repo (create, list, and idempotent re-push all confirmed via the real API), in addition to httptest-mocked unit tests. Fixes gastownhall/beads#5970 * fix(github): update last_sync after relationship push to prevent sync churn When AddSubIssue or AddBlockedBy is called, GitHub updates the issue's updated_at timestamp as a side effect. Without updating last_sync after the relationship push completes, the next sync would see those issues as 'changed' and pull/push them again, causing unnecessary churn. This fix updates last_sync to the current time after a successful relationship push (only if links were created), accounting for any timestamp changes GitHub made during relationship creation. Fixes continuous sync churn observed when running bd github sync multiple times with no local changes. * fix(github): use relationship reader for link sync * fix(bazel): enumerate links.go and its tests in internal/github BUILD.bazel The base advance added internal/github/BUILD.bazel with explicitly enumerated srcs (gazelle-generated). This PR adds links.go, links_test.go and review_fixes_test.go, which the base-generated enumeration cannot know about, so //internal/github:github failed to build: internal/github/tracker.go:41:35: undefined: LinkResolver internal/github/tracker.go:244:30: undefined: RefScope Regenerated with `bazel run //:gazelle` (not hand-edited). Purely additive; no deps change is needed because links.go only imports //internal/types, already declared. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(github): scope the relationship pass to --parent and report skipped links Maintainer review fixup for PR #5971, adopt-pr review iterations 1 and 3. filterGitHubLinkScopedIssues reimplemented four of the SyncOptions selectors and dropped ParentID, so `bd github sync --push --parent X` narrowed the content push to the subtree and then created GitHub sub-issue and blocked-by links for every GitHub-linked issue in the workspace -- outward API writes on issues the user had excluded. It now resolves the subtree and requires membership in it. The traversal moves from gitlab.go to sync_flags.go as buildSyncDescendantSet, beside the --parent flag that populates the field, and the GitLab and GitHub link passes share the one copy. The doc comments that claimed parity with the engine's push loop now enumerate the selectors this path actually applies, and name the three it does not (State, ExcludeIDPrefix, ExcludeIDPatterns -- no GitHub command sets them today). PushLinkResult.UnsupportedSkipped had no reader anywhere in the repo, so a host without the sub-issue/dependency APIs reported nothing at all. pushGitHubDependencyLinks now emits one curated line for it, and a 404 on the source issue is classified apart from a 404 on the relationship endpoint: collapsing the two reported a deleted, renumbered or permission-hidden issue as an unsupported host. The classifying probe is memoized per issue number, so a source carrying both link types costs one extra read. `bd github push --dry-run` printed "Synced N relationship links" for links it had only planned, because PushLinks counts planned links in Created. The `bd github sync` twin already gated that line on its dry-run flag; the push side now does too. Tests: a table test over filterGitHubLinkScopedIssues with a row per selector -- its three ParentID rows fail without the scope fix -- plus a direct pin that a blocks-edge dependent is not a descendant, and httptest cases driving 404 through the list call, the create call and the source-issue probe. Not fixed here: the PR body still describes a `lastSync` advance that commit ee6d65692 removed from this PR. That is a PR-surface edit, not a code change, and is left to the finalize step. Also not fixed here, and named in buildSyncDescendantSet's doc comment so the shared helper does not read as covering every tracker: the ADO link pass has the same scope hole this commit closes for GitHub. pushADOLinks takes no sync options, so `bd ado sync --push --parent X` still writes ADO relations for the whole workspace. That predates this PR, and ADO is outside its diff. Agent-Signature: claude-code-claude-opus-5-5-max on behalf of Gas City Adopt-PR Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Julian Knutsen <julian@gastownhall.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Gas City Adopt-PR <adopt-pr@gascity.local>
264 lines
7.3 KiB
Go
264 lines
7.3 KiB
Go
package github
|
|
|
|
import (
|
|
"bytes"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"net/http"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// HTTP header names. Defined as constants to avoid silent typos in
|
|
// rate-limit detection.
|
|
const (
|
|
headerAccept = "Accept"
|
|
headerAPIVersion = "X-GitHub-Api-Version"
|
|
headerContentType = "Content-Type"
|
|
headerRetryAfter = "Retry-After"
|
|
headerRateLimitRemaining = "X-RateLimit-Remaining"
|
|
headerRateLimitReset = "X-RateLimit-Reset"
|
|
headerRateLimitLimit = "X-RateLimit-Limit"
|
|
headerRateLimitResource = "X-RateLimit-Resource"
|
|
)
|
|
|
|
// APIError is a non-retryable GitHub API failure that carries the HTTP status
|
|
// code, so callers can branch on it instead of matching error strings (e.g.
|
|
// treating 404 from the relationship endpoints as "unavailable here" rather
|
|
// than as a hard failure). Its message is unchanged from the untyped error it
|
|
// replaces.
|
|
type APIError struct {
|
|
StatusCode int
|
|
Body string
|
|
URL string
|
|
}
|
|
|
|
func (e *APIError) Error() string {
|
|
return fmt.Sprintf("API error: %s (status %d)", e.Body, e.StatusCode)
|
|
}
|
|
|
|
// IsNotFound reports whether err is, or wraps, a GitHub 404.
|
|
func IsNotFound(err error) bool {
|
|
var apiErr *APIError
|
|
return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusNotFound
|
|
}
|
|
|
|
// AuthError indicates a GitHub 403/401 that is not a rate limit (bad token,
|
|
// missing scopes, IP allowlist, etc.). Auth errors are not retried.
|
|
type AuthError struct {
|
|
StatusCode int
|
|
Message string
|
|
URL string
|
|
}
|
|
|
|
func (e *AuthError) Error() string {
|
|
if e.Message != "" {
|
|
return fmt.Sprintf("github auth error (status %d): %s", e.StatusCode, e.Message)
|
|
}
|
|
return fmt.Sprintf("github auth error (status %d)", e.StatusCode)
|
|
}
|
|
|
|
// RateLimitErrorKind distinguishes primary (header-driven) from secondary
|
|
// (body-driven) GitHub rate limits, which require different backoff strategies.
|
|
type RateLimitErrorKind int
|
|
|
|
const (
|
|
// RateLimitPrimary is the documented per-token cap, signaled by
|
|
// X-RateLimit-Remaining=0 with a reliable reset epoch.
|
|
RateLimitPrimary RateLimitErrorKind = iota
|
|
// RateLimitSecondary covers abuse / content-creation / concurrency limits.
|
|
// GitHub recommends a 60-second minimum backoff when no Retry-After is sent.
|
|
RateLimitSecondary
|
|
)
|
|
|
|
func (k RateLimitErrorKind) String() string {
|
|
switch k {
|
|
case RateLimitPrimary:
|
|
return "primary"
|
|
case RateLimitSecondary:
|
|
return "secondary"
|
|
default:
|
|
return "unknown"
|
|
}
|
|
}
|
|
|
|
// RateLimitError represents a GitHub-imposed rate limit.
|
|
type RateLimitError struct {
|
|
Kind RateLimitErrorKind
|
|
StatusCode int
|
|
RetryAfter time.Duration // 0 if Retry-After was not present
|
|
ResetAt time.Time // zero if X-RateLimit-Reset was not present
|
|
Remaining int // -1 if absent
|
|
Limit int // -1 if absent
|
|
Resource string
|
|
Message string
|
|
URL string
|
|
}
|
|
|
|
// RateLimitRetryAfter implements tracker.RateLimitedError. Returns
|
|
// Retry-After if set, else time-until-reset for primary limits, else the
|
|
// 60-second secondary minimum.
|
|
func (e *RateLimitError) RateLimitRetryAfter() time.Duration {
|
|
if e.RetryAfter > 0 {
|
|
return e.RetryAfter
|
|
}
|
|
if e.Kind == RateLimitPrimary && !e.ResetAt.IsZero() {
|
|
if d := time.Until(e.ResetAt); d > 0 {
|
|
return d
|
|
}
|
|
}
|
|
if e.Kind == RateLimitSecondary {
|
|
return 60 * time.Second
|
|
}
|
|
return 0
|
|
}
|
|
|
|
func (e *RateLimitError) Error() string {
|
|
parts := []string{fmt.Sprintf("github %s rate limit (status %d)", e.Kind, e.StatusCode)}
|
|
if e.RetryAfter > 0 {
|
|
parts = append(parts, fmt.Sprintf("retry-after=%s", e.RetryAfter))
|
|
}
|
|
if !e.ResetAt.IsZero() {
|
|
parts = append(parts, fmt.Sprintf("reset-at=%s", e.ResetAt.UTC().Format(time.RFC3339)))
|
|
}
|
|
if e.Resource != "" {
|
|
parts = append(parts, fmt.Sprintf("resource=%s", e.Resource))
|
|
}
|
|
if e.Message != "" {
|
|
parts = append(parts, e.Message)
|
|
}
|
|
return strings.Join(parts, ": ")
|
|
}
|
|
|
|
// RetryConfig controls retry behavior for the GitHub client.
|
|
type RetryConfig struct {
|
|
MaxRetries int
|
|
BaseDelay time.Duration
|
|
// SecondaryMinDelay is the floor when GitHub returns a secondary rate
|
|
// limit without a Retry-After header. Per docs.github.com this should be
|
|
// at least 60 seconds.
|
|
SecondaryMinDelay time.Duration
|
|
MaxBackoff time.Duration
|
|
}
|
|
|
|
func DefaultRetryConfig() RetryConfig {
|
|
return RetryConfig{
|
|
MaxRetries: 5,
|
|
BaseDelay: time.Second,
|
|
SecondaryMinDelay: 60 * time.Second,
|
|
MaxBackoff: 5 * time.Minute,
|
|
}
|
|
}
|
|
|
|
// classifyRateLimit returns a *RateLimitError if the response carries any
|
|
// GitHub rate-limit signal. Caller must restrict invocation to 403/429.
|
|
func classifyRateLimit(headers http.Header, body []byte, statusCode int, urlStr string) *RateLimitError {
|
|
if !isRateLimited(headers, body) {
|
|
return nil
|
|
}
|
|
|
|
rlErr := &RateLimitError{
|
|
StatusCode: statusCode,
|
|
Remaining: parseHeaderInt(headers, headerRateLimitRemaining, -1),
|
|
Limit: parseHeaderInt(headers, headerRateLimitLimit, -1),
|
|
Resource: headers.Get(headerRateLimitResource),
|
|
Message: extractGitHubMessage(body),
|
|
URL: urlStr,
|
|
}
|
|
|
|
if reset := parseHeaderInt(headers, headerRateLimitReset, 0); reset > 0 {
|
|
rlErr.ResetAt = time.Unix(int64(reset), 0)
|
|
}
|
|
if ra := headers.Get(headerRetryAfter); ra != "" {
|
|
if seconds, err := strconv.Atoi(ra); err == nil {
|
|
rlErr.RetryAfter = time.Duration(seconds) * time.Second
|
|
} else if t, err := http.ParseTime(ra); err == nil {
|
|
if d := time.Until(t); d > 0 {
|
|
rlErr.RetryAfter = d
|
|
}
|
|
}
|
|
}
|
|
|
|
// Primary limits set Remaining=0 and a reset epoch. Secondary limits
|
|
// never expose Remaining=0; everything else routes there.
|
|
if rlErr.Remaining == 0 {
|
|
rlErr.Kind = RateLimitPrimary
|
|
} else {
|
|
rlErr.Kind = RateLimitSecondary
|
|
}
|
|
return rlErr
|
|
}
|
|
|
|
func parseHeaderInt(headers http.Header, key string, fallback int) int {
|
|
v := headers.Get(key)
|
|
if v == "" {
|
|
return fallback
|
|
}
|
|
n, err := strconv.Atoi(v)
|
|
if err != nil {
|
|
return fallback
|
|
}
|
|
return n
|
|
}
|
|
|
|
// rateLimitBodyMarkers are the documented substrings GitHub uses in its JSON
|
|
// error message to indicate a secondary / abuse rate limit. Required because
|
|
// secondary limits do NOT set X-RateLimit-Remaining=0, so headers alone are
|
|
// not sufficient.
|
|
var rateLimitBodyMarkers = []string{
|
|
"secondary rate limit",
|
|
"abuse",
|
|
}
|
|
|
|
// scanBytes caps body inspection to avoid lowercasing a 50MB response just
|
|
// to detect a marker that would only ever appear in the first few hundred
|
|
// bytes of a JSON error envelope.
|
|
const scanBytes = 4096
|
|
|
|
func isRateLimited(headers http.Header, body []byte) bool {
|
|
if headers.Get(headerRetryAfter) != "" {
|
|
return true
|
|
}
|
|
if headers.Get(headerRateLimitRemaining) == "0" {
|
|
return true
|
|
}
|
|
if len(body) == 0 {
|
|
return false
|
|
}
|
|
head := body
|
|
if len(head) > scanBytes {
|
|
head = head[:scanBytes]
|
|
}
|
|
lower := bytes.ToLower(head)
|
|
for _, marker := range rateLimitBodyMarkers {
|
|
if bytes.Contains(lower, []byte(marker)) {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|
|
|
|
// extractGitHubMessage returns the "message" field from a GitHub JSON error
|
|
// body, or a clamped raw-body fallback for non-JSON responses (e.g. CDN
|
|
// error pages).
|
|
func extractGitHubMessage(body []byte) string {
|
|
if len(body) == 0 {
|
|
return ""
|
|
}
|
|
if body[0] == '{' {
|
|
var env struct {
|
|
Message string `json:"message"`
|
|
}
|
|
if err := json.Unmarshal(body, &env); err == nil {
|
|
return env.Message
|
|
}
|
|
}
|
|
const maxRaw = 200
|
|
s := strings.TrimSpace(string(body))
|
|
if len(s) > maxRaw {
|
|
return s[:maxRaw] + "…"
|
|
}
|
|
return s
|
|
}
|