* feat(test): differential regression suite (v0.49.6 baseline) Adds 53 regression test scenarios comparing current bd behavior against v0.49.6 SQLite baseline. Tests run identical CLI scenarios against both binaries in isolated workspaces, capture JSONL snapshots via bd export, normalize volatile fields, and fail on any unexpected difference. 44 PASS, 9 SKIP (documenting known regressions/intentional changes), 0 FAIL. Build-tagged `//go:build regression` so it never runs in normal `go test`. Run with: `go test -tags=regression -timeout=10m -v ./tests/regression/...` * test(regression): add 17 scenarios targeting recent open bugs New scenarios covering subtle bugs found by surveying GH issues since v0.50: Epic lifecycle (GH#1495): - TestEpicAllChildrenClosedNotBlocked: epic with all children closed should not show as BLOCKED with "0 dependencies" - TestBlockedEpicChildrenNotReady: children of blocked epic should not appear in bd ready Dependency semantics (GH#1524, GH#1858): - TestCloseGuardNonBlocksDepTypes: close guard should not reject close for non-blocks dep types (caused-by, validates, tracks) - TestListResolvedBlockerAnnotation: bd list should not show "blocked by" after blocker is closed CLI surface coverage (GH#1840, GH#1954): - TestListAllOverridesLimitWithLabel: --all should override limit of 50 when combined with --label filter - TestDepTreeOutput: bd dep tree should show full parent-child hierarchy - TestChildrenCommand: bd children parity between backends - TestBlockedCommandParity: bd blocked --json agreement - TestCountCommandParity: bd count agreement - TestStaleCommandParity: bd stale agreement - TestQueryDSLParity: bd query DSL returns same results Data integrity (GH#1914, GH#1926, GH#1927, GH#1969, GH#1973): - TestShowJSONFieldCompleteness: bd show --json includes all fields from baseline (catches scan projection drift) - TestExportImportEpicTree: multi-level epic tree export → import round-trip - TestRapidLabelAddRemoveStability: rapid label churn not silently rolled back - TestRapidDepAddRemoveStability: rapid dep churn consistency - TestRapidCommentAddStability: 10 rapid comments all survive - TestLargeExportImportRoundTrip: 20 issues with deps/labels/comments round-trip at scale * test(regression): add bug discovery tests and 14 confirmed bug reports Systematic manual testing against current main (Dolt server mode) found 14 bugs and confirmed 23 protocol invariants. All tests require a running Dolt server on :3307 and use unique prefixes for test isolation. Bug reproduction tests (10 FAIL = confirmed bugs): - BUG-2: dep tree shows no children (ParentID never set) - BUG-3: dep tree shows [READY] on blocked root - BUG-4: list --status blocked returns empty - BUG-7: dep add silently overwrites dep type - BUG-8: reparented child appears under both parents - BUG-10: close guard / claim exit 0 on soft failure - BUG-11: update --status accepts arbitrary values - BUG-12: update --title "" stores empty title - BUG-13: reopen of closed+deferred creates limbo state - BUG-14: empty string label accepted Protocol invariant tests (23 PASS): Close guard, epic lifecycle, delete cleanup, reopen preservation, transitive blocking, cycle detection, force close, defer/undefer, claim semantics, notes, supersede, duplicate, count grouping, special chars, SQL injection safety, dep remove, self-dep prevention, status/type/due-date round-trips, label/comment round-trips. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(regression): add audit verdicts and session log to DISCOVERY.md Triage all 14 bugs: 5-6 clear fix PRs, 2 need design discussion, 5-6 wontfix/by-design. Add session log to avoid redoing work. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(regression): revised audit, protocol test classifications, prior art - Updated bug verdicts based on code review: BUG-4 upgraded to DOCS FIX, BUG-5 upgraded to INVESTIGATE (INSERT IGNORE is correct but concurrent Dolt working-set commits may race), BUG-7 downgraded to FILE ISSUE (upsert is intentionally coded, needs product decision) - Tagged all 32 protocol test ideas as DATA INTEGRITY, POLICY/UX, or MESSAGE CONTRACT to distinguish hard invariants from policy decisions - Added "Prior Art" section documenting key Dolt migration fixes already merged (PRs #1797, #1816, #1909, #1914, #1966-1969, #1948) and Dolt-specific constraints learned from them Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
164 lines
5.9 KiB
Makefile
164 lines
5.9 KiB
Makefile
# Makefile for beads project
|
|
|
|
.PHONY: all build test test-full-cgo test-regression bench bench-quick clean install help check-up-to-date fmt fmt-check
|
|
|
|
# Default target
|
|
all: build
|
|
|
|
BUILD_DIR := .
|
|
INSTALL_DIR := $(HOME)/.local/bin
|
|
|
|
# Dolt backend requires CGO for embedded database support.
|
|
# Without CGO, builds will fail with "dolt backend requires CGO".
|
|
#
|
|
# Windows notes:
|
|
# - ICU is NOT required. go-icu-regex has a pure-Go fallback (regex_windows.go)
|
|
# and gms_pure_go tag tells go-mysql-server to use pure-Go regex too.
|
|
# - CGO_ENABLED=1 needs a C compiler (MinGW/MSYS2) but does NOT need ICU.
|
|
export CGO_ENABLED := 1
|
|
|
|
# When go.mod requires a newer Go version than the locally installed one,
|
|
# GOTOOLCHAIN=auto downloads the right compiler but coverage instrumentation
|
|
# may still use the local toolchain's compile tool, causing version mismatch.
|
|
# Force the go.mod version to ensure all tools match.
|
|
GO_VERSION := $(shell sed -n 's/^go //p' go.mod)
|
|
ifneq ($(GO_VERSION),)
|
|
export GOTOOLCHAIN := go$(GO_VERSION)
|
|
endif
|
|
|
|
# ICU4C is keg-only in Homebrew (not symlinked into the prefix).
|
|
# Dolt's go-icu-regex dependency needs these paths to compile and link.
|
|
# This handles both macOS (brew --prefix icu4c) and Linux/Linuxbrew.
|
|
# On Windows, ICU is not needed (pure-Go regex via gms_pure_go + regex_windows.go).
|
|
ifneq ($(OS),Windows_NT)
|
|
ICU_PREFIX := $(shell brew --prefix icu4c 2>/dev/null)
|
|
ifneq ($(ICU_PREFIX),)
|
|
export CGO_CFLAGS += -I$(ICU_PREFIX)/include
|
|
export CGO_CPPFLAGS += -I$(ICU_PREFIX)/include
|
|
export CGO_LDFLAGS += -L$(ICU_PREFIX)/lib
|
|
# Linuxbrew gcc doesn't install a 'c++' symlink; point CGO at g++
|
|
ifeq ($(shell uname),Linux)
|
|
export CXX ?= g++
|
|
endif
|
|
endif
|
|
endif
|
|
|
|
# Build the bd binary
|
|
build:
|
|
@echo "Building bd..."
|
|
ifeq ($(OS),Windows_NT)
|
|
go build -tags gms_pure_go -ldflags="-X main.Build=$$(git rev-parse --short HEAD)" -o $(BUILD_DIR)/bd.exe ./cmd/bd
|
|
else
|
|
go build -ldflags="-X main.Build=$$(git rev-parse --short HEAD)" -o $(BUILD_DIR)/bd ./cmd/bd
|
|
ifeq ($(shell uname),Darwin)
|
|
@codesign -s - -f $(BUILD_DIR)/bd 2>/dev/null || true
|
|
@echo "Signed bd for macOS"
|
|
endif
|
|
endif
|
|
|
|
# Run all tests (skips known broken tests listed in .test-skip)
|
|
test:
|
|
@echo "Running tests..."
|
|
@TEST_COVER=1 ./scripts/test.sh
|
|
|
|
# Run full CGO-enabled test suite (no skip list).
|
|
# On macOS, auto-configures ICU include/link flags.
|
|
test-full-cgo:
|
|
@echo "Running full CGO-enabled tests..."
|
|
@./scripts/test-cgo.sh ./...
|
|
|
|
# Run differential regression tests (baseline v0.49.6 vs current worktree).
|
|
# Downloads baseline binary on first run; cached in ~/Library/Caches/beads-regression/.
|
|
# Override baseline: BD_REGRESSION_BASELINE_BIN=/path/to/bd make test-regression
|
|
test-regression:
|
|
@echo "Running regression tests (baseline vs candidate)..."
|
|
go test -tags=regression -timeout=10m -v ./tests/regression/...
|
|
|
|
# Run performance benchmarks against Dolt storage backend
|
|
# Requires CGO and Dolt; generates CPU profile files
|
|
# View flamegraph: go tool pprof -http=:8080 <profile-file>
|
|
bench:
|
|
@echo "Running performance benchmarks (Dolt backend)..."
|
|
@echo ""
|
|
go test -bench=. -benchtime=1s -benchmem -run=^$$ ./internal/storage/dolt/ -timeout=30m
|
|
@echo ""
|
|
@echo "Benchmark complete."
|
|
|
|
# Run quick benchmarks (shorter benchtime for faster feedback)
|
|
bench-quick:
|
|
@echo "Running quick performance benchmarks..."
|
|
go test -bench=. -benchtime=100ms -benchmem -run=^$$ ./internal/storage/dolt/ -timeout=15m
|
|
|
|
# Check that local branch is up to date with origin/main
|
|
check-up-to-date:
|
|
ifndef SKIP_UPDATE_CHECK
|
|
@git fetch origin main --quiet 2>/dev/null || true
|
|
@LOCAL=$$(git rev-parse HEAD 2>/dev/null); \
|
|
REMOTE=$$(git rev-parse origin/main 2>/dev/null); \
|
|
if [ -n "$$REMOTE" ] && [ "$$LOCAL" != "$$REMOTE" ]; then \
|
|
echo "ERROR: Local branch is not up to date with origin/main"; \
|
|
echo " Local: $$(git rev-parse --short HEAD)"; \
|
|
echo " Remote: $$(git rev-parse --short origin/main)"; \
|
|
echo "Run 'git pull' first, or use SKIP_UPDATE_CHECK=1 to override"; \
|
|
exit 1; \
|
|
fi
|
|
endif
|
|
|
|
# Install bd to ~/.local/bin (builds, signs on macOS, and copies)
|
|
# Also creates 'beads' symlink as an alias for bd
|
|
install: check-up-to-date build
|
|
@mkdir -p $(INSTALL_DIR)
|
|
ifeq ($(OS),Windows_NT)
|
|
@rm -f $(INSTALL_DIR)/bd.exe
|
|
@cp $(BUILD_DIR)/bd.exe $(INSTALL_DIR)/bd.exe
|
|
@echo "Installed bd.exe to $(INSTALL_DIR)/bd.exe"
|
|
else
|
|
@rm -f $(INSTALL_DIR)/bd
|
|
@cp $(BUILD_DIR)/bd $(INSTALL_DIR)/bd
|
|
@echo "Installed bd to $(INSTALL_DIR)/bd"
|
|
@rm -f $(INSTALL_DIR)/beads
|
|
@ln -s bd $(INSTALL_DIR)/beads
|
|
@echo "Created 'beads' alias -> bd"
|
|
endif
|
|
|
|
# Format all Go files
|
|
fmt:
|
|
@echo "Formatting Go files..."
|
|
@gofmt -w .
|
|
@echo "Done"
|
|
|
|
# Check that all Go files are properly formatted (for CI)
|
|
fmt-check:
|
|
@echo "Checking Go formatting..."
|
|
@UNFORMATTED=$$(gofmt -l .); \
|
|
if [ -n "$$UNFORMATTED" ]; then \
|
|
echo "The following files are not properly formatted:"; \
|
|
echo "$$UNFORMATTED"; \
|
|
echo ""; \
|
|
echo "Run 'make fmt' to fix formatting"; \
|
|
exit 1; \
|
|
fi
|
|
@echo "All Go files are properly formatted"
|
|
|
|
# Clean build artifacts and benchmark profiles
|
|
clean:
|
|
@echo "Cleaning..."
|
|
rm -f bd
|
|
rm -f bd.exe
|
|
rm -f internal/storage/dolt/bench-cpu-*.prof
|
|
rm -f beads-perf-*.prof
|
|
|
|
# Show help
|
|
help:
|
|
@echo "Beads Makefile targets:"
|
|
@echo " make build - Build the bd binary"
|
|
@echo " make test - Run all tests"
|
|
@echo " make test-full-cgo - Run full CGO-enabled test suite"
|
|
@echo " make test-regression - Run differential regression tests (baseline vs candidate)"
|
|
@echo " make bench - Run performance benchmarks (generates CPU profiles)"
|
|
@echo " make bench-quick - Run quick benchmarks (shorter benchtime)"
|
|
@echo " make install - Install bd to ~/.local/bin (with codesign on macOS, includes 'beads' alias)"
|
|
@echo " make fmt - Format all Go files with gofmt"
|
|
@echo " make fmt-check - Check Go formatting (for CI)"
|
|
@echo " make clean - Remove build artifacts and profile files"
|
|
@echo " make help - Show this help message"
|