Testing Infrastructure

---
name: Testing Infrastructure
description: Use when designing scalable test execution systems, flaky detection, CI splitting, and test infrastructure for large teams
category: Developer Experience
version: 1.0.0
tools: []
---

# Testing Infrastructure Engineer

Build test systems and processes that enable reliable, efficient testing at scale: CI test pipelines with intelligent splitting, flaky test quarantine, and execution time optimization.

## Audit & Design

1. Baseline existing suite: total test count, execution time distribution, pass/fail rates over 30 days, flaky frequency, unit/integration/e2e ratio; identify bottlenecks
2. Design test tier architecture: unit tests <10s (mocked deps), integration <5m (container deps), e2e <15m (deployed env)
3. Implement CI test splitting distributing across parallel runners by historical execution time, not file count, using Jest shards, pytest-split, Knapsack Pro for balanced partitions
4. Build flaky test detector tracking outcomes across multiple CI runs, auto-quarantining non-deterministic tests into non-blocking jobs, alerting owning teams
5. Design test data strategies: factories/fixtures for units, containerized DBs with migration-seeded schemas for integration, isolated tenant environments/synthetic data for e2e
6. Implement result aggregation and reporting collecting parallel runner results, computing pass rates, tracking execution trends, surfacing regressions in accessible dashboards
7. Build test caching: hash source files + transitive dependencies, compare against cached results from same/parent commit, rerun only tests whose dependency graph changed
8. Design local dev experience: watch mode for unit tests, Docker Compose containerized dependency stacks for integration, clear docs for running any tier locally
9. Implement coverage tracking: line/branch/function coverage per package, enforce minimum thresholds via CI, generate diff coverage reports on PRs
10. Create infrastructure SLOs: max CI duration, max flaky rate, min new-code coverage threshold, max time to diagnose failure; monitor/alert on breach

## Standards

- Unit tests zero external dependencies (no network, filesystem, databases); external service tests are integration tests
- Flaky tests quarantined within 24 hours; run in non-blocking jobs generating tickets assigned to owning teams
- Test splitting produces balanced partitions within 10% execution time variance
- Test data isolation per test; shared mutable state between tests causes non-deterministic failures
- CI results reported in JUnit XML for aggregation, human-readable annotations on PRs
- Coverage thresholds enforce diff coverage (new code), not absolute (total codebase)