incident-commander

---
name: incident-commander
description: Use when declaring an incident, coordinating a multi-team response during an outage, leading a post-mortem, or setting up on-call practices. Covers severity classification and timeline reconstruction.
category: Engineering
version: 1.0.0
tools: []
---

# Incident Commander

**Category:** Engineering Team

Incident response framework for **availability/reliability incidents** (outages, degradations, failed deploys): severity classification, timeline reconstruction into a coherent narrative, and post-incident review.

**This is not security incident triage.** For security events (ransomware, intrusion, data exfiltration, forensics), route to a security incident-response process instead. Both use SEV1-SEV4 labels; this framework scores operational impact (users, revenue, SLA), while security triage classifies attack types and forensic handling.

## Key Capabilities

- Severity triage based on impact and urgency metrics
- Timeline reconstruction — turn scattered logs and events into a coherent incident narrative
- Post-Incident Review generation with multiple RCA frameworks (5 Whys, Fishbone, Timeline)
- Communication templates for stakeholder updates and escalations
- Runbook generation from recurring incident patterns

## Incident Response Framework

### Severity Classification System

**SEV1 — Critical Outage.** Complete service failure affecting all users or critical business functions: customer-facing services completely unavailable, data loss or corruption, security breach with customer data exposure, revenue-generating systems down, or SLA violations with financial penalties. Response: immediate escalation, Incident Commander assigned within 5 minutes, executive notification within 15 minutes, public status-page update within 15 minutes, war room established. Communicate every 15 minutes until resolution.

**SEV2 — Major Impact.** Significant degradation affecting a subset of users or non-critical functions: partial degradation affecting >25% of users, performance issues causing frustration, non-critical features unavailable, internal tools impacted. Response: on-call response within 15 minutes, Incident Commander assigned within 30 minutes, status-page update within 30 minutes, stakeholder notification within 1 hour. Communicate every 30 minutes during active response.

**SEV3 — Minor Impact.** Limited impact with workarounds available: single feature or component affected, <25% of users impacted, performance degradation without significant UX impact. Response: within 2 hours during business hours, next business day acceptable outside hours, internal notification, optional status-page update. Communicate at key milestones only.

**SEV4 — Low Impact.** Cosmetic bugs, documentation issues, monitoring gaps, or planned maintenance. Response: within 1-2 business days via standard ticketing, no special escalation. Communicate on the standard development cycle.

### Incident Commander Role

**Command and control** — own the response process, decide resource allocation, coordinate across technical teams and stakeholders, maintain situational awareness across all response streams.

**Communication hub** — provide regular updates, manage external communications (status pages, customer notices), shield responders from external distractions.

**Process management** — ensure proper tracking and documentation, drive toward resolution without sacrificing quality, coordinate handoffs, plan rollback strategies.

**Post-incident leadership** — ensure a thorough PIR happens, drive preventive measures, share learnings broadly.

### Decision-Making Framework

**Emergency decisions (SEV1/2):** the Incident Commander has full authority, biases toward action over analysis, documents decisions for later review, and consults subject-matter experts without getting blocked on them.

**Resource allocation:** can pull in any necessary team member, escalate to senior leadership, approve emergency spend for external resources, and set communication channels and cadence.

**Technical decisions:** lean on technical leads for implementation detail, make the final call on speed-vs-risk trade-offs, approve rollback vs. fix-forward, and coordinate testing/validation.

## Communication Templates

**Initial notification (SEV1/2):** state name, severity, start time, current status, impact description, affected services, symptoms, initial root-cause assessment, response team (Incident Commander, technical lead, SMEs), next-update time, status-page and war-room links.

**Executive summary (SEV1):** a 2-3 sentence customer-impact and business-implication summary, key metrics (time to detection, time to engagement, estimated customer impact, ETA to resolution), and the leadership actions required (customer-communication approval, PR coordination, resource decisions, vendor engagement).

**Customer communication:** state what's affected in plain language, when engineering was alerted, the update cadence being followed, what is known so far, what the team is actively doing, and any workaround — or state plainly that none exists yet.

### Stakeholder Communication Cadence

| Stakeholder | SEV1 | SEV2 | SEV3 | SEV4 |
|-------------|------|------|------|------|
| Engineering Leadership | Real-time | 30min | 4hrs | Daily |
| Executive Team | 15min | 1hr | EOD | Weekly |
| Customer Support | Real-time | 30min | 2hrs | As needed |
| Customers | 15min | 1hr | Optional | None |
| Partners | 30min | 2hrs | Optional | None |

Internal stakeholders typically include engineering leadership, product management, customer support, sales/account management, the executive team, and legal/compliance. External stakeholders include customers, partners, vendors, regulators (for regulated industries), and public/media for public-facing outages.

## Runbook Generation Framework

Every runbook has three layers: **detection playbooks** (monitoring alert definitions, triage decision trees, escalation trigger points, initial response actions), **response playbooks** (step-by-step mitigation, rollback instructions, validation checkpoints, communication checkpoints), and **recovery playbooks** (service restoration, data-consistency checks, performance validation, user notification).

A runbook should cover: quick-reference severity indicators and key contacts; detection (monitoring alerts and manual detection signs); initial response in the first 0-15 minutes (assess severity, establish command, page the Incident Commander for SEV1/2, initial investigation of recent deploys and error logs); named mitigation strategies with concrete steps and a rollback plan for each; recovery and validation steps; common pitfalls; and a link to deeper reference material.

## Best Practices

**During response:** stay composed under pressure and make decisive calls with incomplete information; communicate confidence while acknowledging uncertainty; document every action, decision rationale, and timeline as it happens; use clear, jargon-free language and keep updating even without new information; prefer rollbacks to risky fixes under pressure; validate fixes before declaring resolution; plan for secondary and cascading failures.

**Post-incident:** run a blameless process that focuses on system failures, not individual mistakes; assign PIR action items to specific owners with due dates and track them publicly, prioritized by risk and effort; share PIRs broadly and update runbooks with the lessons learned.

Present the severity classification, timeline, communication drafts, and runbook directly in the chat as structured markdown; @mention the Incident Commander or on-call teammate rather than only logging the assessment.