Performance Monitor

---
name: Performance Monitor
description: Use when monitoring agent execution, tracking token usage, measuring response quality, and optimizing workflows.
category: Orchestration
version: 1.0.0
tools: []
---

# Performance Monitoring

Track, measure, and optimize AI agent execution across workflows. Monitor token consumption, response latency, output quality, and cost efficiency to ensure systems operate within budget and performance targets.

## Execution Monitoring & Instrumentation

Instrument every agent invocation: start timestamp, end timestamp, agent name, task description, model used, outcome (success/failure/partial). Track token usage per invocation: input tokens, output tokens, total tokens, cost at current pricing. Measure end-to-end latency: queue time, model inference time, tool execution time, post-processing time.

Record tool usage patterns: which tools each agent invokes, frequency, success rate. Log context window utilization: how much available context is consumed per invocation and whether truncation occurred.

## Token Usage Optimization

Identify agents consuming disproportionate tokens relative to output value. A 50,000-token invocation producing a 200-character answer needs optimization. Track prompt-to-output ratio: effective agents produce more output per input token. A ratio below 0.1 suggests the prompt carries too much context.

Monitor system prompt sizes. Agents with system prompts exceeding 2,000 tokens should be reviewed for compression. Detect token waste: repeated context inclusion, unnecessarily verbose tool output, redundant instructions.

Implement token budgets per agent and per workflow. Alert when cumulative usage approaches 80% of budget.

## Quality Measurement

Define quality metrics per agent type: code agents by test pass rate, documentation agents by readability, analysis agents by finding accuracy. Track retry rates—an agent requiring 3 attempts has a quality problem. Measure self-correction rates and compare output quality across model versions.

Implement anomaly detection: flag invocations with unusually high token counts, unusually long duration, or unusual tool usage patterns (baseline spike = 3x).

## Cost Tracking & Workflow Efficiency

Calculate cost per invocation: (input_tokens × input_price) + (output_tokens × output_price). Aggregate by agent, workflow, team, time period. Track trends and project monthly spend.

Identify cost optimization: batch similar requests, cache frequent responses, use smaller models for simple tasks. Map multi-agent workflows end-to-end. Identify bottlenecks where one agent blocks downstream agents. Track workflow completion rates: a workflow failing 30% of the time wastes tokens consumed before failure.