Voice Assistant

---
name: Voice Assistant
description: Use when building voice interfaces with speech-to-text, text-to-speech, natural language understanding, and dialog management for Alexa or Google Assistant.
category: Specialized
version: 1.0.0
tools: []
---

# Voice Assistant Development

Build conversational voice interfaces with speech recognition, natural language understanding, dialog management, and speech synthesis for Alexa and Google Assistant.

## Voice Design Principles

- Design the voice user interface (VUI) defining intents (user goals), slots (parameters), sample utterances (≥20 per intent), and dialog flow with slot elicitation and disambiguation
- Implement speech-to-text: Whisper (offline/self-hosted), Google Cloud Speech, or Amazon Transcribe with language-specific fine-tuning
- Build natural language understanding: platform NLU (Alexa Skills Kit, Dialogflow) or custom NER for domain-specific entities
- Design dialog management as a state machine: track conversation context, manage multi-turn interactions, handle topic switching, maintain session state

## Response Design

- Write speech-optimized text: short sentences (≤30 words), no abbreviations, SSML markup for pronunciation control
- Use neural TTS: Amazon Polly Neural, Google Cloud TTS WaveNet, Coqui TTS (self-hosted)
- Select voice appropriate to brand persona, apply SSML-driven prosody, optimize audio format (Opus for streaming, MP3 for cached responses)
- Every response must end with actionable prompt or explicit session closure (silence without guidance is UX failure)

## Platform Implementation

- **Alexa**: Lambda function or HTTPS endpoint handling LaunchRequest, IntentRequest, SessionEndedRequest, with session attribute management and progressive response support
- **Google Assistant**: Actions SDK or Dialogflow CX with webhook fulfillment, intent matching, parameter extraction, rich response types (cards, carousels, suggestions), voice-only compatibility

## Error Handling & Analytics

- Error recovery: unrecognized speech (reprompt with examples), ambiguous input (clarifying question), out-of-scope (guide back to capabilities), service errors (apologize and suggest retry)
- Track intent recognition rates (target >95%), slot fill success, dialog turn counts, task completion, and user drop-off points
- Response latency: utterance end to audio playback start ≤2 seconds
- Voice responses: max 30 seconds for informational content
- Test intent recognition accuracy with sample utterance set
- Validate multi-turn scenarios end-to-end, test error recovery paths
- Verify TTS output quality, SSML rendering, and platform certification requirements