The Local Voice Revolution: For a decade, mainstream voice control meant inviting Amazon or Google microphones into your private living spaces and tolerating 1.5-second cloud round-trips that fail whenever your ISP hiccups. By deploying an on-premises Wyoming Voice Pipeline (combining Whisper, Piper, and openWakeWord) on Home Assistant Core, you achieve 100% private, zero-cloud voice automation with sub-350ms total execution latency. Here is the step-by-step architecture, hardware latency benchmarks, and satellite configuration guide.

1. The Four-Stage Wyoming Architecture

Home Assistant’s voice assistant architecture relies on the Wyoming protocol—a lightweight, event-driven TCP protocol designed by the Open Home Foundation. Wyoming decouples voice processing into four modular, independently scalable services that run as local microservices (Docker containers or Home Assistant add-ons):

[ Room Microphone: ESP32-S3 / Companion App ]
│ (Audio Stream: 16 kHz 16-bit Mono PCM)
▼
┌───────────────────────────────────────────────┐
│ Stage 1: Wake Word Detection │
│ • microWakeWord (on ESP32) OR openWakeWord │ → <50ms trigger
└───────────────────────┬───────────────────────┘
│ (Audio Buffer)
▼
┌───────────────────────────────────────────────┐
│ Stage 2: Speech-to-Text (STT) │
│ • faster-whisper (tiny.en / base.en / distil) │ → ~120ms transcription
└───────────────────────┬───────────────────────┘
│ (Text String: “turn off kitchen lights”)
▼
┌───────────────────────────────────────────────┐
│ Stage 3: Intent Parsing & Execution │
│ • Home Assistant Assist Engine (HassIL) │ → ~15ms local state match
└───────────────────────┬───────────────────────┘
│ (Response Text: “Turned off 4 lights”)
▼
┌───────────────────────────────────────────────┐
│ Stage 4: Text-to-Speech (TTS) │
│ • Piper (en_US-lessac-medium) │ → ~75ms audio synthesis
└───────────────────────┬───────────────────────┘
│ (Audio Output)
▼
[ Speaker Output: Total <300ms ]

2. Hardware Latency Benchmarks: Choosing the Right Host

Running neural speech recognition locally requires adequate CPU vector instructions. While a Raspberry Pi 4 can execute Whisper, the transcription delay introduces painful pauses that ruin conversational responsiveness.

Hardware Platform Whisper Model STT Latency Piper TTS Latency Total Pipeline Round-Trip Practical Grade
Raspberry Pi 4 / 5
ARM Cortex-A72 / A76
tiny.en (INT8) 850 ms – 1,400 ms 320 ms 1,500 ms – 2,200 ms ❌ Unacceptably Sluggish
Intel N100 Mini PC
Beelink EQ13 (AVX2)
base.en (INT8) 140 ms – 180 ms 65 ms 280 ms – 350 ms ★★★★★ Exceptional Sweet Spot
Intel Core i5-12400 / 13400
6-Core Desktop SFF
distil-medium.en 85 ms – 110 ms 35 ms 180 ms – 230 ms ★★★★★ Blazing Sub-Second
NVIDIA GPU (CUDA / TensorRT)
GTX 1650 / RTX 3060
large-v3 45 ms – 70 ms 20 ms 120 ms – 160 ms Overkill / High Idle Power

3. Step-by-Step Installation Guide (Home Assistant Core / OS)

Step 1: Deploy Whisper and Piper via Wyoming

  1. In Home Assistant, navigate to Settings → Add-ons → Add-on Store.
  2. Install the Whisper add-on. Under the configuration tab:
    • Set model to base or base-int8 (ideal speed/accuracy balance).
    • Set language to your primary language (e.g., en).
  3. Install the Piper add-on. Under configuration:
    • Select high-natural voice model: en_US-lessac-medium or en_US-ryan-medium.
  4. Install openWakeWord to handle wake words locally on the server if not using edge hardware wake-words.
  5. Start all three add-ons. Home Assistant will auto-discover them under Settings → Devices & Services via the Wyoming integration.

Step 2: Assemble the Voice Assistant Pipeline

  1. Go to Settings → Voice Assistants and click Add Assistant.
  2. Name the pipeline Local Sovereign Voice.
  3. Configure the pipeline components:
    • Wake Word: openWakeWord (e.g., “Okay Nabu”, “Hey Jarvis”, or custom model).
    • Speech-to-Text: Whisper.
    • Conversation Agent: Home Assistant (uses native local intent matching).
    • Text-to-Speech: Piper.
  4. Toggle “Set as Preferred Assistant”.

4. Satellite Hardware: ESP32-S3 with On-Device microWakeWord

To place physical voice satellites throughout your home without relying on your phone, deploy custom ESP32-S3 satellites running ESPHome.

The game-changer for satellite design is microWakeWord: wake-word inference runs directly on the ESP32-S3’s dual-core Xtensa microcontroller. The device continuously buffers audio locally, but streams zero audio across the Wi-Fi network until the wake word is verified on the chip itself:

  • Recommended Hardware: ESP32-S3-BOX-3 or M5Stack CoreS3. Features dual noise-canceling I2S microphones and high-output DAC speakers in a clean desktop enclosure.
  • Network Security: Place voice satellites on an isolated IoT VLAN alongside your smart plugs and switches. Satellites communicate strictly with the Home Assistant server over TCP port 6053 (ESPHome native API).

5. Eliminating Entity Confusion: Aliasing Best Practices

Local intent parsing uses deterministic regex and grammar trees rather than generative AI guesswork. If you say “Turn on the lamp” but your entity is named light.living_room_corner_standing_fixture_dimmer, the intent matcher will fail.

The Aliasing Rulebook

Navigate to Settings → Voice Assistants → Exposed Entities. Click on each device and add multiple spoken aliases:

Target: light.living_room_lamp
Aliases: “lamp”, “reading light”, “couch lamp”, “corner light”.

Assign every entity to an explicit Area (e.g., Living Room, Master Bedroom). When talking to a satellite located in the Living Room, saying “Turn on the lights” will automatically contextualize to only the lights in that specific room!

Frequently Asked Questions

Does local Home Assistant voice work when the internet is down?

Yes. The entire Wyoming pipeline (Wake Word, Whisper STT, Intent Engine, and Piper TTS) runs entirely on your local server. When your fiber or cable internet connection drops, voice control operates with zero interruption and zero degradation.

How much RAM does Whisper and Piper consume on Home Assistant?

Running faster-whisper with the base.en model consumes roughly 450MB of RAM. Piper with a medium voice model consumes approximately 180MB. Together with openWakeWord, the total memory footprint of the complete local voice stack is under 1.2GB of RAM.

Can I use a local LLM (like Ollama) with Home Assistant Assist?

Yes. You can configure Home Assistant’s conversation agent to route queries to an on-premises Ollama instance running Llama 3 or Mistral. Home Assistant will first attempt to match deterministic commands locally; if no device command matches, it passes the transcript to the local LLM for conversational answers.

// LOCAL-FIRST HOME AUTOMATION & PROTOCOLS

// The Local Mesh

Bi-weekly architectures for local-first smart homes, Home Assistant YAML patterns, Matter/Thread co-existence, and private NVR pipelines.