Whisper for Home Assistant (app)

About

Whisper is an app that adds local speech-to-text transcription to Home Assistant. Once running, it is discovered automatically by the Wyoming integration, which connects it to Home Assistant. It supports a default language, with an auto mode that detects the spoken language but runs much slower, and several Assist pipelines can use different languages at once. A choice of Whisper models is available, including compressed and distilled variants, and the default picks a smaller model on ARM devices. Custom models can be loaded from a local folder or from the HuggingFace Hub, and a beam size setting trades accuracy for performance. It can also translate spoken audio into English, and it requires Home Assistant OS on amd64 or aarch64.

Automatic summary of the docs and configuration of home-assistant/apps. Something wrong, or are you the maintainer and want your own text here? Get in touch

Supports

Updated with Home Assistant Ships and updates with every Home Assistant release.

Setting it up

Install it under Settings, Apps, then start it.

Permissions

Needs no special permissions.

Configuration

Beam sizeNumber of candidates to consider simultaneously during transcription (0 = auto). Increasing the beam size will increase accuracy at the cost of performance.
LanguageLanguage to pre-load model for during start-up.
ModelWhisper model that will be used for transcription. The default auto selects tiny-int8 for ARM devices like the Raspberry Pi 4 and base-int8 otherwise. Compressed models (int8) are slightly less accurate than their counterparts, but smaller and faster.
Custom modelPath to a converted model directory, or a CTranslate2-converted Whisper model ID from the HuggingFace Hub like "Systran/faster-distil-whisper-small.en". If custom_model_type is set to 'transformers', a HuggingFace transformers Whisper model ID from HuggingFace like 'openai/whisper-tiny.en' may be used.
Custom model typeSpeech-to-text backend to use for custom_model if stt_library is set to auto.
Initial promptDescription of audio that can help Whisper transcribe unusual words better.
Speech-to-text librarySpeech-to-text backend library to use. The default auto will select the best backend/model based on language/hardware. Overridden by custom_model_type when custom_model is set.
Whisper taskTask to perform with the model. The default transcribe transcribes audio in the spoken language, while translate translates the spoken audio into English.
Sherpa streamingUse streaming model with sherpa backend. This overrides the default English model with a faster but less accurate streaming model.
VAD clipUse voice activity detection (VAD) to clip silence from audio before transcription. This is a latency win for silence-heavy audio, especially with batch backends like sherpa and funasr. Streaming backends are unaffected.
Local files onlyOnly use models that have already been downloaded, and never check online for updates. Leave this off until your models have downloaded, then turn it on to keep the add-on offline and skip the update check on every request.
Debug loggingPrint DEBUG level messages to the app's log.