Using Espressif's speech recognition on an "Unexpected Maker ProS3" ESP32-S3 development board.
I wanted to play around with voice recognition on "the edge". I had a ProS3 and an I2S MEMS microphone available. ESP-Skainet seemed like a good place to start.
I just used a breadboard to make up the "board" used for this project, wiring the I2S microphone to the ESP32-S3.
- Wire an I2S microphone module to an ESP32-S3.
- Build and flash the project to the board using ESP-IDF.
- Say "Hey, Wanda".
- Say one of the voice commands (see Custom Voice Commands).
- ProS3 ESP32-S3 Dev board.
- I2S Microphone Module (incorporates MSM261S4030H0).
TODO: Add a nice wiring diagram.
ESP-Skainet only supports a handful of specific development boards out of the box.
Taking the hardware_driver component from ESP-Skainet, I stripped out the factory boards and created a custom BSP for the project "board".
The result is in components/ewth_hardware_driver. I wanted to differentiate it from the base hardware_driver to avoid tripping over myself; the naming isn't great but I'll worry about that later.
I had mixed results with wake word detection. Alexa and Hey, Wanda worked well; the Hi, ESP model wouldn't work at all. Whether it's my Australian accent or some other issue remains undetermined.
The Multinet 7 speech command worked pretty well for voice command detection. I played around with a number of words and phrases, and found it was very good at recognising some, while it struggled with others. This again could be an accent issue.
- Wake word model:
wn9_heywanda_tts(Wakenet 9) - Speech command model:
mn7_en(Multinet 7)
Using g2p-en, I converted some graphenes to phonemes as follows:
| Graphene | Phoneme |
|---|---|
hello |
hcLb |
goodbye |
GwDBi |
test |
TfST |
ewan |
YocN |
I had trouble getting it to recognise "hello" so I added a few additional phonemes for it, based on what it was reporting as detected.
These are saved in data/commands_en.txt. The root CMakeLists.txt file copies this file into the appropriate directory
- Haven't optimised anything yet.
- Everything is included in the build. Need to refine.