Help for simple voice assistant setup

[mirkosertic]

Hi there!

I am struggling a little bit with a simple voice assistant setup to get sample rate handling and conversion right.

I am using an ESP32 Audio-Kit. The microphone should be enabled all the time and set to a constant sample rate for audio streaming.
However, I want to occasionally play back either a WAV or MP3 file from URL stream sources. The source file sample rates differ from source to source.

So as I understood, I have to use the FormatConverterStream on the output side. However, this sometimes seems to work, and sometimes the rate conversion seems to be wrong, I get a very distorted or slowed down playback.

I am pretty sure that I am doing something wrong. So my question is if someone could point me to an example where I can use for instance a single StreamCopy instance and bind this to varying sources while keeping a constant audio sample output rate AND recording at a constant audio sample rate at the same time would be great!

Thank you all,
Mirko

[pschatzmann]

I is a bit unclear what you are doing: Are you mixing the different inputs or just switching between the different sources ?

The AudioKit uses I2S in douplex mode, so when your mp3 changes the sample rate, the microphone is automatically changed to the same rate.

So I don't understand why you want to convert anything. But when you send the output from the decoer to a FormatConverter, you can fix the sample rate to a fixed output rate which will not change the I2S.

You can do this both on the input side or output side of the copy.

[mirkosertic]

Hi! Thanks for your reply!

Well, I am basically switching between recording and playback mode.

Recording must be done at a given sample rate, and playback as I understand now will switch the microphone to the sample rate of the playback source, which might be a different one than the one required for recording. So now I understand that I have to reconfigure the microphone to the required sample rate after playback has finished, right? Is this just done by calling begin(AudioInfo) of the source stream again?

Thank you,
Mirko

[pschatzmann]

After playback, just call setAudioInfo(info) with the recording values on I2S. If this updates only the sample rate, this is much more efficient.

[mirkosertic]

Wow, that was fast! Thanks again. Final question: I only have to call begin() if every thing changes then, like sample rate, number of channels and bit depth, right?

[pschatzmann]

No, setAudioInfo() will work in any case...
The point that I wanted to make was, that for I2S I have implemented an efficient way to just update the sample rate. If more is changing, I2S is automatically restarted by setAudioInfo() as well.

[mirkosertic]

Ok, then my question is fully answered. Thank you for this great library and your support!