Not an issue just thought I would ask >> beamforming

[StuartIanNaylor]

I have been thinking about how to digitally do the same as this analogue application.
https://invensense.tdk.com/wp-content/uploads/2015/02/Microphone-Array-Beamforming.pdf

So stereo I2S stream split tracks and add delay & invert one then sum back as a mono stream.

If you ever fancy giving it a go please do as interested as using mems with some directionality would be a great plus to the omnidirectional nature.

[garygru]

Yes, got for it! It should be no big deal. You'll probably want the highest sample rate available or even better a fractional delay because sample delay is a bit coarse, otherwise.
If you need help let me know. I'm going to add a minimal hardware abstraction soon. So these MEMS mics should work fine then.

[StuartIanNaylor]

Need loads of help :) I am learning C slowly and the ESP32 wrover and been playing with voice AI and can realise the advantage but playing catch-up with skills needed.

The noise rejection and directionality only helps with background noise but being able to point a mic at a source does help much over what a omnidirectional will provide.
I have been looking blankly at https://github.com/atomic14/ICS-43434-breakout-board (I am using cheap aliexpress INMP441 mems but I2S)

As in https://github.com/atomic14/ICS-43434-breakout-board/blob/main/sketch/I2SSampler.cpp

Its converted to 16bit here but guess its also where the Endfire beamforming could go.

void I2SSampler::processI2SData(uint8_t *i2sData, size_t bytes_read)
{
    int32_t *samples = (int32_t *)i2sData;
    for (int i = 0; i < bytes_read / 4; i++)
    {
        // you may need to vary the >> 11 to fit your volume - ideally we'd have some kind of AGC here
        addSample(samples[i] >> 11);
    }
}

I am going to try but hoping I can get more competent interested as I am not sure how much load this will eventually take.
From the above invensense app note we always have this trade off between sampling rate, mic distance and frequency effects.

I am pretty confident the ESP32 could make a great wifi beamforming mic, but also less confident if it could also squeeze in the KWS of https://www.youtube.com/watch?v=re-dSV_a0tM&t=345s

I have been thinking for a while that distributed microphone arrays if cheap and easy enough could be beneficial to some of the snake oil wrapped up in high tech beamformers that are costly but also much less effective than many try to present them as.

But yeah I would really like to see what sort of results you can produce if you play about with mic spacing and sampling rate.
The KWS is a bonus if it will squeeze in, to only broadcast mic until silence so you can hook up to an ASR/Intent processor that definately will be too much for the ESP.
So after KWS hit you broadcast a ASR sentence until detected silence and also send the KW hit value data so you can pick from a distributed array of KWS mics and select the best audio as its just the highest or only KW hit.

So fingers crossed I might triggered your curiosity :)

I am thinking this is almost the code needed minus delay but still scratching my head.

https://github.com/FrankBoesing/Teensy-Audio-Effect-Remove-Vocals

The delay is minimal due to sampling rate and mic distance guess its a small ring buffer.

[StuartIanNaylor]

Gary slowly get more info.

Going back to https://invensense.tdk.com/wp-content/uploads/2015/02/Microphone-Array-Beamforming.pdf

Frequency Response
The differential array beamformer does not have a flat frequency response, but rather has a high-pass filter response characteristic
up to the null frequency. The response of a first-order beamformer (two microphone elements) rises with frequency at 6 dB/octave
and flattens above the aliasing frequency.

Is it possible on the ESP32 to apply low pass gain to correct the above so there is a flat frequency response?

[garygru]

Yeah, frequency response is a mess compared to an omnidirectional mic. I think you need a first order shelf to compensate for the low frequency loss. Nevertheless this also boosts noise!
Currently only second order (12db/oct.) pass filters are implemented in yummyDSP but (first order) shelf and peak filters are just a matter of work

[StuartIanNaylor]

Yeah I just had a go with a Pi and Python and expected my aversion to code nowadays and rustiness to be the problem but for some reason my idea all goes to hell when I try to play through a snd_loopback adapter so I can the other side as a source.

A single but (first order) is your filter not just a stacked first order? I guess you could run a low pass filter from the low cut off and then apply uniform gain but attenuating to add gain will garner noise like you say.

Its interesting though as the adafruit mems should be able to do 64Khz and that might even allow 3rd/4th order endfire arrays but haven't a clue to if the frequency response follows the filter response.
I wonder if that is why in the app note example the went with a fairly low null notch of 12Khz as that is 6 octaves to 100hz.

Thinking about it that is quite a lot of gain to flatten at the lowest frequencies but after equalisation but it will be interesting to how it sounds.
After all this it might be better going old school and getting electrets and maybe a better quality ADC that onboard as so far from what I have read it seems to be noisy.

Have you had a look at the MASE stuff in the ADF as think that is just software but just not opensource which is a shame but still indicative the esp32 can do more advanced beamforming than just simple endfires as don't think its uses the codecs for that.
How you setup custom boards is confusing but you can for the ADF.
https://github.com/espressif/esp-sr/tree/390c17b99b7cb54f39128eaf355cf16b2a322dbf/acoustic_algorithm

Still can not work out if or not hardware codec based...

PS have you had a play with any of the more advanced dsp beamforming techniques?