audio over espnow: data transfer rate ?

[mkoenings]

Hello,
I am trying to use the ESPNowStream to transport audio data ( from the audiokit codec chip via I2S) to a receiver.
I am using the communication example code with the measuring stream. I read in your fantastic docs that you could expect a data rate of about 33000 to 37000 bytes per second, which matches the numbers I got. So we can transfer 8000 samples/sec without encoding with this rate.
There is a FAST_ESP_NOW_HACK macro in the communication.h, which allows to change the data rate of the ESP32.
Looking at these numbers https://blogs.ntu.edu.sg/ps9888-2020-g16/2020/07/11/esp-now-data-transfer/ I was wondering if this could allow to get higher data rates in the examples, by enabling the hack, and setting it to a higher value, like listed here
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_wifi.html#_CPPv415wifi_phy_rate_t
I can sucessfully enable the higher rate on transmitter and receiver ( info message is printed) , but there is no effect, the data transfer rate remains the same. What could be the issue here?
Thank you!

[pschatzmann]

Unfortunately I did not manage to have working either.
I think that a config setting is required, so we can't really use this in Arduino.
https://www.esp32.com/viewtopic.php?t=12781

[mkoenings]

I could get a better rate by replacing the API call

esp_wifi_internal_set_fix_rate(getInterface(), true, cfg.rate)
with
esp_wifi_config_espnow_rate(getInterface(), cfg.rate)
as described here.
I can see that the throughput is better with higher rate settings ( using the receive-measure example) , the highest rating makes the transmitter crash. Also the numbers are not going up linearly, I am not sure if maybe the measurement routine is too slow ? But I can transmit 22050 samples per sec without compression with WIFI_PHY_RATE_MCS4_LGI! I will try other settings,
There is some buzzing static noise though (on top of the quite clear transmission ) , which can not be removed by increasing the buffers. This is the next problem to tackle... :-)

[pschatzmann]

Thanks for your feed-back. I have updated my code to use your proposed logic...

[mkoenings]

please note that the new procedure has only 2 parameters. line 124 will not compile.

[aqildad-create]

Thanks for your feed-back. I have updated my code to use your proposed logic...

what was data rate before using proposed logic ? and what was data rate after proposed logic ?

I am getting around 330/=sample per seconds so 330x250 bytes = 82,500 bytes per seconds without proposed logic with out your esp api ? Is this normal ?

I notice with sbc encoder at receiver when ever sample rate decrease from 330 to 300 samples then audio distorted . and sbc encoder at receiver need around 330 sample to run quality audio .. Is this normal ?

also sample rate changes from 330 sample to 300 sample to audio quality is changing accordingly . Is this normal ?

If the sample rate further decrease to 250 samples some time then audio decrease more .. so how to handle this . buffer is not a solution because if sample decrease from 300 sample even if their is buffer still audio is distorted . how to mange this ?

and why some time esp date rate decreased ? Is this normal ?

How to get high samples rate ?

[mkoenings]

I could manage to transfer up to 48Khz uncompressed sample rate, with good quality with a physical rate of 54 Mbits . Unfortunately the transmission of audio suffers from small errors in the transmission which create loud crackling/popping noises here and there. Enlarging buffers helps with this issue, but i could not make it go away 100% ( also the latency goes up) . I am wondering how "professional" audio transmission systems deal with this? Is there a "simple to use" error correction functionality out there ? ( I could not find anything, that I could use in the arduino environment). Since I only need mono transmission, maybe the second audio channel could be used for this ? The same way symmetric cabeling works: removing static noise by inverting and re-inverting the phase which cancels out the noise. In terms of samples, the second channel could be used to fix errors in the first channel? Any Ideas are appreciated! Thanks

[pschatzmann]

I suggest that you try to use a codec.
I would guess that the ones used by bluetooth might be a good choice: sbc, aptx or lc3

[mkoenings]

I tried the LC3 but with strange sounding results. I will try the aptx as well. Thanks! Am Fr., 14. Okt. 2022 um 10:36 Uhr schrieb Phil Schatzmann < ***@***.***>:

…

I suggest that you try to use a codec. I would guess that the ones used by bluetooth might be a good choice: aptx or lc3 — Reply to this email directly, view it on GitHub <#393 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADMD7IWKLVLDD4MSAYYZEDWDELPNANCNFSM6AAAAAAQ4NNNIM> . You are receiving this because you authored the thread.Message ID: <pschatzmann/arduino-audio-tools/repo-discussions/393/comments/3877087@ github.com>

[mkoenings]

I cant make the codecs work: the aptx create a lot of crackling noise, sounds like buffer underuns. I tried increasing the buffers but with no luck. Lc3 creates a very thin and hollow sounding output, not crackling but with drop-outs.
I wonder where in the code I can optimize this, the same sketches for send and receice work ok (not perfect, but with a steady sound) , without the codecs. I also tried starting out from the communication-codec-espnow samples, simply replacing the SPC codec with APTX ald LC3 , but no luck. Any Ideas?
Thanks!

[mkoenings]

I have very good results with the LC3 now! it turns out reading the manual is helpful sometimes :-)
https://www.bluetooth.com/specifications/specs/low-complexity-communication-codec-1-0/ there is a pdf.
It describes the input and output buffer sizes in the tables. I could make it work in 48khz , 16 bit, single channel with buffer sizes:
Input 720 bytes (360 samples), payload 250 bytes ( the espnow payload size) .
Thanks again for making it so easy to code with your libraries!

[jajito]

Hi mkoenings.
And hi Phil and tjanks for your marvellous work in thieese libraries.
I'm trying to configure LC3, but only get cracks.
How do you pass this parameters (specially the buffer sizes) to LC3?
Could you send some code?
Thanks!

[mkoenings]

please read the doc on Lc3: https://www.bluetooth.com/specifications/specs/low-complexity-communication-codec-1-0/ In the first pdf, there is a table ( section 2.2 table 2.1 ) Please note that 44Khz is not supported in this library. I used 48kHz, 16bit, 1 channel, 720byte stream buffer ( 360x2) for the i2S, I used 8 buffers with 90 bytes size but I can't share the code unfortunately. Good luck Michael Am Sa., 22. Apr. 2023 um 13:34 Uhr schrieb jajito ***@***.***

…

: Hi mkoenings. And hi Phil and tjanks for your marvellous work in thieese libraries. I'm trying to configure LC3, but only get cracks. How do you pass this parameters (specially the buffer sizes) to LC3? Could you send some code? Thanks! — Reply to this email directly, view it on GitHub <#393 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADMD7NTRT6TFZ4EUFAYODDXCO64VANCNFSM6AAAAAAQ4NNNIM> . You are receiving this because you authored the thread.Message ID: <pschatzmann/arduino-audio-tools/repo-discussions/393/comments/5694663@ github.com>

[jajito]

Thanks for the fast reply.
I am doing this in setup():

 auto kit_cfg = kit.defaultConfig(RXTX_MODE);
  kit_cfg.sd_active = false;
  kit_cfg.input_device = AUDIO_HAL_ADC_INPUT_LINE2; // linea
  //kit_cfg.input_device = AUDIO_HAL_ADC_INPUT_LINE1; //micros
  kit_cfg.bits_per_sample = 16;
  kit_cfg.channels = 1;
  kit_cfg.sample_rate = 48000;
  kit_cfg.buffer_count = 8;
  kit_cfg.buffer_size = 90;
  kit.begin(kit_cfg);
  //kit.setVolume(100);
  es8388_set_mic_gain(MIC_GAIN_0DB );
  
  //start esp-now
  auto now_cfg = now.defaultConfig();
  now_cfg.mac_address = "A8:48:FA:0B:93:02";
  now.begin(now_cfg);
  now.addPeers(peers);
  
  // start encoder
  auto enc_cfg = encoder.defaultConfig();
  enc_cfg.channels = 1;
  enc_cfg.sample_rate = 48000;
  enc_cfg.bits_per_sample = 16;
  encoder.begin(enc_cfg);

But I don't know how do I pass the 720byte stream buffer

[jajito]

And this is the configuration of LC3encoder, espnow and the Lyrat.

AudioKitStream kit; // Access I2S as stream
ESPNowStream now;
EncodedAudioStream encoder(&now, new LC3Encoder()); // encode and write to ESP-now
StreamCopy copier(encoder, kit);  // copies sound into i2s

[mkoenings]

ltry like this: StreamCopy copier(encoder, kit, 720); Am Sa., 22. Apr. 2023 um 17:24 Uhr schrieb jajito ***@***.***

…

: AudioKitStream kit; // Access I2S as stream And this is the configuration of LC3encoder, espnow and the Lyrat. ESPNowStream now; EncodedAudioStream encoder(&now, new LC3Encoder()); // encode and write to ESP-now StreamCopy copier(encoder, kit); // copies sound into i2s — Reply to this email directly, view it on GitHub <#393 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADMD7K6FJPJAZENBZRH3XTXCPZ4HANCNFSM6AAAAAAQ4NNNIM> . You are receiving this because you authored the thread.Message ID: <pschatzmann/arduino-audio-tools/repo-discussions/393/comments/5695690@ github.com>

[jajito]

Thanks, but not working...

[pschatzmann]

I just added an example with an SBC codec and it was working at the first trial...
I would be surprised if the LC3Encoder would be the issue here....
In the meantime Espressif has officially released the esp_wifi_config_espnow_rate(), so you can define the rate now in the config w/o the need to activate the FAST_ESP_NOW_HACK

[jajito]

Hi Phil, congrats for your titanic job.
I could make the SBC codec example work (when it was in the sandbox section), with no problem.
Not only with the sinewave example, but with an audio live source from the Lyrat. It's fine.
The scheme is line-> coder ->espnow->decoder->i2s audio out with another ESP32.
It works with the SBC codec, perfect.
I am trying to transmit audio in real time (one channel) to try to make a wireless transmitter for an in-ear monitor system.
And the SBC codec works, but the latency is too big for that purupose.
I have reduced the number of buffers and it's size, as Mark Koenigs explained, and there is less lag, but with real live sound there are crackles and drop outs. With 4 buffers of 256, the sound is acceptable, and the lag in that situation is high enough to not to be useful as a real time audio transmission.
I suppose that I am trying to get more of what the ESP32 can deliver, but I was trying other codecs to see if the lag was smaller and could make the live transmitter.
So I tried the aptx codec and only got random craks not related to the audio input.
An tried the LC3 codec, and get craks related to the audio input, but not real audio.
I'm sure that this is not an issue of your library or the LC3encoder(your job is flawless), but I could not manage to configure to work as I did with the SBC.
Thanks again for your work, and congratulations!

[jajito]

Hi Phil.
I have tried several of the codecs tests of your "tests/codecs" directory with my Lyrat.
The SBC codec works, but I could not make work the aptx and the lc3.
Only cracks.

[pschatzmann]

I committed some corrections: #809

[pschatzmann]

I think the following might be worth to consider:

• optimize the i2s buffer configuration
• optimize the ESPNowStreamConfig
  • optimize the ESPNowStream buffer configuration
  • try a higher rate
  • reduce the delays
• optimize the copy size

Not sure if this really helps, but I would try to use a consistent buffer size which would be the size (or a multiple) of an encoded packet.

As you have noticed, any buffer is adding a lag. However it is critical to have enough data in the buffer to prevent any buffer underflow which will cause artifacts

[aqildad-create]

Phil sir your lib is fantastic.

I have a question does your ESPNowStream have buffer management for cracking sound ??

I notice without your espnow class data transfer vary and some time low as half of average transfer rate which usually I get .is this normal ? and this cause sound cracks with live writes so with your ESPNowStream audio will also cracks when data transfer rate become low ?? or you have management for this ? how much buffers you use to avoid cracks ??

how much approx. data transfer usually vary with espnow ?

[pschatzmann]

ESPNowStream is storing the data in a buffer on the receiving side: I don't think buffer overflows are possible because of the synchronization/confirmtation logic. However there is the risk of buffer underflows if you can not transmit the data fast enough.

So carefully select the sampling rate and the buffer size to prevent underflows.

[aqildad-create]

I observe with espnow some time data send fail and can confirm that data send fail with acknowledgment . so in this scenario if buffer is at receiver and data send fail then You send that data again or you ignore the fail data and send new fresh packets ?

I observe with espnow some time half of data send fail and some time all data send success . . so how have deal this within your library ? Message ID:

…

<pschatzmann/arduino-audio-tools/repo-discussions/393/comments/5711575@ github.com>

[pschatzmann]

Did you ever look at the available ESPNowStreamConfig parameters ?

[aqildad-create]

I go through the documentation and cant see parameter related to data send falled , is it will send data again and if yes how many attempt it will send again if data send loss ?

and also in the mean time data coming from i2s stream will store in the buffer and this will go on until data success send to espnow again and then it will fast send data until buffer in the sender become empty.

i am searching for these parameter in ESPNowStreamConfig class . pls help and guide .

[pschatzmann]

• delay_after_failed_write: obviously defines the delay after a failure
• write_retry_count: obviously defines how many trials: -1 means endless, 0 means no retry, and a positive value the number of retries

[aqildad-create]

Thanks and apology for miss reading . You used Default delay after success write is 2 ms ? Why it is required ? Why no delay after success write ? What is its advantages ? And which buffer data will be store if espnow write failed ? What if this buffer overflow ? What is maximum size of this buffer ? also i notice with espnow when i cover espnow around with hand or box then data transfer of espnow reduce allot . did you observe the same ? If yes how you handle this ? further i don't have latency issue so what are your recommended parameter for this scenario ?