Struggling to play audio over Bluetooth if there's any delay in the main loop

[clementprdhomme]

Hello,

I have a Feather ESP32 v2 (ESP32-PICO-MINI-02) which I use to do the following:

• Read a MP3 file from an SD card breakout connected over SPI with SDFat at 16 MHz
• Play the audio over Bluetooth using BluetoothA2DPSource.h via callbacks
• Drive a 128x128 greyscale display using the same SPI with u8g2 at 40 MHz

The MP3 file I'm testing with is this one. I reduced its bit rate to 128 kb/s because I couldn't manage to play it at 256 kb/s (original) despite setting those two variables:

#define MP3_MAX_OUTPUT_SIZE 2048 * 5
#define MP3_MAX_FRAME_SIZE 3200

If I'm not sending the pixel buffer to the screen, the audio plays nicely. If I do, the audio stutters more than once a second. The buffer takes in total 6 ms to send.

Below is a simplified sketch that reproduces the issue I'm describing. In the main loop, I added a 6 ms delay to simulate the screen.

#include <SdFat.h>
#include <AudioTools.h>
#include <AudioLibs/AudioA2DP.h>
#include <AudioCodecs/CodecMP3Helix.h>

#define SD_CONFIG SdSpiConfig(13, SHARED_SPI, SD_SCK_MHZ(16))

SdFs sd;
FsFile _file;

char* btDeviceName = "Polk Boom Swimmer Duo";
char* fileName = "fun-life-128.mp3";
int buffer_count = 30;
int buffer_size = 512;
AudioSourceCallback source = AudioSourceCallback();
MP3DecoderHelix decoder;
SynchronizedNBuffer<uint8_t> buffer(buffer_size,buffer_count, portMAX_DELAY, 10);
QueueStream<uint8_t> out(buffer);
AudioPlayer player(source, out, decoder);
BluetoothA2DPSource a2dp;

bool sdMounted = false;

bool openFile(String path, FsFile& file) {
  if (!sd.exists(path)) {
    return false;
  }

  file = sd.open(path);
  return true;
}

bool closeFile(FsFile& file) {
  if (!file.isOpen()) {
    return false;
  }

  file.close();
  return true;
}

Stream* onGetNextStream(int offset) {
  if (_file.isOpen()) {
    closeFile(_file);
  }

  FsFile newFile;
  if (openFile(fileName, newFile)) {
    _file = newFile;
    return &_file;
  }
  
  return nullptr;
}

int32_t getAudioData(uint8_t *data, int32_t bytes) {
   size_t result_bytes = buffer.readArray(data, bytes);
   return result_bytes;
}

void setup() {
  Serial.begin(115200);

  while(!sdMounted) {
    delay(100);
    if (sd.cardBegin(SD_CONFIG)) {
      if(sd.volumeBegin()) {
        sdMounted = true;
      }
    }
  }

  Serial.println("SD card mounted");

  source.setCallbackSelectStream(&onGetNextStream);

  out.begin();

  player.setDelayIfOutputFull(0);
  player.setVolume(0.1);
  player.begin(); 
  player.getStreamCopy().copyN(5); 

  a2dp.start_raw(btDeviceName, getAudioData);  
}

void loop() {
  delay(6);
  player.copy();
}

I have read the wiki multiple times and searched issues and discussions intensively but still struggle to see how I could mitigate the impact of this delay. Is there any setting I can tweak to play smooth audio? Was this library conceived to be the only thing running on the ESP32?

On this wiki page it is written:

So far I never needed to use any of the ESP32 multicore functionality because all the examples are very simple and everything is fitting on a single core. However if you write some more complex scenarios which are e.g. updating a screen, process user input w/o impacting the audio, you should consider to use multiple tasks.

Is there any example from anyone else of such a configuration? How does that improve performance if the second core is reserved to Wifi/Bluetooth communication?

Thank you very much for your advice.

[pschatzmann]

From my point of view, running a bluetooth source and mp3 decoding is bringing the EP32 quite to the limit so adding some more processing is challenging: I would suggest to investigate the impact of adding an additional task doing SPI Display output on core 1 or core 2.

You can use the FreeRTOS API directly or use this library: https://www.pschatzmann.ch/home/2022/11/06/freertos-addons-a-c-api-for-freertos/

Another thing you can look into is running the decoding on the Wifi/Bluetooth core. I think this would help because you can get rid of the synchronization but I am not sure if mp3 decoding is causing too many delays. I expect that it would get much easier however if you switch so a more efficient audio format: e.g. PCM WAV (or ADPCM WAV if you prefer a compressed format) .

[clementprdhomme]

Thank you so much for all your answers! I have quite some bit of research ahead!

I'll come back when I have done some experiments. Hopefully that can be helpful to someone else.