Pauses during streaming

[y-nk]

Hello,

It's a long shot but i'm looking into debugging my issues. My main goal is to find how to debug (rather than plain answers).

I started out with this:

minimal sketch using MP3DecoderHelix

#include <SPI.h>
#include "AudioTools.h"
#include "AudioLibs/A2DPStream.h"
#include "AudioLibs/AudioSourceSD.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "AudioCodecs/CodecWAV.h"


#define HSPI_SCLK 26
#define HSPI_MISO 33
#define HSPI_MOSI 25
#define HSPI_SS 27

const char *startFilePath = "/";
const char *ext = "mp3";

SPIClass hspi(HSPI);
AudioSourceSD source(startFilePath, ext, HSPI_SS, hspi);
A2DPStream out;
MP3DecoderHelix decoder;
AudioPlayer player(source, out, decoder);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  hspi.begin(HSPI_SCLK, HSPI_MISO, HSPI_MOSI, HSPI_SS);

  // setup output
  auto cfg = out.defaultConfig(TX_MODE);
  cfg.name = "Bose Color II SoundLink";
  cfg.auto_reconnect = false;
  out.begin(cfg);

  // setup player
  player.begin();
  out.setVolume(100);
}

void loop() {
  player.copy();
}

This compiles well, connects to BLE and plays the file with approx 200ms gaps of silence between each chunk of sound. I thought that maybe the issue would be with my board not powerful enough to decode the mp3 on the fly (no PSRAM) so I thought to swap to wav (the SD card being big enough, it doesn't seem like a problem to me).

From the sketch below i literally only swapped MP3DecoderHelix to WAVDecoder and mp3 to wav.

minimal sketch using WAVDecoder

#include <SPI.h>
#include "AudioTools.h"
#include "AudioLibs/A2DPStream.h"
#include "AudioLibs/AudioSourceSD.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "AudioCodecs/CodecWAV.h"


#define HSPI_SCLK 26
#define HSPI_MISO 33
#define HSPI_MOSI 25
#define HSPI_SS 27

const char *startFilePath = "/";
const char *ext = "wav";

SPIClass hspi(HSPI);
AudioSourceSD source(startFilePath, ext, HSPI_SS, hspi);
A2DPStream out;
WAVDecoder decoder;
AudioPlayer player(source, out, decoder);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  hspi.begin(HSPI_SCLK, HSPI_MISO, HSPI_MOSI, HSPI_SS);

  // setup output
  auto cfg = out.defaultConfig(TX_MODE);
  cfg.name = "Bose Color II SoundLink";
  cfg.auto_reconnect = false;
  out.begin(cfg);

  // setup player
  player.begin();
  out.setVolume(100);
}

void loop() {
  player.copy();
}

This also compiles well, connects to BLE and the logs show good amount of data being dump through the player, although no sound is being played.

As mentioned in another issue, I've even tried to swap to different mp3 codecs, and also different bitrates leading to interesting results (mostly panic - 64kbps file with Helix did it too)

I don't know if:

• it's my use of VSPI with custom pins (i've swapped with hspi)
• it's the sd card reader i'm using (i've swapped with 1 other)
• it's the sd card i'm using (i've swapped with 3 others)
• it's the files i'm using (i've a bunch of them)

I'm sorry to ask of your time for this, but would you have any pointers to help resolve this?

[pschatzmann]

Usually no PSRAM should be necessary.

For MP3 the sd speed is usually not any problem, but it is more likely when using uncompressed data with WAV:
Did you test if the audio is ok with anouther output: If you don't have an external I2S DAC, you can use the internal DAC of the ESP32 with the AnalogAudioStream and test it with some earphones:

The output goes to the DAC pins GPIO25 (Channel 1) and GPIO26 (Channel 2). Just connect GND on one of the channels with the earphones to check if it is breaking up there as well.

If this is working then the issue is most likely on the A2DP side, which is sometimes quite challaging for this scenario:
You can test the A2DP side with some sine tests:

• basic-generator-a2dp to test the basic functionality with a callback
• streams-generator-a2dp to test the stream implementation using a queue

I committed today some small change to the A2DPStream that gives you access to the buffer information: e.g with out.buffer().levelPercent() you can monitor the level of the buffer e.g. in the loop.

[pschatzmann]

Try to increase the SPI frequency: This can be indicated in the constructor of the AudioSourceSD

[y-nk]

I've already tried setting it to 80Mhz using SPI.setFrequency(80000000); but no changes to these pauses.

I don't think we can pass the SPI frequency setting directly in AudioSourceSD constructor, but thanks to our previous work we can pass an SPI instance to it, so this should be enough.

EDIT: well well well. I wanted to follow your advice and double down on this.

I don't have pauses anymore (at least on AnalogAudioStream). What changed:

Since I needed to test the analog output, I changed the custom SPI from:

-#define HSPI_SCLK 26
+#define HSPI_SCLK 27
-#define HSPI_MISO 33
+#define HSPI_MISO 32
-#define HSPI_MOSI 25
+#define HSPI_MOSI 33
-#define HSPI_SS 27
+#define HSPI_SS 14

...where all I can see is that I'm not using the DAC pins as GPIO anymore.

also (and probably the easiest to witness):

-AudioLogger::instance().begin(Serial, AudioLogger::Info);
+AudioLogger::instance().begin(Serial, AudioLogger::Debug);

I can easily compile and toggle the log level and witness the pauses in Debug but not in Info / Warning.

I will be swapping back to A2DP now, see if it carries on there too. There's no reason why not.

[y-nk]

Currently debugging with those sketches:

🟢 Using `AnalogAudioStream`, has no pause during play

#include <SPI.h>
#include "AudioTools.h"
#include "AudioLibs/A2DPStream.h"
#include "AudioLibs/AudioSourceSD.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "AudioCodecs/CodecWAV.h"


#define HSPI_SCLK 27
#define HSPI_MISO 32
#define HSPI_MOSI 33
#define HSPI_SS 14

#define VSPI_SCLK 18
#define VSPI_MISO 21
#define VSPI_MOSI 19
#define VSPI_SS   5

const char *startFilePath = "/";
const char *ext = "mp3";

SPIClass hspi(HSPI);

AudioSourceSD source(startFilePath, ext, HSPI_SS, hspi);

AnalogAudioStream out;

MP3DecoderHelix decoder;
AudioPlayer player(source, out, decoder);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  hspi.begin(HSPI_SCLK, HSPI_MISO, HSPI_MOSI, HSPI_SS);
  hspi.setFrequency(80000000);

  auto config = out.defaultConfig(TX_MODE);
  out.begin(config);

  player.begin();
}

void loop() { player.copy(); }

🔴 Using `A2DPStream`, has pauses during play

#include <SPI.h>
#include "AudioTools.h"
#include "AudioLibs/A2DPStream.h"
#include "AudioLibs/AudioSourceSD.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "AudioCodecs/CodecWAV.h"


#define HSPI_SCLK 27
#define HSPI_MISO 32
#define HSPI_MOSI 33
#define HSPI_SS 14

#define VSPI_SCLK 18
#define VSPI_MISO 21
#define VSPI_MOSI 19
#define VSPI_SS   5

const char *startFilePath = "/";
const char *ext = "mp3";

SPIClass hspi(HSPI);

AudioSourceSD source(startFilePath, ext, HSPI_SS, hspi);

A2DPStream out;

MP3DecoderHelix decoder;
AudioPlayer player(source, out, decoder);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  hspi.begin(HSPI_SCLK, HSPI_MISO, HSPI_MOSI, HSPI_SS);
  hspi.setFrequency(80000000);

  // setup output
  auto config = out.defaultConfig(TX_MODE);
  config.name = "Bose Color II SoundLink";
  config.auto_reconnect = true;
  out.begin(config);

  player.begin();
}

void loop() { player.copy(); }

No need to play the 7 differences game, the only thing which changes is the out variable being an A2DPStream or a AnalogAudioStream (and related config, obviously)

Looking at the Serial prints being almost "in sync" with the pauses of the stream, I'm going to remove them and see the effect. ← Doesn't work either.

[y-nk]

I've added the logs of the buffer percent as you suggested and it shows a distinct pattern, although i'm not really sure that this shows a problem or rather a normal "pumping" behavior of the buffer.

void loop() {
  Serial.print("[");
  Serial.print(millis());
  Serial.print("] buffer=");
  Serial.println(out.buffer().levelPercent());  

  player.copy();
}

We can see an alternate of buffer being (almost) full then half.

[2591] buffer=30.00
[2600] buffer=89.99
[33979] buffer=99.99
[34079] buffer=50.00
[2591] buffer=30.00
[2600] buffer=89.99
[33979] buffer=99.99
[34079] buffer=50.00
[34158] buffer=89.99
[34252] buffer=99.99
[34352] buffer=50.00
[34431] buffer=89.99
[34520] buffer=99.99
[34620] buffer=50.00
[34700] buffer=89.99
[34790] buffer=99.99
[34890] buffer=50.00
[34910] buffer=93.33
[35059] buffer=99.99
[35159] buffer=50.00
[35178] buffer=93.33
[35331] buffer=99.99
[35431] buffer=50.00
[35450] buffer=93.33
[35603] buffer=99.99
[35703] buffer=50.00
[35722] buffer=93.33
[35871] buffer=99.99
[35971] buffer=50.00

As a table

millis()	diff with n-1	buffer size
2591	X	30.00
2600	9	89.99
33979	31379	99.99
34079	100	50.00
34158	79	89.99
34252	94	99.99
34352	100	50.00
34431	79	89.99
34520	89	99.99
34620	100	50.00
34700	80	89.99
34790	90	99.99
34890	100	50.00
34910	20	93.33
35059	149	99.99
35159	100	50.00
35178	19	93.33
35331	153	99.99
35431	100	50.00
35450	19	93.33
35603	153	99.99
35703	100	50.00
35722	19	93.33
35871	149	99.99
35971	100	50.00

I've no clue if the buffer is behaving as it should. But we can see clearly some loop() iterations being way slower than others.

[pschatzmann]

The buffer is filled by the decoder and is there to make sure that the A2DP callback always has some data to send. So it is a good thing to see that it never runs empty.

You could try to decrease the size to make RAM available for the other processes.

[pschatzmann]

I was just running a test with ESP32 3.0.5.
I made sure that the

• CPU frequency was set to it's max value: 240 MHz and
• the PSRAM was disabled and
• the Partition Size was at Huge APP.
• Arduino Core debug level was to Warning to see any A2DP warnings and errors
• Log level was set to Warning to prevent any unnecessary delays

I did not notice any breakups. Then I realized that I was using SDFAT instead of SD!

After using AudioSourceSD the sketch started to crash outside of my code...

[pschatzmann]

I could trace down the exception to running out of RAM: decreasing the buffer resolved the issue with the exception.
With the reduced buffer, I can reproduce your issue with AudioSourceSD.

I suggest you change to AudioSourceSDFAT...

[y-nk]

Thanks a lot for taking the time to try on your own, I really appreciate it. I will try to move to SDFAT and let you know the results.

If it works, I will see if i need PR AudioSourceSDFAT to include a custom spi setting (similarly to AudioSourceSD), although I believe it may look different as this library has different API.

[pschatzmann]

There the frequency is in the constructor which I did not change for my tests.
I think you can define the port via SdSpiConfig

[pschatzmann]

ps. I just activated PSRAM and with that AudioSourceSD is also working fine.
So it seems to be an issue with the limited RAM.

[y-nk]

I will try to find a board with PSRAM to make some tests as well.