Assistance Needed for Streaming and Playing MP3 from SPIFFS over I2S on ESP32

[bergamifer]

Hello everyone,

I’m Fernando from Argentina and I'm currently working on an artistic project with the ESP32 where I intend to download an MP3 file from the internet, save it to the SPIFFS (Serial Peripheral Interface Flash File System), and then play it using I2S (Inter-IC Sound) interface.

Intended Functionality:
Download the macmiller.mp3 file from the web.
Save it to the SPIFFS.
Play the audio file through an I2S interface.

Error message i get on serial monitor:
rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1344
load:0x40078000,len:13964
load:0x40080400,len:3600
entry 0x400805f0
ets Jul 29 2019 12:21:46

What could be the cause of these resets and how can I troubleshoot them?
What modifications should I make to ensure the I2S setup and MP3 playing are correctly integrated with the file download part?
Is there any consideration to take into account regarding memory management when working with SPIFFS and I2S simultaneously?
Any insights, code examples, or resource recommendations would be highly appreciated. I am happy to provide more code details or error logs if needed.

Thank you in advance for your time and assistance.

Here's the code:

#include <Arduino.h>
#include <HTTPClient.h>
#include <SPIFFS.h>
#include <WiFi.h>

#include "AudioTools.h"
#include "AudioLibs/AudioSourceSPIFFS.h"
#include "AudioCodecs/CodecMP3Helix.h"

const char* ssid = "sssssssssss";
const char* password = "ppppppppp";
const char* url = "https://plantaalta.chela.org.ar/domosonica/macmiller.mp3";
const char* path = "/macmiller.mp3";

I2SStream i2s;
MP3DecoderHelix decoder;
AudioPlayer *player = nullptr; // Puntero al reproductor para poder controlarlo después de setup

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);
  
  // Iniciar SPIFFS
  if (!SPIFFS.begin(true)) {
    Serial.println("An Error has occurred while mounting SPIFFS");
    return;
  }
  
  // Conectar a WiFi
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi...");
  }
  Serial.println("Connected to WiFi");

  // Descargar el archivo MP3
  HTTPClient http;
  http.begin(url);
  int httpCode = http.GET();
  
  if (httpCode > 0) {
    if (httpCode == HTTP_CODE_OK || httpCode == HTTP_CODE_MOVED_PERMANENTLY) {
      File file = SPIFFS.open(path, FILE_WRITE);
      if (!file) {
        Serial.println("Failed to open file for writing");
        return;
      }
      // Escribir archivo en SPIFFS
      http.writeToStream(&file);
      file.close();
      Serial.println("Download finished");
    }
  } else {
    Serial.printf("HTTP GET... failed, error: %s\n", http.errorToString(httpCode).c_str());
    http.end();
    return;
  }
  http.end();

  // Configuración de I2S
  auto cfg = i2s.defaultConfig(TX_MODE);
  cfg.sample_rate = 44100;
  cfg.bits_per_sample = 16;
  cfg.channels = 2;
  cfg.pin_bck = 27;
  cfg.pin_ws = 33;
  cfg.pin_data = 32;
  i2s.begin(cfg);

  // Crear una fuente de audio y el reproductor
  AudioSourceSPIFFS source(path);
  player = new AudioPlayer(source, i2s, decoder); // Usamos new para crear una instancia dinámica

  // Iniciar la reproducción
  player->begin();
  Serial.println("Playback started");
}

void loop() {
  if (player && *player) {
    player->copy(); // Copiar datos de la fuente de audio al I2S
  }
}

`

[pschatzmann]

https://github.com/pschatzmann/arduino-audio-tools/wiki/It's-not-working

[bergamifer]

Dear Phil Schatzmann,

Thank you very much for your response and for the guidance provided. I appreciate the time you've taken to outline the potential issues and how to address them.

I wanted to share with you that I have already visited the generic troubleshooting page you referenced, and while it contains valuable information for many scenarios, I seem to be facing a more specific challenge that isn't directly addressed there.

My current issue revolves around integrating SPIFFS file downloading code with I2S playback code. I am able to run each piece of code successfully on its own, but the difficulty arises when attempting to combine them into a cohesive program that downloads an MP3 file to the SPIFFS and then plays it via I2S.

I am using the most up-to-date version of the Arduino core for ESP32 and have taken care to ensure the proper formatting of the SD, the accuracy of the WiFi credentials, and the consistency of my sketch. Additionally, I have adjusted the log level as you suggested to get as much information as possible.

However, when I attempt to compile and run the combined code, the ESP32 continuously resets without reaching the "Connecting to WiFi..." or "Download finished" messages that would indicate the downloading process is even starting.

I understand the complexities of working with integrated systems and how vital it is to approach troubleshooting methodically. I've been incrementally building up my code, testing each component, and ensuring each part works before integrating.

I'm reaching out in hopes of some specific guidance on how to successfully merge these functionalities, or if there's an alternative approach to achieving this task that I may have overlooked. Your expertise and any further insights you can provide would be greatly appreciated.

Thank you once again for your support and for the work you do in maintaining such valuable resources for the community.

Best regards,

Fernando

[pschatzmann]

Why don't you play it directly or you could play it and download it at the same time ?
E.g. Take one of my url playing examples and sent the output to a MultiOutput that consists of your file and the I2SStream.
The thing that seems strange to me is that you put all your logic into the setup...

Or you could take my file download example as a starting point...

[bergamifer]

Hi Phil,
Thank you for your response and the suggestions provided. I have been experimenting with the stream_url_mp3 (helix and mad) examples, but unfortunately, I encountered some issues with streaming breaks that I couldn't resolve despite adjusting the buffer sizes and other parameters. The logs are showing continuous copying of bytes, but the breaks still occur, which seems odd:

[I] StreamCopy.h : 139 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops
[I] StreamCopy.h : 139 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops
[I] StreamCopy.h : 139 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes - in 1 hops
[I] StreamCopy.h : 139 - StreamCopy::copy 832 -> 832 -> 832 bytes - in 1 hops          <<<<<<<  the audio fails here
[I] StreamCopy.h : 139 - StreamCopy::copy 1024 -> 1024 -> 1024 bytes

On another note, the context of my project involves working with 16 ESP32 boards simultaneously, and in a subsequent phase, I will need them to be synchronized with each other at a sample by sample rate. This is one of the reasons I'm focusing on downloading files rather than streaming — to avoid the latency that WiFi connections might introduce.

I understand that this adds a layer of complexity and there might be some limitations to what I can achieve, but I am hoping to make it work with your help. Any further guidance or advice you could offer would be greatly appreciated.

Thank you for your time and assistance!

[pschatzmann]

Looks more like a network (infrastructure) issue..

[bergamifer]

Dear Phil,

Thank you for your input. I’ve conducted tests with the online radio example, and it works seamlessly without any transmission errors, which suggests that the network infrastructure is stable and reliable. However, the streaming issues persist when implementing the stream_url_mp3 function. Despite adjusting the buffer sizes, I haven’t been able to eliminate the streaming breaks.

Given that the network appears to be functioning correctly with other examples, could there be another factor at play specific to the stream_url_mp3 implementation? Additionally, considering the project’s scale with 15 ESP32 boards running concurrently, I am aiming to have them synchronized while playing downloaded files to avoid latency issues associated with Wi-Fi connections.

Any insights or suggestions you might have to help resolve these streaming challenges or advice on better synchronizing the playback across multiple devices would be greatly appreciated.

Best regards,
Fernando

[pschatzmann]

What kind of test did you do exactly ?
If you need all esp32 playing the same audio, I suggest to look into Snapcast

[bergamifer]

Hi Phil, thank for your responses, I've had very busy days.

Today I'm all in with this project and I have to say that it seems that all of the examples regarding mp3/flac/raw stream are not working. I'm uploading them to an ftp server and when I play it from my phone or other computers I can listen to the files ok. But when I'm streaming from the boards it seems to have small pauses. I tried changing buffer sizes with no better results. Is the code ok? I'll try a local server and see if it works better but it doesn't look like there's any trouble on that end.

About snapcast, it sounds very interesting, though I need synchronization at sample level, so I'm thinking of dedicating one input for an external clock source. Anyway I'll give it a try in the future.

This is one of the codes I've been trying with no results:

#include "AudioTools.h"
#include "AudioCodecs/CodecMP3Helix.h"
#include "driver/dac.h"

// AudioConfigLocal.h
#define DEFAULT_BUFFER_SIZE 512 // Aumentar si es necesario
#define I2S_BUFFER_SIZE 1024       // Aumentar si es necesario
#define I2S_BUFFER_COUNT 20        // Aumentar si es necesario


URLStream url("sssssssssss","pppppppppp");
I2SStream i2s; // final output of decoded stream
EncodedAudioStream dec(&i2s, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier(dec, url, 4096); // copy url to decoder


void setup(){
  dac_output_enable(DAC_CHANNEL_1);
  dac_output_voltage(DAC_CHANNEL_1, 150);  // amplifier volume control

  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Error);  

  // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  config.pin_bck=27;
  config.pin_ws=33;
  config.pin_data=26;
  //config.mode = I2S_STD_FORMAT;

  i2s.begin(config);

  // setup I2S based on sampling rate provided by decoder
  dec.setNotifyAudioChange(i2s);
  dec.begin();

// mp3 radio
  url.begin("https://plantaalta.chela.org.ar/domosonica/pads_22050_16bits.mp3","audio/mp3");
}

void loop(){
  copier.copy();
}

Thank you for you help!

[pschatzmann]

I am not sure if it is a good idea to use the driver/dac.h functionality directly and the defines do not have any impact like this.
I just tried my streaming mp3 example with the latest version and it is still working great with my url.

I have some issues with your url as well....

Did you create this mp3 ?

ps. I added an example for ftp today...

[pschatzmann]

I tend to think that the issue is on the server side: You don't hear this on your desktop because all audio players have a big buffer: this is something that we can't afford on microcontrollers...

Did you try to play the same from a file on your microcontroller ?

[pschatzmann]

I downloaded your mp3 with curl, copied it to a SD drive and played it with the help of the sdmmc API and it was playing great!
This confirms that the issue is on your server: it can't provide the data fast enough and adds long delays!

When you set the log level to Info, you can see when it is choking: Like you have highlighted already above.
There are work arounds, but it is better to fix the root cause of the issue!

ps if you plan to use mp3 with very high bitrates, do not forget to read the LibHelix wiki. But I would recommend that you avoid this.

[bergamifer]

Dear Phil,

I hope this message finds you well. I wanted to take a moment to express my deepest gratitude for the invaluable assistance you've provided to the coding community. Your guidance has been instrumental for enthusiasts like myself, who sometimes struggle due to lack of experience. Your willingness to share knowledge and provide support is greatly appreciated.

Your recommendation to set up a local FTP server on a Raspberry Pi was a game-changer for my project. It worked seamlessly, and for that, I am incredibly thankful.

I'm excited about the progress I've made and was wondering if you'd be interested in having my project featured in the "show and tell" section. It would be an honor to showcase the work that was successful, in part, due to your advice.

Additionally, I'm exploring the potential for a "player" feature that allows for changing MP3 files mid-routine. I'm aware of the "player" routine you posted and am curious if it could serve this purpose. Could you provide some insight into this?

I'm also seeking a straightforward method to externally interrupt the "write sample to i2s" routine, aiming for a synchronization level that Snapcast doesn't seem to provide. Any suggestions you might have on this would be highly beneficial.

Lastly, I've attached the code snippet that worked for me after tweaking some parameters as per the LibHelix wiki guidelines. Any feedback or improvements you could suggest would be welcomed.

Warm regards,
Fernando

This is the code that worked:

#include "WiFi.h"
#include "ArduinoFTPClient.h" // install https://github.com/pschatzmann/TinyFTPClient
#include "AudioTools.h" // https://github.com/pschatzmann/arduino-audio-tools
#include "AudioCodecs/CodecMP3Helix.h" // https://github.com/pschatzmann/arduino-libhelix
//#include "AudioLibs/AudioKit.h" // https://github.com/pschatzmann/arduino-audiokit
#include "driver/dac.h"


//Tuve que agregar estas lineas de codigo porque el buffer no era suficiente
#define MP3_MAX_OUTPUT_SIZE 2048 * 5
#define MP3_MAX_FRAME_SIZE 3200
   

WiFiClient cmd;
WiFiClient data;
FTPClient client(cmd, data);
FTPFile file;

I2SStream i2s;
//AudioKitStream kit; // or replace with e.g. I2SStream


MP3DecoderHelix mp3;
EncodedAudioStream decoder(&i2s, new MP3DecoderHelix());  // Decoding stream
//EncodedAudioStream decoder(&kit, &mp3); 
StreamCopy copier(decoder, file);

void setup() {
  Serial.begin(115200);

  dac_output_enable(DAC_CHANNEL_1);
  dac_output_voltage(DAC_CHANNEL_1, 160);  // amplifier volume control

  AudioLogger::instance().begin(Serial, AudioLogger::Error);

  // connect to WIFI
  WiFi.begin("xxxxxxxx", "xxxxxxxx");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  // optional logging
  FTPLogger::setOutput(Serial);
  // FTPLogger::setLogLevel(LOG_DEBUG);

  // open connection
  client.begin(IPAddress(192, 168, 30, 91), "xxxxxxx", "xxxxxxxx");

  // copy data from file
  file = client.open("/transfer/DS17.mp3");

  // open output device
  //kit.begin();

 // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  config.pin_bck = 27;
  config.pin_ws = 33;
  config.pin_data = 26;
  //config.mode = I2S_STD_FORMAT;

  i2s.begin(config);

  decoder.begin();

}

void loop() 
{ 
  copier.copy(); 
}

[pschatzmann]

You can also set the codec parameters in the sketch with
mp3.setMaxPCMSize(2048 * 5);
mp3.setMaxFrameSize(3200);

I think your dac_output_enable and dac_output_voltage doen't really do what you expect and are not necessary. If you want to control the volume here is the documentation.

What kind of synchronization logic are you planning to implement exactly ?

[bergamifer]

Dear Phil,

Thank you for the codec parameter tips; I've incorporated them into my sketch with the setMaxPCMSize and setMaxFrameSize methods as you suggested.

Regarding synchronization, I'm currently looking to achieve simultaneous audio playback across multiple boards using a WiFi-triggered command. I haven't settled on a specific implementation yet, but I'm under some time pressure to figure out a viable solution. Do you have any recommendations for a simple way to send such a trigger so that all the boards start playback at the exact same moment?

Also, I wanted to clarify that the dac_output_enable and dac_output_voltage functions are indeed serving their purpose in my setup. I'm using them to control the volume through the DAC on the ESP32, which is functioning flawlessly. It's quite useful for managing the volume at a more "analog" stage before it reaches the amplifier.

[pschatzmann]

For how long will the audio play?

[bergamifer]

It's meant to play songs so let's say 4 minutes or so for each song

[bergamifer]

I just need to hit a play button somewhere and somehow easy to trigger the start of the song

[pschatzmann]

The easiest is to send a udp brodcast to all machines. However I don't know how big the time differences will be.

The Snapcast protocol is sending the server time in regular intervals so that the clients can adjust. With each audio segment, the information when this is supposed to be played is sent, to make sure that all clients work in time.