How to resample URLStreams to 48KHz before sending via I2S to Speaker?

[chess-levin]

Hi folks!

I'm trying to find out how to resample every URLStream that is not 48KHz.

I bought a TSA5001 to transmit I2S data directly via Bluetooth. The TSA5001 supports only a fixed sampling rate of 48KHz. I'd like to add the TSA5001 to my own webradio project.

TSA 5001 Specs:

• Bluetooth v5.3 specification support
• I2S digital input
• Sampling Rate: 48KHz
• Bit per Sample: 16 bit, 24bit, 32bit.
• aptX, aptX Low Latency, aptX HD, SBC and AAC
• Bluetooth protocol: A2DP

Below is my code that does no resampling. The playback of a source sampled with 44,1KHz has the same 'interrupts' as if I'm trying without ResampleStream. There is a comment saying " // setup I2S based on sampling rate provided by decoder". But I don't understand how give this information from the decoder to the ResamplingStream instead of the i2s out.

I haven't yet looked at Phil's player example. Can I find some information for my question there or anywhere in this huge audio-tools project?

Thanks in advance for your support!

#include "AudioTools.h"
#include "AudioCodecs/CodecMP3Helix.h"

#define SAMPLE_RATE     48000
#define CHANNEL_COUNT   2
#define BITS_PER_SAMPLE 16

URLStream url("","");

AudioInfo info(44100,2,16);
I2SStream i2s; // final output of decoded stream
MetaDataOutput out1; // final output of metadata
EncodedAudioStream dec(&i2s, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier(dec, url); // copy url to decoder
MultiOutput out;
ResampleStream resample(dec);

// callback for meta data
void printMetaData(MetaDataType type, const char* str, int len){
  Serial.print("==> ");
  Serial.print(toStr(type));
  Serial.print(": ");
  Serial.println(str);
}

void setup(){
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Error);  

  out.add(out1);
  out.add(resample);

  // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  
  config.pin_data = 27;
  config.pin_bck = 26;
  config.pin_ws = 25;

  auto rcfg = resample.defaultConfig();
  rcfg.sample_rate = SAMPLE_RATE;
  rcfg.bits_per_sample = BITS_PER_SAMPLE;
  rcfg.channels = CHANNEL_COUNT;
  rcfg.copyFrom(info);

  resample.begin(rcfg);

  // mp3 radio
  //url.begin("http://stream.srg-ssr.ch/m/rsj/mp3_128", "audio/mp3");  // 48KHhz
  //url.begin("http://icecast.radiobremen.de/rb/bremenzwei/live/mp3/128/stream.mp3", "audio/mp3"); // 48KHhz
  url.begin("http://stream.berliner-rundfunk.de/brf/mp3-128/internetradio" , "audio/mp3"); // 44,1KHz

  // setup metadata
  out1.setCallback(printMetaData);
  out1.begin(url.httpRequest());

  i2s.begin(config);

  // setup I2S based on sampling rate provided by decoder
  dec.begin();

}

void loop(){
  copier.copy();
}

[pschatzmann]

Your code does not make any sense: You need to create an output chain!
UrlStream - copy -> EncodedAudioStream ->ResampleStream -> I2SStream
This will give you the arduments that you pass in the constructor.

ps. the wiki describs the logic that has been implemented for the propagation of format changes, so my expectation is that you would not need to do anything here

[pschatzmann]

I have no idea how you think your above sketch would work.

First you need to figure out what metdata your source is providing. If it is using the Icecast protocol you can use the ICYStream to receive the data and also get the metadata from it.

If it is ID3 which is contained in MP3 data you will need to send the recived MP3 data not only to the decoder but also to the MetaDataOutput !

[chess-levin]

Sorry, I forgot to remove some lines. I changed my code in my last post. The above code is running and produces correct playback for 48KHz, 128Kbp streams (e.g Swiss Jazz Radio and icecast.radiobremen.de). The two other streams

 url.begin("http://stream.berliner-rundfunk.de/brf/mp3-128/internetradio" , "audio/mp3");          // 44,1KHz, 128Kbps
 url.begin("http://rnrw.cast.addradio.de/rnrw-0182/dein90er/low/stream.mp3",  "audio/mp3");      // 24KHz, 64Kbps

producing a permanently interrupted, scratchy playback. And the one with 24KHz, 64Kbps is also played at a wrong pitch.

My first goal is to find out how to unify URLStreams with different sampling rates to send always 48KHz to TSA5001 that is connected to my I2S output. Therefore I removed everything from the code (like MultiOutput and MetaDataOutput).
As I understand your first response and the article you mentioned, I don't have to use ResampleStream, because the decoder class should hand this information downstream to the next in the chain.

When I solved this issue, I'll tackle the next problem regarding MP3 MetaData.

[pschatzmann]

You need to use ResampleStream! but the dec.addNotifyAudioChange(outI2S); is not necessary because it is automatically added in that case. Just start with the output chain that I have described and when this is working you can add additional complexity

[chess-levin]

I made another attempt using ResampleStream again (see code below). That works in general - the playback is perfect! Thank you for your food for thought!

BUT:

1. Sometimes it happens, that the Wifi is so bad that some TCP frames drop out. After this has happened the playback falls back to old interrupted behavior and it'll never switch back to the correct playback. I have to restart the URLStream to get a clean playback. It's like the filter chain gets out of sync and then there is no resync anymore.
2. The stream with this props 24KHz, 64Kbps is not correctly resampled. The playback's pitch is to high.
3. Same behavior for the stream with this pros 48KHhz, 64Kbps

Any idea on this?

#include "my.h"

#include "AudioTools.h"
#include "AudioCodecs/CodecMP3Helix.h"

#define SAMPLE_RATE     48000
#define CHANNEL_COUNT   2
#define BITS_PER_SAMPLE 16

URLStream source(AP_SSID, AP_PWD);

AudioInfo finalAudoPropsInfo(SAMPLE_RATE, CHANNEL_COUNT, BITS_PER_SAMPLE);

I2SStream outI2S;                                           // final output of decoded stream
ResampleStream resample(outI2S);                            // resample to guarantee finalAudoPropsInfo at I2S output
EncodedAudioStream dec(&resample, new MP3DecoderHelix());   // Decoding stream
StreamCopy copier(dec, source);                             // copy url to decoder

void setup(){
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Warning);  

  auto i2sConfig = outI2S.defaultConfig(TX_MODE);
  i2sConfig.copyFrom(finalAudoPropsInfo);
  i2sConfig.pin_data = 27;
  i2sConfig.pin_bck = 26;
  i2sConfig.pin_ws = 25;
  outI2S.begin(i2sConfig);
  
  auto rsConfig = resample.defaultConfig();
  rsConfig.copyFrom(finalAudoPropsInfo);
  resample.begin(rsConfig);

  // mp3 radio-stations using different stream parameters
  //source.begin("http://stream.srg-ssr.ch/m/rsj/mp3_128", "audio/mp3");                               // 48KHhz, 128Kbps
  //source.begin("http://icecast.radiobremen.de/rb/bremenzwei/live/mp3/128/stream.mp3", "audio/mp3");  // 48KHhz, 128Kbps
  source.begin("http://icecast.radiobremen.de/rb/bremenzwei/live/mp3/64/stream.mp3", "audio/mp3");  // 48KHhz, 64Kbps
  //source.begin("http://stream.berliner-rundfunk.de/brf/mp3-128/internetradio" , "audio/mp3");          // 44,1KHz, 128Kbps
  //source.begin("http://rnrw.cast.addradio.de/rnrw-0182/dein90er/low/stream.mp3",  "audio/mp3");      // 24KHz, 64Kbps

  // make sure that resample is updated
  dec.addNotifyAudioChange(resample);
  
  // setup I2S based on sampling rate provided by resample
  dec.begin();
}

void loop(){
  copier.copy();
}

[pschatzmann]

Not sure if this helps, but I would define the rscfg.to_sample_rate to keep it fixed. Otherwise you risk that you end up with the wrong final output rate whenever the decoder reports another rate..

[chess-levin]

Hm, the playback of a 44,1KHz stream is still not perfect. The pitch is to high.

I see many helpful information (e.g. setStepSize from ResampleStream et al) AudioLogger::instance().begin(Serial, AudioLogger::Info); when setting Logger to Info, but the log-output of the copier spams the log window. I can't find a way how to exclude the log output of the copier to keep the log small. Is there one?

The stepSize value I would expect is above zero: s = 1.08843537415 = 48000/44100, but the log shows the invers value 1/s = 0.918750?

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:13232
load:0x40080400,len:3028
entry 0x400805e4
[    13][D][esp32-hal-cpu.c:244] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[E] MetaDataICY.h : 269 - icy-metaint not defined
[I] MetaDataICY.h : 278 - void audio_tools::ICYUrlSetup::executeCallback(void (*)(audio_tools::MetaDataType, const char*, int))
[I] MetaDataID3.h : 569 - virtual void audio_tools::MetaDataID3::begin()
[I] AudioTypes.h : 104 - sample_rate: 48000
[I] AudioTypes.h : 105 - channels: 2
[I] AudioTypes.h : 106 - bits_per_sample: 16
[I] I2SConfigESP32.h : 90 - rx/tx mode: TX_MODE
[I] I2SConfigESP32.h : 91 - port_no: 0
[I] I2SConfigESP32.h : 92 - is_master: Master
[I] I2SConfigESP32.h : 93 - sample rate: 48000
[I] I2SConfigESP32.h : 94 - bits per sample: 16
[I] I2SConfigESP32.h : 95 - number of channels: 2
[I] I2SConfigESP32.h : 96 - signal_type: Digital
[I] I2SConfigESP32.h : 98 - i2s_format: I2S_STD_FORMAT
[I] I2SConfigESP32.h : 100 - auto_clear: true
[I] I2SConfigESP32.h : 102 - use_apll: true
[I] I2SConfigESP32.h : 107 - buffer_count:6
[I] I2SConfigESP32.h : 108 - buffer_size:512
[I] I2SConfigESP32.h : 113 - pin_bck: 26
[I] I2SConfigESP32.h : 115 - pin_ws: 25
[I] I2SConfigESP32.h : 117 - pin_data: 27
[I] ResampleStream.h : 222 - begin step_size: 1.000000
[I] AudioTypes.h : 104 - sample_rate: 48000
[I] AudioTypes.h : 105 - channels: 2
[I] AudioTypes.h : 106 - bits_per_sample: 16
[I] ResampleStream.h : 267 - setStepSize: 1.000000
[I] ResampleStream.h : 122 - input_size: 8.000000 / output_size: 8.000000 / factor (eff): 1.000000
[I] URLStream.h : 79 - virtual bool audio_tools::URLStream::begin(const char*, const char*, MethodID, const char*, const char*): http://stream.berliner-rundfunk.de/brf/mp3-128/internetradio
[I] Url.h : 58 - Url::parse
[I] Url.h : 96 - url->http://stream.berliner-rundfunk.de/brf/mp3-128/internetradio
[I] Url.h : 97 - host->stream.berliner-rundfunk.de
[I] Url.h : 98 - protocol->http
[I] Url.h : 99 - path->/brf/mp3-128/internetradio
[I] Url.h : 100 - port->80
[I] URLStream.h : 393 - void audio_tools::URLStream::login()
[   218][D][WiFiGeneric.cpp:1035] _eventCallback(): Arduino Event: 0 - WIFI_READY
[   308][V][WiFiGeneric.cpp:340] _arduino_event_cb(): STA Started
[   314][D][WiFiGeneric.cpp:1035] _eventCallback(): Arduino Event: 2 - STA_START
[   327][V][WiFiGeneric.cpp:97] set_esp_interface_ip(): Configuring Station static IP: 0.0.0.0, MASK: 0.0.0.0, GW: 0.0.0.0
..[   956][V][WiFiGeneric.cpp:355] _arduino_event_cb(): STA Connected: SSID: BLOCKLAND, BSSID: 3c:a6:2f:87:9e:53, Channel: 1, Auth: WPA2_PSK
[   959][D][WiFiGeneric.cpp:1035] _eventCallback(): Arduino Event: 4 - STA_CONNECTED
[   985][V][WiFiGeneric.cpp:369] _arduino_event_cb(): STA Got New IP:192.168.1.158
[   985][D][WiFiGeneric.cpp:1035] _eventCallback(): Arduino Event: 7 - STA_GOT_IP
[   990][D][WiFiGeneric.cpp:1098] _eventCallback(): STA IP: 192.168.1.158, MASK: 255.255.255.0, GW: 192.168.1.1

[I] URLStream.h : 369 - WiFiClient
[I] HttpRequest.h : 222 - process connecting to host stream.berliner-rundfunk.de port 80
[I] HttpRequest.h : 337 - connected 1 timeout 60000
[I] HttpRequest.h : 233 - Free heap: 180124
[I] HttpHeader.h : 271 - HttpHeader::write
[I] HttpHeader.h : 417 - -> GET /brf/mp3-128/internetradio HTTP/1.1
[I] HttpHeader.h : 205 -  -> Host: stream.berliner-rundfunk.de
[I] HttpHeader.h : 205 -  -> Connection: keep-alive
[I] HttpHeader.h : 205 -  -> Accept-Encoding: identity
[I] HttpHeader.h : 205 -  -> Accept: audio/mp3 
[I] HttpHeader.h : 337 -  -> <CR LF>
[I] HttpRequest.h : 288 - Request written ... waiting for reply
[I] HttpHeader.h : 248 - Data available: 708
[I] HttpHeader.h : 459 - HttpReplyHeader::readExt
[W] URLStream.h : 336 - Redirected to: http://topradio-de-hz-fal-stream03-cluster01.radiohost.de/brf_mp3-128?ref=internetradio&amsparams=internetradio
[I] Url.h : 58 - Url::parse
[I] Url.h : 96 - url->http://topradio-de-hz-fal-stream03-cluster01.radiohost.de/brf_mp3-128?ref=internetradio&amsparams=internetradio
[I] Url.h : 97 - host->topradio-de-hz-fal-stream03-cluster01.radiohost.de
[I] Url.h : 98 - protocol->http
[I] Url.h : 99 - path->/brf_mp3-128?ref=internetradio&amsparams=internetradio
[I] Url.h : 100 - port->80
[I] HttpRequest.h : 222 - process connecting to host topradio-de-hz-fal-stream03-cluster01.radiohost.de port 80
[I] HttpRequest.h : 337 - connected 1 timeout 60000
[I] HttpRequest.h : 233 - Free heap: 178476
[I] HttpHeader.h : 271 - HttpHeader::write
[I] HttpHeader.h : 417 - -> GET /brf_mp3-128?ref=internetradio&amsparams=internetradio HTTP/1.1
[I] HttpHeader.h : 205 -  -> Host: topradio-de-hz-fal-stream03-cluster01.radiohost.de
[I] HttpHeader.h : 205 -  -> Connection: keep-alive 
[I] HttpHeader.h : 205 -  -> Accept-Encoding: identity
[I] HttpHeader.h : 205 -  -> Accept: audio/mp3 
[I] HttpHeader.h : 337 -  -> <CR LF>
[I] HttpRequest.h : 288 - Request written ... waiting for reply
[I] HttpHeader.h : 239 - Waiting for data...
[I] HttpHeader.h : 248 - Data available: 845
[I] HttpRequest.h : 164 - no CONTENT_LENGTH found in reply
[I] URLStream.h : 87 - size: 0
[I] URLStream.h : 231 - Request written ... waiting for reply
[I] URLStream.h : 94 - ==> http status: 200
[I] AudioEncoded.h : 332 - virtual void audio_tools::EncodedAudioOutput::addNotifyAudioChange(audio_tools::AudioInfoSupport&)
[I] StreamCopy.h : 143 - StreamCopy::copy  1024 -> 1024 -> 1024 bytes - in 1 hops
[I] AudioTypes.h : 104 - sample_rate: 44100
[I] AudioTypes.h : 105 - channels: 2
[I] AudioTypes.h : 106 - bits_per_sample: 16
[I] ResampleStream.h : 267 - setStepSize: 0.918750