Arbitrary Waves Resampling and Mixing

[HobmanX]

I am using ESP32-Pico-D4 dev board and programming via Arduino 2.3.2 on windows 11. It is about a year since I have been on this forum discussion page and at that time I tried out some examples but never got them to run or combine sucessfully. So I am hoping to try again as I have been getting emails and notification to say you are still developing and have improved the system.
Basically I want to be able to play sounds generated from 2 arbitrary wave arrays, Re-sample so as to give various frequencies changing as demanded in real time by pushbuttons etc. Mix these sounds together and output to the on board/chip DAC pins 25 and 26 as a stereo or 2 mono signals. My problem in the past is finding examples that show how to combine the different functions and the order to call them in the sketch. So I was hoping now there are maybe some examples or someone could outline the order of the functions in each step.

[pschatzmann]

I am providing example for each of your requirements, but it is up to you to put things together.
Start with a simple sketch and extend it step by step: this allows you to figure out the issue when you make a mistake.

[HobmanX]

Thanks for reply. My problem is there are lots of individual examples but the problem is how to combine them.
Example..... sine from array that I want to change frequency on demand....... Do I resample the wave array or do I generate a sound from the array first then resample and then if wanting to mix in another sound what is copied to mixer , is it the sound or the resample sound.
Is there any documentation where I can read up on this. The Wiki is just a dictionary of words and names, doen't explain how it all goes together

[pschatzmann]

Neither one, nor the other: Read the introduction carefully again: all the answers can be found there! It even explains how to set the frequency in a Sine Generator! This Wiki also explains how you can build a chain to put things together.

Try to understand the examples: They are always only a few lines of code, so this should not be too difficult and have a look at the class documentation and the available methods: In the Wiki I am providing the relevant links to make this step as easy as possible.

Resampling you would use e.g. when you have sound file at a lower or higher frequency then what the output (device) is requiring. Or you could use it to change the playback speed of a recording. For your sine it would theoretically work as well, but you can change the frequency directly and there is no need to make things complicated.

I don't quite understand your question about mixing: you just mix the input that you want to hear and repeating the documentation: the audio format for all your input needs to match and if it does not, you e.g resample or reformat to make it match.

[HobmanX]

This is the code "Changing the Signal" example in your introduction pages.....
There are some syntax errors in it, which are not hard to sort out one they are found but can be confusing when just trying to get it going first time......
AudioInfo info(44100,2,16); ....... changed to....... AudioInfo info(44100,1,16);
volume.begin(config_vol); ...... changed to......vol.begin(config_vol);
Had problems gettiing it to compile then included the files AudioConfig.h and AudioLogger.h and it compiled OK
Now I dont seem to need them... it now compiles with then commented out......Do I need them or Not.
With Amplitude set at 32000 I get a "lob sided sine wave". If I reduce amplitude to 16000 it looks OK.
Also with the Volume bit included I get what looks like a low resolution staircase sinewave when the input amplitude is down say at 8000 and vol at 0.5 say.

"It even explains how to set the frequency in a Sine Generator!"........... that is not my problem I will not be using sinewaves generator I will be using Arbitrary waves and want to change the frequency and amplitude and do mixing etc.

`
//#include <AudioConfig.h>
//#include <AudioLogger.h>

#include "AudioTools.h"

AudioInfo info(44100,1,16);
SineWaveGenerator<int16_t> sineWave(32000); // subclass of SoundGenerator with max amplitude of 32000
GeneratedSoundStream<int16_t> in(sineWave); // Stream generated from sine wave
//CsvStream<int16_t> csv(Serial);
AnalogAudioStream out;
//VolumeStream vol(out);
StreamCopy copier(out, in);
//StreamCopy copier(vol, in);

// Arduino Setup
void setup(void) {
// Open Serial
// Serial.begin(115200);
// AudioLogger::instance().begin(Serial, AudioLogger::Info);

// start I2S

// Serial.println("starting I2S...");
auto config = out.defaultConfig(TX_MODE);
config.copyFrom(info);
out.begin(config);

// setup volume
// auto config_vol = vol.defaultConfig();
// config_vol.copyFrom(info);
// config_vol.volume = 0.5; // half the volume
// vol.begin(config_vol);

// Setup sine wave
sineWave.begin(info, N_B4);
}

void loop() {
copier.copy();
}
`

[pschatzmann]

Thanks for letting me know. I have corrected the errors and it should compile now w/o errors.
I suggest that you leave the amplitude at 32000 (or remove it complexly to get the full int16_t range) and manage the volume via VolumeStream.

Your staircase is a display problem: there are no staircases when you render/hear the audio

If you want to change the playback speed you can, as I have explained abouve, resample, if you just want to change the frequency/pitch but not the speed you can try to use pitch shifting.

[pschatzmann]

Basic functionality does not need to be included and you can't get more basic than the logger and configuration.
In the example I am using I2S: This is a stereo protocol, therefore I was defining 2 channels.

You can use the internal DAC, but I would recommend to use a proper Audio I2S DAC since they are dirt cheap and provide a perfect audio quality.

[HobmanX]

The main reason for using internal DAC is I wanted the project to be all one board. Also I find the internal DAC gives me adequate quality, I have tried other cheap DAC and not really much better. I want my project as small as possible so I have the ESP32-Pico-Kit-D4 but I am unable to find the proper pins for External I2S to an external DAC. I have read through the datasheets and pin listings and cannot find any pins listed as I2S. Does your libraries support the ESP32-Pico-kit-D4. I cannot find any reference to it or similar.

[pschatzmann]

I dont understand your statement: "I cannot find any pins listed as I2S". Did you read the link that I sent you to the I2S Dacs. There you can see the pins that have been predefined, but you can use basically any other pin as you like...

ps. the individual board does not matter, it is the processor type that is relevant...

[HobmanX]

First of all, it looks like I cannot get the posting of code to work on this forum... what am I doing wrong....?
The statement in my last post means that I cannot find any pins that work for external I2S on my ESP32-Pico-Kit-D4 I have also found out now that on the bottom of board it says V4.1. I have looked at all the datasheets I can find online and all the pin-out diagrams I can find and none of the pins is labelled as I2S option or BCLK, or WS, or DIN or DOUT. They are listed for all the other functions eg GPIOxx. TOUCHx, ADC_CHx, SD_DATAx etc, etc.
The chip on the board is ESP32-PICO-D4 and any diagrams I read online for it or other regular ESP32 gives the same pin descriptions
so I used what you classify as Regular ESP32 pins..... 14, 15, 22. and I config these pins in the sketch.
When I look at the PICO Dev Board I see that chip pin 14 goes to board pin IO25, Chip pin 15 goes to board pin 26, and chip pin 22 goes to board pin IO2.
I am using a PT8211 DAC and I connect IO25 to pin1 BCK, IO26 to pin2 WS, and IO2 to Pin DIN.
I do not get any output on Pin IO2 or analog out from the PT8211
If I stay connected to the same pins on the board but config sketch using 25, 26, 2 I get loud high freq noise signal from Dout and the PT8211 output.
So thats my problem...... what pins do I use or what datasheets do I need to read....?

/**
 * @file streams-generator-i2s.ino
 * @author Phil Schatzmann
 * @brief see https://github.com/pschatzmann/arduino-audio-tools/blob/main/examples/examples-stream/streams-generator-i2s/README.md 
 * @copyright GPLv3
 */
 
#include "AudioTools.h"

AudioInfo info(44100, 2, 16);
SineWaveGenerator<int16_t> sineWave(32000);                // subclass of SoundGenerator with max amplitude of 32000
GeneratedSoundStream<int16_t> sound(sineWave);             // Stream generated from sine wave
I2SStream out; 
StreamCopy copier(out, sound);                             // copies sound into i2s

// Arduino Setup
void setup(void) {  
  // Open Serial 
 // Serial.begin(115200);
 // while(!Serial);
  //AudioLogger::instance().begin(Serial, AudioLogger::Info);

  // start I2S
 // Serial.println("starting I2S...");

   auto config = out.defaultConfig(TX_MODE);
  // config.copyFrom(info);
  config.pin_bck = 14;
  config.pin_ws = 15;
  config.pin_data = 22;
  // I2SStream.begin(config);
   out.begin(config);

  // Setup sine wave
  sineWave.begin(info, N_B4);
  Serial.println("started...");
}

// Arduino loop - copy sound to out 
void loop() {
  copier.copy();
}

[joba-1]

use 3 backticks for multiline code, not just one (for syntax highlighting add the language behind the ticks):

```c++
int c;
```

int c;

[pschatzmann]

The pin numbers indicated are always GPIO numbers so 14 is GPIO 14.

There is the regular ESP32 and there are C3, S2, S3, H2 and C6.
As far as I know your PicoD4 is based on a regualar ESP32.

But for I2S this does not really matter and it works all the same for all of them. The differences are in the default pins and the internal DAC and ADC.

So when you specify the pins, as you did above, you are on the save side...

[HobmanX]

Thanks for quick reply
When I click code in the toolbar I just get 2 ticks and I was pasting the code in between, so now add 3 then paste add3 at end. the <> on the toolbar is not needed..??
Regards the sketch program..... what am I doing wrong....... do I config 14,15,22 in the sketch....It give me no output and 25,26,2 gives me noise bad output...? My internal DAC output analog audio on pins 25 and 26...OK.

[joba-1]

The intended usage of <> is: first write or paste the code, then mark the code, then press <>.
iirc it will automatically insert one or three ticks based on how many lines you marked. Nothing marked counts as single line -> you get only 2 ticks. So much for intuitive :D Please don't shoot the messenger ;)

[pschatzmann]

I corrected the code myself to make it readable...

[pschatzmann]

I do not know the PT8211 DAC. You might need to change the format in the config.

[HobmanX]

I may need to change format OK, but should I not see activity on the ESPchip pins 14,15 and 22 or on the boards pins 25,26,and 2

Should it be the ESPchip pins that are used in config or the Board IO numbers...?

[pschatzmann]

you were writing: I am using a PT8211 DAC and I connect IO25 to pin1 BCK, IO26 to pin2 WS, and IO2 to Pin DIN.
so your config should be

  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info);
  config.pin_bck = 25;
  config.pin_ws = 26;
  config.pin_data = 2;
  out.begin(config);

and you confirmed that you get an output this way...

[HobmanX]

Regards the sketch, Yes it works now, PT8211 is LSBJ so it needed the config for .... I2S_LSB_FORMAT
And using the numbers for the ESPpico board pins in the config and not the ESPchip pins.

Phil said......"The pin numbers indicated are always GPIO numbers so 14 is GPIO 14." ...... I don't have a GPIO 14...... ESP32chip pin14 is GPIO 25 and ESPchip pin15 is GPIO 25 which are the analog out pins for the internal DAC now we are using them as BCK and WS. So it looks like config is not setup to the chip pins but to the dev board pins..... so can I change these to whatever is a spare on the dev board...??.

Tried the code <> it worked, It needed me to select the text first then it wrapped in the 3 ticks..???

auto config = out.defaultConfig(TX_MODE);
 config.copyFrom(info); 
 config.i2s_format = I2S_LSB_FORMAT;
 config.pin_bck = 25;
 config.pin_ws = 26;
 config.pin_data = 2;
 out.begin(config);

PS........Have now tried config on pins ...... 18 = BCK, 23 = WS, 19 = DOUT and it works......... and these are the numbere on the Board pins, they actually equal the GPIO listing but not the ESPchip pin numbers....

[HobmanX]

I was trying simply playing arrays of sinewaves and I notice that the frequency comes out at twice what I thought it would be.
I generally in other applications use array of 256 points for 1 cycle of a wave which is 172 Hz at a rate of 44100. That would give me a frequency of 172 Hz out when played at 44100 or recorded in Audacity at 44100, but with the ESP32 I am getting 344 Hz. I also tried one of the examples that had a 45 point sine array and same with it ,it comes out twice the freq. So if I set the rate in the sketch to 22050 it plays at the correct frequency. Why does the ESP32 need to be set at 22050 rate or are there other setting I should be using...??

[pschatzmann]

I bet you messed up with the number of channels: In the sketch you indicated that you provide a stereo signal!
So either change the number of channels from 2 to 1 or provide a stereo recording.

[pschatzmann]

I thought you wanted to play different tones on different channels, so I don't see how changing the sample rate would be a solution. But what makes you think that it is not possible to change the sample rate for i2s ? Did you look at the class documentation ?

I rather think the solution should be that you record your samples in a much higher sampling rate and then you down-sample instead of creating samples in a low rate and up-sample....

[HobmanX]

But what makes you think that it is not possible to change the sample rate for i2s ?
I can change the sample rate by changing the variable at the top of the sketch, but that means the it will stay at that rate while this instance of the sketch is running. I want to be able to change the frequency of the output wave on demand as the sketch is running......either by changing the sample rate or resampling stepsize, or pitch shifting etc... I am unable to find a command that will allow the samplerate to be changed as the program loop is running. I can do it with resample stepsize command in loop, but as I said before I am getting clicks in the audio at some settings.

record your samples in a much higher sampling rate.......
How do you mean me to do that, is it just more samples or less samples in the array. As I said earlier in other MCUs I normally use 256 point array but it was low freq out so I tried 128 point array that brought up the frequency. but I want to vary the frequency so still need some of the methods, resample or pitchshift etc.
Doing this with other micros I dont have any problems, I don't get any clicks etc and I can change the freq quickly and as often as I want.
I have heard alot about the ESP32 and that is was fast so I wanted to try it and I have several of them, but I seem just not able to get it to do what I want. It will do simple things like play an array at a frequency or resample at a frequency but over a range of frequencies it seems to have problems...?

[pschatzmann]

You would provide much more samples and to select the frequency you would select a step size below 1.0

Have another look at the documentation and search for sample rate. AudioStream is a subclass of AudioInfoSupport!
Alternatively you could also just call begin again the loop.

[HobmanX]

My problem is understanding this. When I click on the link documentation I get a page of text...... what is it telling me. Is ther no documentation anywhere that explains these pages and how to read them and relate to code. I have programmed for years using Basic, and C, and Unix and LInux and using Arduino and other IDE. For each of them there are alot of textbooks that explains the functions and how to put together the code...... Is there nothing like that for this system...?

[HobmanX]

I have read through the documentation and the .h files, but I am still unable to get any option to allow change of samplerate to be called within loop. I am unable to follow the logic of the code so as to be able to try logical options. Below are some that I tried but it always plays at the samplerate that is defined within setup.
If I use the word "config" in loop it tells me that it is not in scope...??? but I got it to compile with using the word "info" but it does not do any change when code is running. Also the word "samplerate" seems to be spelt various ways in the header files so it keeps telling that it is not valid. Can you please give an example of code that would work in this situation....

#include "AudioTools.h"
int sampR = 44100;
int countr = 0;

AudioInfo info(44100, 1, 16);
int16_t sine_array[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383, 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, -24350, -21062, -17363, -13327, -9031, -4560  };
GeneratorFromArray<int16_t> sineWave(sine_array,0,false);
GeneratedSoundStream<int16_t> sound(sineWave);             // Stream generated from sine wave
I2SStream out; 
//AnalogAudioStream out; 
//StreamCopy copier(out, sound);                             // copies sound into i2s
StreamCopy copier(out, sound);                        // copies sound to out

// Arduino Setup
void setup(void) {  
 
  // start I2S

  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  config.i2s_format = I2S_LSB_FORMAT;
  config.pin_bck = 18;
  config.pin_ws = 23;    // 18, 23, 19 for ESP32-PICO-D4
  config.pin_data = 19;
  config.sample_rate = sampR;
  out.begin(config);

  // Setup sine wave
  sineWave.begin(info);
}

// Arduino loop - copy sound to out 
void loop() {
  countr = countr + 1;
   copier.copy();
   if (countr >= 1000);
  {
  sampR = 11000;
  info.sample_rate = sampR;
  out.begin();
  countr = 0;
  }
}
  

[pschatzmann]

it should be

info.sample_rate = new_rate;
out.setAudioInfo(info);

I suggest to revisit the class documentation because this is the most relevant source of information that will be always up to date because it is generated based on the source code. For a class it shows all the methods that you can call and gives a complete picture how to use an object that is instantiated from this class. The method names are usually speaking, but in most of the cases there is also a short explanation. E.g.

It also shows all the parent classes which let's you investigate the common functionality shared with other classes. E.g. for the I2SStream you see that it is inheriting from AudioStream which e.g. inherits from AudioInfoSupport. If you click on it you see that it just has 2 methods: one for setting the audio info and one for asking for the actual value and the documentation shows for which other classes you can use these methods. AudioStream also inherits from the Arduino Stream, so you can use this class also for any functionality provided by Arduino or other libraries which expect a Stream.

[HobmanX]

This is what I put in loop hoping it would switch smoothly between 2 tones...... a higher and then lower pitch sinewave. The sinewave plays OK and changed frequency but there is a gap of about 80 to 100 ms between each transition.....???....... can the gap be eliminated..??
In the last post I had in setup ..... config.sample_rate = sampR;........I have taken this out....I think it should not be there...??
I will try and read up on the object programming but when I read say W3 schools online the objects are simple and easier followed, its the terminology in all this audio stuff I would like a textbook on.

void loop() {
  countr = countr + 1;
 
 // countr2 = countr2 + 1;
     copier.copy();

   
   if (countr == 150)
  {
  sampR = 44100;
    info.sample_rate = sampR;
  out.setAudioInfo(info);
  countr = 0;
  }
  else if (countr == 100)
  {
  sampR = 22050;
    info.sample_rate = sampR;
  out.setAudioInfo(info);
   }
  
}

[pschatzmann]

Not sure, you could try to call the IDF method directly to set the sample rate:
esp_err_ti2s_set_sample_rates(i2s_port_ti2s_num, uint32_t rate)

[HobmanX]

Thanks for reply, but I have no idea how to apply it.
I did try ........ esp_err_t i2s_set_sample_rates(i2s_port_ti2s_num, uint32_t rate)
as.......... esp_err_t i2s_set_sample_rates(1, 44100) ........ but it didn't like it, told me not defined in this scope and other errors.

Is the Audio using DMA and can run in background continuously while the loop polls other tasks..??

[pschatzmann]

See https://github.com/pschatzmann/arduino-audio-tools/wiki/Multicore-Processing

It would be i2s_set_sample_rates((i2s_port_t)0, 44100); if you did not specify any port in your i2s config. But you can also use my latest audio-tools version. This is the method that I am now calling when only the sample rate has changed.

[HobmanX]

Yes, thanks, that has made changes. There are now no gaps but still discontinuity between the transitions. So the clock must stop and restart....??? or maybe a buffer problem.....???

#include "AudioTools.h"
unsigned long ToneTime;
bool HiLo = true;

AudioInfo info(44100, 1, 16);
int16_t sine_array[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383, 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, -24350, -21062, -17363, -13327, -9031, -4560  };
GeneratorFromArray<int16_t> sineWave(sine_array,0,false);
GeneratedSoundStream<int16_t> sound(sineWave);             // Stream generated from sine wave
I2SStream out; 
//AnalogAudioStream out; 
StreamCopy copier(out, sound);                        // copies sound to out

// Arduino Setup
void setup(void) {  
 
  // start I2S

  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  config.i2s_format = I2S_LSB_FORMAT;
  config.pin_bck = 18;
  config.pin_ws = 23;    // 18, 23, 19 for ESP32-PICO-D4
  config.pin_data = 19;
  out.begin(config);

  // Setup sine wave
  sineWave.begin(info);
}

void loop() {
     copier.copy();
   
   if (millis() - ToneTime >= 500)
   {
  ToneTime = millis();
  HiLo = !HiLo;
  
  if (HiLo == true) 
  {
 // info.sample_rate = 44100;
  i2s_set_sample_rates((i2s_port_t)0, 44100);
  //out.setAudioInfo(info);
  }
  if (HiLo == false)
  { 
  //info.sample_rate = 22050;
  i2s_set_sample_rates((i2s_port_t)0, 22050);
  //out.setAudioInfo(info);
  }
   }
 }

[pschatzmann]

One of my main scenarios is to play different (e.g. mp3) files, each having a different sample rate: so there is normally no audio when the rate is changed. I don't think this is intended to dynamically change when playing audio.

[HobmanX]

OK, I was hoping it would be alternative to resampling and give a cleaner wave, just playing the array faster or slower...??
Can Resampling be improved...... is it the ESP32 that limits it..... is it not capable of doing the maths fast enough or not doing floating maths...??
I maybe will try Maximillion, I think I tried it about a year ago but it also had problems. Why is Maximillion on your Web pages, Do you support it as well or are you a co-author of it also..??

[pschatzmann]

Can you share your input file and maybe some examples with the issue that you have with resampling.
I am really convinced that the solution is really with using resampling and preparing the input file properly.

My audiotools library just provides an integration into Maximilian as I am doing this with other DSP libraries and I have ported the Maximilian library to be used as Arduino Library

[HobmanX]

This is a typical sound array I would be using. This is a triangular wave generated in Audacity so that I can see what is happening. The screen shots are the audio output from ESP32 as sketch was running....it was recorded in Audacity.
My test was played at 22050but is similar at other rates. In real application I may want to record real sound and play them back so not much pre-processing.
Sorry for the long listed array....... how do I get it to paste across the page..??

#include "AudioTools.h"

AudioInfo info(22050,2,16);

// 253 point triangle chirp
int16_t tr253[] = {
0,
2255,
4531,
6828,
9145,
11482,
13839,
16215,
18610,
21024,
23456,
22261,
19637,
16996,
14338,
11664,
8973,
6266,
3543,
805,
-1949,
-4716,
-7499,
-10295,
-13105,
-15928,
-18764,
-21613,
-21043,
-18013,
-14972,
-11920,
-8857,
-5783,
-2698,
397,
3501,
6615,
9737,
12869,
16009,
19157,
21023,
17704,
14378,
11045,
7706,
4361,
1010,
-2347,
-5708,
-9075,
-12445,
-15820,
-19198,
-18728,
-15187,
-11644,
-8098,
-4549,
-1000,
2552,
6104,
9658,
13211,
16765,
19275,
15566,
11858,
8151,
4446,
743,
-2959,
-6657,
-10353,
-14046,
-17735,
-16459,
-12623,
-8790,
-4963,
-1141,
2676,
6487,
10292,
14090,
17881,
14655,
10722,
6798,
2881,
-1027,
-4926,
-8816,
-12697,
-16567,
-14490,
-10484,
-6489,
-2505,
1468,
5428,
9377,
13313,
16434,
12368,
8316,
4277,
253,
-3757,
-7752,
-11732,
-15697,
-12622,
-8533,
-4461,
-406,
3633,
7653,
11656,
15535,
11413,
7309,
3225,
-840,
-4885,
-8910,
-12915,
-13030,
-8912,
-4815,
-739,
3314,
7345,
11354,
13499,
9382,
5288,
1218,
-2827,
-6848,
-10843,
-12934,
-8833,
-4759,
-711,
3310,
7304,
11270,
11447,
7380,
3342,
-668,
-4649,
-8600,
-12521,
-9152,
-5135,
-1149,
2806,
6730,
10622,
10147,
6163,
2213,
-1705,
-5590,
-9441,
-10437,
-6498,
-2595,
1274,
5107,
8905,
10092,
6211,
2367,
-1441,
-5211,
-8943,
-9186,
-5374,
-1601,
2134,
5829,
9484,
7789,
4058,
368,
-3282,
-6890,
-9653,
-5972,
-2334,
1261,
4813,
8322,
7386,
3808,
275,
-3213,
-6657,
-8339,
-4831,
-1369,
2048,
5417,
8740,
5444,
2060,
-1276,
-4563,
-7802,
-5687,
-2391,
856,
4053,
7200,
5604,
2403,
-747,
-3845,
-6891,
-5235,
-2138,
907,
3898,
6836,
4622,
1637,
-1294,
-4170,
-6727,
-3806,
-941,
1869,
4621,
5622,
2828,
91,
-2588,
-5208,
-4389,
-1730,
870,
3410,
5645,
3069,
554,
-1901,
-4295,
-4130,
-1704
};


GeneratorFromArray<int16_t> waveTR(tr253,0,false);
GeneratedSoundStream<int16_t> sndTR(waveTR);             // Stream generated from array1
ResampleStream resample(sndTR);
I2SStream out; 
StreamCopy copier(out, resample);                        // copies sound to out

// Arduino Setup
void setup(void) {  
  // Open Serial 
 // Serial.begin(115200);
 // AudioLogger::instance().begin(Serial, AudioLogger::Warning);

  // define resample
  auto rcfg = resample.defaultConfig();
  rcfg.copyFrom(info);
  rcfg.step_size = 0.5;
  resample.begin(rcfg); 

  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  config.i2s_format = I2S_LSB_FORMAT;
  config.pin_bck = 18;
  config.pin_ws = 23;     //  18, 23, 19 for ESP32-PICO-D4
  config.pin_data = 19;
  out.begin(config);

 // Setup sine wave
  auto cfg1 = sndTR.defaultConfig();
   sndTR.begin(cfg1);


}

// Arduino loop - copy sound to out 
void loop() {
  copier.copy();
}

[pschatzmann]

Here is the easiest solution approach:

• Use a sampling rate in Audacity which is very high: e.g. 96000 samples per second.
• Try to record a sample with the highest pitch that you want to play back
• Use down-sampling to play it back on the ESP32.

If you still want to support up-sampling and play it faster: here are the steps

• Use a sampling rate in Audacity which is very high: e.g. 96000 samples per second.
• Analyse - display Frequency Range. Calculate the maximum Frequency that is possible after upsampling
• Remove the Frequencies above the limit using Effect Filter Curve
• Use up and down-sampling to play it back on the ESP32.
  (theoretically you could add a low pass filter before the upsample, but I think the above method is more efficient and you can avoid this processing step on the ESP32)

With a sample rate of 44100 you can only represent a frequency range of 22050 hz. If you upsample a singnal that has some frequency components that end up > 22050 after up-sampling, this will result in aliasing noise!

Can you share the Frequency Range of your signal as well ?

[HobmanX]

Yes will do, I know I am at computer now but in middle of something else, so will do in about an hour. But as I said I Only generated 1 cycle of what I called the wave I didnt repeat it say 1000 times to make a long sample that is needed for plotting spectrum. But I will do it again and build it and see. I can easily do plot spectrum of the output played from the ESP32. thanks for your support

[pschatzmann]

No worries, I think the recorded spektrum is not important. It is more important to find if your original signal has frequencies that need to be removed.

ps. I think you can save yourself a lot of work if you save and play back wav files. This also enables you to test them in Audacity (with Loop Play)

[HobmanX]

This is the spectrums of theESP32 outputs recorded as wavs.

[HobmanX]

I think the previous waves and spectrums etc are too high a freq content. this now is a spectrum for a wave of 90 samples and 2000Hz triangular wave about 4cycles. I would need time to try these and get maybe better examples.

[HobmanX]

I took the 90 sample wave and resampled up to 256 samples in Audacity and created a 256 point array. I can resample in ESP32 at 0.5, 1.0, 1.5, and 2.5 and no sign of distortion and sounds smooth. And on playback from ESP32 the sprectrum looks similar to last post..
So you seem to have pointed to something I was doing wrong, Thanks, I will play about a bit more tomorrow. I can get ESP32 to samplerate of about 154000 above that the waveform is still good but breaks up with gaps between bits that play OK.

[pschatzmann]

I made a test case with a sine to output as CSV on the plotter and I confirm for certain factors the interpolation seems to have an issue

[HobmanX]

"certain factors" what do you mean...... do you mean certain step sizes.....or.... do you mean factors within the calculations of the interpolations..... or.... the harmonic content of my array...??

my idea what started me looking at this was if Audacity can resample almost any wave why can't ESP32 or any fast MCU do it, if it is only number crunching...?

By the way how do you get the Arduino plotter to show a steady wave plot triggered as in a scope or are you outputing the CSV and plotting it from saved csv data..?

From your plot it seems that the Zero crossings and peaks present the main problem...??

[pschatzmann]

Yes, step sizes.

I committed a correction with the corresponding test case.

I think it looks much better now:

[HobmanX]

Thanks, Do I need to update the library to get this. some weeks ago I used git login to install your library.... didn't really know what I was doing but it worked........ so what do I do now to update with your changes.....??
attached is some more tests I was doing, most played OK except when I was stretching the speeds..?

[pschatzmann]

cd to the directory libraries/arduino-audio-tools and on the command line you type the command: git pull

[pschatzmann]

The mixer does not allow to change the overall volume, but just the weight per input.
If you want an overall volume control just add a VolumeStream before I2S.

There is a Wiki for volume management

[HobmanX]

I have put together a sketch that seems to be mixing 2 resampled sine inputs giving volume control on inputs and also volume control on outputs......Is there any other config needs setup. It plays sines OK but still a bit "rough" when mixing higher harmonic content waves.

AR1sketch.txt

[HobmanX]

I think the "rough" playing when mixing happens when the stepsize is less than 1.0 as I was mixing some low frequencies with higher ones by doing calculations and not realising stepsize was working out less than 1.0. Step size above 1.0 even awkward decimals seems to be OK. Was the less than 1.0 not sorted with your last update OR am I still not picking up the correct library....??
The attached jpg shows glitch if stepsize is less that 1.0 ......in the example below stepsize was 0.375....... the position and size of glitch depends on stepsize and samplerate....??..........above 1.0 resample is now very good...

[HobmanX]

I think I have given up on the Audio Tools methods of resampling and mixing. I just simply can not get it to work and give a clean sound. The waveform do not look bad but there is "noise" generated at some frequencies when higher and lower are resampled and mixed. My frequencies are not high...... all within the audio range.
I have tried Maximillian and it plays better giving a cleaner sound........ but I seem to have problem playing square waves from an array. My square array has even 50% mark/space but when it is played it comes out a short square wave and then a gap to the next wave. .....OK ...now sorted out see next post......
I have tried both small array say 100 points and also large arrays 256 points and 500 points. Sine waves seem to play OK or combinations of sinewaves..
The waves play better than audio tools but instead of a roughness at some frequencies I get a low freq beat as though some were out of tune. Also I have noticed that even playing 2 waves, one at say 440 and other at say 110 the amplitude changes slowly as though they are beating together and slightly out of tune. But If I zoom into the recording and look at the waves I can see that each wave gradually moves out of phase with each other.... I couldn't find any size as to the size of the array tables, do they have to be 256 of larger....??.......probably doesn't matter size but need to change begin and end values in the play4 statement so it knows the size of the array........

[HobmanX]

PS........ The reason the square wave had gaps is I did not set the start and end of the wave in the play4 statement. It was still at 0, 255, and the wave was 100 points so needed to be 0, 99. but there is still drift in time between the 2 samples playing. I am using one wave table of 100 points and playing 110 Hz out of Right and 440 Hz out of Left as put in the play statement, play4(110 ,0, 99) and play4(440, 0 .99). If I look in the recording and count from where both waves rise together...... after about 33 cycles of the 100 Hz wave the relative position has drifted about 1 cycle of the 400 Hz wave...... this may be what gives me the slow beating sound on some freqs....... Is there any way of keeping them in sync..? ............

[pschatzmann]

I can't reproduce your issue in the AudioTools: A sine wave with a frequency of 3000 hz and a step size of 0.375 gives

[pschatzmann]

I tried to compare the values with some precalculated values in Excel and it seems that after each write operation some samples get lost.

[HobmanX]

Hi Phil.... you wrote......." A sine wave with a frequency of 3000 hz and a step size of 0.375 gives....." how did you run that.... was it using the sketch I posted earlier..????. How do you mean a frequency of 3000 Hz..?
In all my sketches in these posts I have been using single cycle arrays generally 256 points...... I have just tested my sketch again now.... with stepsize of 1.0 I get full cycle per cycle out...172 Hz and at stepsize of 2.0 I get twice that , both good sine waves. If I hae stepsize of 0.9 the glitches you see in my last picture posted start to appear maybe a glitch every 3rd cycle. If I try say 0.5 then more glitched..... then 0.375 glitches every cycle as in the picture posted.. If you post your test sketch I can run it here.

But I think I will not be using the Audio tools in any case as it seems very noisy and has all these problems that seem hard to sort out.
Maximilian seems to work much better, it gives a much better sound and waveforms look normal, but it still has the problem that the waves are drifting and causing a "beating sound" even when waves are say, 1,5 times the other or 1.375 times the other.
2 waves in unison play together OK and probably 2 waves an octave apart ........ so it may be no use to me either...?

[HobmanX]

This is the example copied from the Wiki page...Resampling.
I have added a sine array from one of the examples, and modified it so It will run from Array or generator sinewave.
Both give the glitches I have been talking about......but have figured out it glitches only for Resampling Input. Resampling output seems to work but I cannot imagine a scenario where I would need resampling output.
I would be grateful if you could try this sketch and see if it works and let me know what I have coded wrongly.....

#include <AudioTools.h>

int16_t arraysine1[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383,
 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, 
 -24350, -21062, -17363, -13327, -9031, -4560  };

int sampleRate = 44100;
uint8_t channels = 1;
uint8_t bitsPerSample = 16;

AudioInfo info(sampleRate, channels, bitsPerSample);

//SineWaveGenerator<int16_t> sineWave(32000);           // subclass of SoundGenerator with max amplitude of 32000
//GeneratedSoundStream<int16_t> in(sineWave);  

GeneratorFromArray<int16_t> sineWave(arraysine1,0,false);     //  using sinewave array
GeneratedSoundStream<int16_t> in(sineWave);            // Stream generated from sine wave array

//CsvStream<int16_t> out(Serial); 
AnalogAudioStream out;          // internal DAC analog output 25 and 26

//ResampleStream resample(out);       // resampled output
//StreamCopy copier(resample, in);    // copies input stream to resample to be resampled on stream out

ResampleStream resample(in);        // resampled input
StreamCopy copier(out, resample);   // copies the resampled input stream to stream out

// Arduino Setup
void setup(void) {  
  // Open Serial 
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  // define resampling info
  auto rcfg = resample.defaultConfig();
  rcfg.copyFrom(info);
  rcfg.step_size = 0.5;
  resample.begin(rcfg);

  // start I2S internal
  //Serial.println("starting I2S...");
  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  config.channels = channels;
  out.begin(config);


          // Setup sine wave Generator
  //sineWave.begin(channels, sampleRate, N_B4);
            // Serial.println("started...");

         // Setup the sinewave from Array
   auto cfg1 = sineWave.defaultConfig();
   sineWave.begin(cfg1);
  

}

// Arduino loop - copy sound to out 
void loop() {
  copier.copy();
}`
```

[pschatzmann]

I committed a correction: Your example is working fine for me now. Instead of sineWave.begin(cfg1); it would be better just call in.begin(info);

I hope I have the calculations right now for the ResampleStream: If you run the sketch in log level Debug, you see the detail calculation for each sample

[HobmanX]

Yes, working better now for step less than one and also greater then one.
I run it in Debug.....all I see is numbers rolling up the screen and I can only capture a screen full to look at it. I see the calculations..??
Is there no descriptive text anywhere to read up on this stuff...??
I think it has also helped Maximilian...... Is that possible from what you have done...... I will need to do more checks....Thanks

[pschatzmann]

The step index is a float value that is increased by the step size
I am printing -

• the integer index positions one smaller and one bigger then the step size
• the effective values related to this positions (e.g. coming from your table)
• the actual floating point step -> and the calculated/interpolated value for that step size using the values above

For the calculation, I am using a linear interpolation.

[HobmanX]

No, Maximilian has not improved still beating or waving (or like vibrato) on some frequencies mixes.
Audio tools is alot better and no beating or waving sounds. and good waveforms out.
Max sample rate is 44100 so will probably need array of 128 or less to get output frequency up.
Played a 128 point array and recorded in Audacity at 44100 and got the following...
ESP32 samplerate 44100 recorded 128 samples in Audacity....OK
Esp32 samplerate 22050 recorded 256 samples in Audacity....OK
ESP32 samplerate 11025 recorded 223 samples for a 128 array in Audacity........perculiar would expect 512....???
ESP32 samplerate 11025 recorded 446 samples for a 256 array in Audacity.......is 2x for 128 array but............???

What is datatype for samplerate..... in headerfiles it is int and in some sketches it is uint16_t and other sketches it is just a number...?

[pschatzmann]

I think this is a feature of the ESP32 analog API. Using the APLL should resolve the issue: just set config.use_apll = true;

I did not do any thinking when I was setting up the fields of AudioInfo and defined everything as int: so it is about time to correct this: you can use uint16_t in a sketch if you want to use frequencies below 65'535, which in most of the times if ok.
Otherwise use an int or an uint32_t or with the last commit a sample_rate_t.

[HobmanX]

No, config.use_apll = true did not fix it.
It gave me the correct number of samples but glitches everywhere...?
I put it in here......
auto config = i2s.defaultConfig(TX_MODE);
config.copyFrom(info);
config.use_apll = true;
i2s.begin(config);

and tried both false and true.
False is same as not having it in and gives wrong num of samples , wrong frequency
True gives me correct frequency num of samples but glitches
I also tried with other sample rates.... and same effects..... ??

[pschatzmann]

I think this is a problem of the ESP32 core and not my library.
If I remember right different versions of the ESP32 core show different behaviors.

Using a proper DAC with I2S gives more flexibility though some DACs only support some defined frequencies

[HobmanX]

I have isolated the problem to within the mixer or mixer configuration
Input sound and input volum1 copied to i2s worked when mixer was added, only volum1 added mixer connected to i2s problem was back....no output except short bursts...... And I rearranges the connections to have no copier1 or no copier2......

#include <AudioTools.h>

int16_t AR1[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383, 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, -24350, -21062, -17363, -13327, -9031, -4560  };

int sampleRate=44100;
uint8_t channels = 1;               // channels set at 1    
uint8_t bitsPerSample = 16; 

AudioInfo info(sampleRate, channels, bitsPerSample);
GeneratorFromArray<int16_t> waveAR1(AR1,0,false);
GeneratedSoundStream<int16_t> sndAR1(waveAR1);             // Stream generated from array1
VolumeStream volum1(sndAR1);
ResampleStream resample1(volum1);
InputMixer<int16_t> mixer;
AnalogAudioStream i2s;
 StreamCopy copier(i2s, mixer);

  // Arduino Setup

  void setup(void) {  
 // Serial.begin(115200);

    // define resample1
  auto rcfg1 = resample1.defaultConfig();
  rcfg1.copyFrom(info);
  rcfg1.step_size = 0.6;     
  rcfg1.channels = 1;
  resample1.begin(rcfg1); 

    // start I2S internal
  //Serial.println("starting I2S...");
  auto config = i2s.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  config.use_apll = false;
  i2s.begin(config);

  // start I2S External
  /*
 auto config = i2s.defaultConfig(TX_MODE);
  config.copyFrom(info);
  config.use_apll = true;
  config.i2s_format = I2S_LSB_FORMAT;
  config.pin_bck = 18;
  config.pin_ws = 23;
  config.pin_data = 19;
  i2s.begin(config);
  */

 mixer.add(resample1);     //SOUND1
 // mixer.add(volum2);         // SOUND2
  mixer.begin(info);

  // Setup sine wave
  auto cfg1 = waveAR1.defaultConfig();
  //auto cfg2 = waveAR2.defaultConfig();
  waveAR1.begin(cfg1);
  //waveAR2.begin(cfg2);

    // set initial volume
  //volume.begin(config); 
 // volume.setVolume(0.5);    
   volum1.begin(config); 
  volum1.setVolume(1.0);
  //  volum2.begin(config); 
 // volum2.setVolume(vold);

}

// Arduino loop - copy sound to out 

void loop()
   {
     // copier1.copy();    /// NOW DONT HAVE copier1 or copier2
       copier.copy();
    
   }    // end of loop.....

[pschatzmann]

Yes, the issue is the combination of resampling & mixing on the input side. I think the mixing is requesting individual samples, but the resampling needs a big array of data to be able to do the interpolations.

Please add back the Serial.begin(), otherwise you will not be able to see the error messages!

[HobmanX]

This is what I get on serial terminal....long list

[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0
[E] ResampleStream.h : 394 - Failed to write 2 bytes: 0

[HobmanX]

If it is a problem with size of array why does it not happen no matter the step size......?

What is next step....... Is this the error page you wanted me to post......?

[HobmanX]

It looks like..... I can send sndAR1 stream direct to i2s via copier and it works....
I can also send resample1 direct to i2S and it works and shows new values of resampled freq of wave.
I can also include volume of output and the above works at the controlled volume.
When I ....... StreamCopy copier(volume, mixer); and copy mixer out to either volume or i2s direct it works but only for sound stream sndAR1 added as input to the mixer. If I instead add resample1 as mixer input it does not work.....
So the mixer does not like the resample1 input.
I have tried with different arrays etc....

Also...... even with setpsize set at 1.0 I still get the same Error 394 printed out but the sound wave is outputed OK....?

[pschatzmann]

I was redesigning the read parts of the resampler a bit and the combination mentioned above is working now as well.

[HobmanX]

Hi Phil..... I have just done brief test but all seems to be good now.... Thank you very much.
And it now plays a steady tone when mixing.... no waving or vibrato..??........ so it is more steady than Maximillian.
Regarding resampling and mixing is it better or more efficient or quicker for your calculations to use an array that only needs a small step (near1.0) for the freq. required, rather than using an array that needs a step of say 2.0 or more....... OR are the calculations just the same and provide the same accuracy.
I am using a 128 array and a 512 array with step size of 1.0 to 2.5 for the 128 and now that less tha 1.0 is working probably 0.25 to 2.5 for the 512 array. I will also probably use 0.25 to 2.5 for the 128 as well.
And.... can I use any size of array or does this complicate the calculations......... most othe MCUs use a particular size .... 256 or
512......??
I will test it out further over the next while, hopefully all works..............coffee time.....cheers.

[pschatzmann]

The processor need to make the same number of calculations for all step sizes, so from that point it does not matter.
In order to make the right choice, just think about what it does considering the input data.

I did not expect to see so many issues, after all, my individual tests were all working fine...