ESP32-PICO-KIT Arduino and Audio Tools Resampling need faster polling of Touch Pins

[HobmanX]

This is my first post, so here goes.....
I am trying to get sounds of different frequencies to happen as soon as possible after touching a combination of pads
Its not just one pad. I need to scan say 7 or 8 touch pins then evaluate the state what is touched and then play a sound depending on the pads touched. The sketch below does a priority type eval giving a sound for the highest one in the list that is untouched. The resulting sound needs to have minimum latency and may be long or short sounds. So my question is.... Is there any more efficient way of doing this that shown in the code below....? ? ?..... to try and get it to run faster

//xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Also in Arduino Library in file "esp32-hal-touch.c" I changed the SleepCycles and MeasureCycles down from 1000 to 300.
This improved the speed and latency but also gave lower touchRead values needing lower threshold now set at 5.
The sound and waveform all look good and no distortion ......just too slow..?

/*
Internal Private Touch Data Structure and Functions
*/

#if SOC_TOUCH_VERSION_1 // ESP32
static uint16_t __touchSleepCycles = 0x300; // was 1000
static uint16_t __touchMeasureCycles = 0x300; // was 1000
#elif SOC_TOUCH_VERSION_2 // ESP32S2, ESP32S3
static uint16_t __touchSleepCycles = TOUCH_PAD_SLEEP_CYCLE_DEFAULT;
static uint16_t __touchMeasureCycles = TOUCH_PAD_MEASURE_CYCLE_DEFAULT;
#endif

//xxxxxxxxxxxxxxxxxxxxxx

PS ...... I tried using the add code <> brackets and also Ctrl E .... but unable to paste in sketch to look like code in preview..??
so muts be doing something wrong..??

//xxxxxxxxxxxxxxxx

// Sketch for ESP32-PICO-KIT using Arduino 2.0.3 with Audio Tools Library Resampling to DAC pin 25 out.

#include "AudioTools.h"

#define SA 0.4
#define SB 0.6
#define SC 0.8
#define SD 1.0
#define SE 1.2
#define SF 1.4
#define SG 1.6
#define SH 1.8
#define SK 2.0

const int pinPico[] = {27, 14, 12, 13, 15, 2, 32, 33}; // working pico touchpins ???
int Sndplay = 0;
int lastSnd = 1;
float SndArray[]={SK, SH, SG, SF, SE, SD, SC, SB, SA};
int SndIndx=1;

uint16_t sample_rate=44100;
uint8_t channels = 1;
SineWaveGenerator<int16_t> sineWave(32000);
GeneratedSoundStream<int16_t> sound(sineWave);
ResampleStream<int16_t> resample(sound);

//CsvStream<int16_t> out(Serial); // this was original line in audio tool example

AnalogAudioStream out; // changed to this to Analog out via internal DAC pin 25

StreamCopy copier(out, resample);

// Arduino Setup
void setup(void) {

// Setup sine wave
sineWave.begin(channels, sample_rate, N_B4);

// define resample
auto resample_cfg = resample.defaultConfig();
resample_cfg.step_size = 1.0;
resample_cfg.channels = channels;
resample.begin(resample_cfg);

// Define CSV Output xxxxxxxxx is this config still correct now Analog via DAC pin xxxx ??
auto config = out.defaultConfig();
config.channels = channels;
out.begin(config);
}

// Arduino loop - copy sound to out
void loop() {
copier.copy();

// SndIndx = random(8); // xxxxx use this random() to test sounds 0 to 7 playing at loop speed xxxxx fast good ....

SndIndx = TouchESP(); // xxxxx use this line TouchESP() to test speed with touchRead polling xxxxxx Too Slow ....

if (SndIndx != lastSnd)
{
resample.setStepSize(SndArray[SndIndx]); // update resample rate
lastSnd = SndIndx;
}
}

int TouchESP(void) {             // start get Touch ESP pins ................
  Sndplay = 0;
for (int i = 0; i <= 7; i++)
   {
    if (touchRead(pinPico[i]) <= 5) 
   {
  bitSet(Sndplay, i);  
   }
     }   

//this next code gets highest priority bit of combination untouched xxxxxxx
for (int k = 7; k >= 0; k--) {
if (bitRead(Sndplay,k) == 0)
{
SndIndx = 7-k;
return (SndIndx);
}
else
{
SndIndx = 8;
}
} // xxxxxxxxxxxxxx
return(SndIndx);
} // end TouchESP pins.....................

//xxxxxxxxxxxxxxxxxxxxxxxxxxxx

[pschatzmann]

I suggest that you explore if you can make it work with attachinterrupt

[HobmanX]

Thanks for reply. I did think about interrupts but my problem is what has to occur to provide the interrupt and what process do I attach it to. As I see it I have a bundle of 7 or 8 pins that at any moment in time ESP32 needs to know the state of what is touched and what is untouched and play a sound depending on that combination, not just any specific pin touched or untouched. I suppose for 8 pins 0 to 7 there are 255 possibilities of sounds depending on the encoding/decoding. For example say if none touched (call that min touch) and play a low freq sound, then if all touched (call that max touch) play a high freq sound. The pins in between may not be all evenly "weighted" (as bits in binary) so say touching the top 3 pins would play a different sound from touching the bottom 3 pins. For the example shown in my code I used a priority encoder to simplify things which sets a higher priority on each untouched pin up the list and play a sound for that priority and "don't care" about other lower untouched at that instance. This gives 8 sounds for untouched and if a sound for all touched then 9 sounds alltogether. But various methods of encoding should be possible or just use all 255. The problem is still how to poll a group quickly or how to generate an interrupt for the actual state of a group at any one instance.

[pschatzmann]

Reading pins is quite fast and if the Arduino API is too slow you can still access the register values directly as in this example: https://github.com/pschatzmann/logic-analyzer/blob/main/src/config_esp32.h

[HobmanX]

Thanks for reply but to understand setting up access to register values directly I would need to read up on it a bit more ??
I put the following in at the top of the loop to print ont the millis() at the start of each repeat of the loop.....
unsigned long bbb = millis();
Serial.println(bbb);
I get a loop time of about 12 milli secs ... and does not seem to matter whether with the line for polling the touch pins or the line for the random() number.... both gave about 12 milli secs loop time........which does not seem too bad....??
I checked the CPU speed it tells me it is 240 MHz. On projects with other MCUs not running even as fast ( maybe 120 MHz) I have not had any problems with polling speed (and poll even more pins) so cannot understand why here. I think it is simply latency. If I touch 2 pads repetively one after thr other as fast as I can manually I can hear the changing pitches so I don't hear if it is late because it coincides with the next sound which is repeating. But if I touch several different ones one after the other then I hear it coming late, actually when I am hitting the next pad. So it looks like it can change fast enough but the sound is delayed too much. Its like a mp3 player playing a song....it does not matter if the sound is half a second late as long as it plays the song at the correct speed. But it matters here...... I would need to hook up my scope and try and detect the actual waveform out the DAC pin to see how long it is after the Pin is touched for that sound. I have only tried your sinewav that was in the example is there any digital filtering or ADSR envelops being used in the background etc. that may cause delays...? ?

[pschatzmann]

It's in the documentation: Wiki - Performance

[HobmanX]

Thanks for reply, Is this the type of thing I have to do...... Just add the #defiines into the sketch
I tried these, I think they made a slightly faster transition but still sounds as though delays.
I may need a "more harsh" sound. The sine wave is a soft sound so giving a soft change between freq changes that I am maybe perceiving as delays..? ?
Does this same sketch work on arbitrary Audio arrays...?
The actual touch contact is not as good as in other MCUs, It needs a heavier touch or moist skin contact.?? which may be giving me unreliable touching at fast rates.

#define USE_AUDIO_LOGGING false
#define DEFAULT_BUFFER_SIZE 512 //was1024
#define I2S_BUFFER_SIZE 256 // was 512
#define I2S_BUFFER_COUNT 5 // was 10 // 20

[HobmanX]

When checking audio delay at output pin 25 DAC on scope I am unable to see the instant the sound changes relative to being triggered to change. I see the freq changing but not as an instant I can capture. So I toggled a pin using ( digitalWrite(21, !digitalRead(21)); ) at the top of the loop and measured 12 ms for each repeat of the loop with the Touch polling running the audio resampling library.
I then changed the resampling statement ....... resample.setStepSize(SndArray[SndIndx]); // update resample rate .......... to
........ tone(25, SndArray[SndIndx]); ...... (having setup the sketch for tone ) and measured the loop time again and now with running the same Touch polling and with the tone sound control I get a loop time of only 9 us. Considerably shorter so most of the delay must be in the Audio Library. I don't see any change on scope times if I use the #includes etc but I am probably not using them correctly... ? ? ?

[pschatzmann]

You should change the buffer size via the config
The copy buffer size is specified in the constructor of the StreamCopy

With 32000 samples per second, 16bit , a buffersize of 128 and a buffer_count of 2 you should get a lag of 8msecs end to end

[HobmanX]

I changed it in AudioConfig.h to 128 buffersize and count of 2. It has helped, it is now more or less workable with no speed to spare.

in AudioConfig.h

#ifndef I2S_BUFFER_SIZE
#define I2S_BUFFER_SIZE 128 // 512 xxxxxx
#endif

#ifndef I2S_BUFFER_COUNT
#define I2S_BUFFER_COUNT 2 // 10 // 20 xxxx
#endif

Will this not get overwritten due to updates etc of Arduino ide......? ??

[HobmanX]

PS .... I am still getting 11.5 ms to 12 ms for the sketch loop time as measured on the scope from the toggle on pin 21.
even though the speed of the sound changes has improved

[pschatzmann]

You would change the values only in the AudioCofnig.h if you intend to use these values in all your sketches. Otherwise you would change it in your sketch via the config object. The same is valid if you optimize the copy buffer size which defines how many bytes you process in each loop step.

The alternatives for improving on the pin reading times we have already discussed...

[HobmanX]

Thanks for your help. My problem is my old school studies it was all C and procedural code and not much on Classes and Objects or OOPs.
Don't normally prog down to this level but I am starting to get some idea and am browsing for articles on the web to describe the basics.
Have you any pointers or links to easy reading docs etc to describe the basic concepts and maybe more specific to your libraries

[pschatzmann]

I don't really have any recommendation because I was learning by doing. Copared to simple c the big difference is that the structs (=classes) not have only variables but they can have methods as well and that you have some cool class initialization and destruction functionality and arrage your structs/classes in hierarchies.

If you want to learn my library the best starting point is the

• Wiki
• Class documentation

[HobmanX]

Have been reading but find it difficult to translate into coding in the sketch. Touch is fast and modified the .h file and .cpp file so delay is now OK until I try more. I have played the Array to Analog example worked OK and changed the array to different sound and put in a 256 point array. This to me is about 172 Hz so put in factor of 2.75 to get it to play at 440. It was an arbitrary wave (not sine) fairly "ragged" so I thought would need more points that in the examples 45 point sine. Is that the way to do it to maintain a good wavefore at higher freqs. Does it actually play the wave at 2.75 the speed or does it resample and drop out points etc to increase the freqs. I tried the 2 tone example where iy mixes 2 sinewaves. That worked but due to too much amplitude of the wave the output did not look anything like a sine. So in the output of my Sine and myOtherSine I multiplied each by 0.3 so when they were added the came out as proper sines mixed.
This sketch used the sinewave oscillator. I tried to do same with addind 2 sine arrays but could not figure out the combination of the objects to get it to work. I couldn't figure out where to add them together or put them in the output statement as in the 2 tones sketch...??

[pschatzmann]

Yes to increase the precision you can increase the sampling rate (or in your working more samples).

I don't really know what you mean that the sketch used the sinewave oscillator: If you check in the documentation, where are at least 3 or alternative implementations to choose from and even more options if you use the supported DSP libraries.

If you want to combine multiple inputs using the provided classes, I suggest that you read the corresponding chapter about splitting and merging in the Wiki.

[HobmanX]

My problem at the moment is how to (add or combine as in a mixer) 2 waves together ( not just add them as 2 separate outputs) and send output to pin 25 the DAC analog out. Your example-maximilian - 2Tones does this and I can get it to work ( I changed the out to AnalogAudioStream out;. Initially the output was too large amplitude so output was a badly distorted sine but in the play output statement line I put in a factor multiply each wave by 0.3 then add them( as in + plus ) and the output was a perfect sine combination of the 2 waves.
In the 2 Tones sketch the sines are declared "maxiOsc mySine,myOtherSine;" and in the play output function " output[0]=(mySine.sinewave(440)*0.3)+(myOtherSine.sinewave(441)*0.3);) " . What I want to do now is do the same but instead of using the MaxiOsc I want to add together 2 Audio Arrays. My problem is I have read through the other example sketches but cannot decipher or juggle the right combinations to get the arrays to work in the play output type function....???
You refer to "the documentation" what documentation do you mean, is it just the wiki pages, while they are helpful the just basically describe the examples there. I feel I would need more descriptive help that describes the basic concepts in this instance. For example I can take the 2 Tones sketch and instead of the MaxiOsc lines of code, I can use the similar lines from "Generate from Array sketch" to get input from the arrays and change out to DAC but when I try to change the "Play output" bit it tells me "sinewave" not declared in scope etc no matter what combination I pick...???

[pschatzmann]

The readme has a link to the class documentation: https://pschatzmann.github.io/arduino-audio-tools/html/modules.html
The Wiki also has examples that show how to mix Streams on the input and output side: https://github.com/pschatzmann/arduino-audio-tools/wiki/Splitting-and-Merging-Audio

[HobmanX]

I have read through the docs and through the examples anf have come up with this as a means of reading in 2 Audio Arrays (sinewaves) agenerating the 2 sinewaves, then feeding them in to the resampler so that I can change the frequency of each. I then add the resampled out to a mixer as 2 inputs. So that I can vary the amplitude of the mixed combination I added a value for "weight". I then followed on in the sketch setting up the configurations and configured the output "copier" to output the resample Mixer to AnalogAudio StreamOut.
This appears to work and gives me a good quality mixed sinewave combination out on pin 25 DAC. If I set channels - 2 at the top as in original example test sketch I get interferrance on the sinewave, so set it at 1 and get good waveform.
I havent tried my touch routines to it yet but I notice if I change the stepp value in the loop the resampler does not "see" the change, so wondering if this bit is working...??. Would welcome youe comments as to "Is this the right approach.?" or what else??.
I removed any ref to Maximillian and Audio kit. ......Are these required..?? or what are they required for..??

<
/**

• @file streams-generator_inputmixer-audiokit.ino ..... adapted from this sketch
• @brief Tesing input mixer
• @author Phil Schatzmann
• @copyright GPLv3
  */

#include "AudioTools.h"
//#include "AudioLibs/AudioKit.h" // not using audiokit. Sketch works to DAC analog out
float stepp = 2.0;
uint16_t sample_rate=44100;
uint8_t channels = 1; // The stream will have 2 channels
int16_t sine_array1[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383, 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, -24350, -21062, -17363, -13327, -9031, -4560 };
int16_t sine_array2[] = {0, 4560, 9031, 13327, 17363, 21062, 24350, 27165, 29450, 31163, 32269, 32747, 32587, 31793, 30381, 28377, 25820, 22761, 19259, 15383, 11206, 6812, 2285, -2285, -6812, -11206, -15383, -19259, -22761, -25820, -28377, -30381, -31793, -32587, -32747, -32269, -31163, -29450, -27165, -24350, -21062, -17363, -13327, -9031, -4560 };
GeneratorFromArray<int16_t> sineWave1(sine_array1,0,false);
GeneratedSoundStream<int16_t> sound1(sineWave1); // Stream generated from sine array1
GeneratorFromArray<int16_t> sineWave2(sine_array2,0,false);
GeneratedSoundStream<int16_t> sound2(sineWave2); // Stream generated from sine array2
ResampleStream<int16_t> resample1(sound1);
ResampleStream<int16_t> resample2(sound2);

InputMixer<int16_t> mixer;
//AudioKitStream out;
AnalogAudioStream out;
StreamCopy copier(out, mixer); // copies sound into i2s

// Arduino Setup
void setup(void) {

mixer.add(resample1, 0.25);
mixer.add(resample2, 1.0);

// define resample1

auto rcfg1 = resample1.defaultConfig();
rcfg1.step_size = 0.25;
rcfg1.channels = channels;
resample1.begin(rcfg1);

// define resample2
auto rcfg2 = resample2.defaultConfig();
rcfg2.step_size = stepp;
rcfg2.channels = channels;
resample2.begin(rcfg2);

// start I2S
//Serial.println("starting I2S...");
auto config = out.defaultConfig(TX_MODE);
config.sample_rate = sample_rate;
config.channels = channels;
config.bits_per_sample = 16;
out.begin(config);

// Setup sine waves
auto cfg1 = sineWave1.defaultConfig();
auto cfg2 = sineWave2.defaultConfig();
cfg1.channels = channels;
cfg1.sample_rate = sample_rate;
cfg2.channels = channels;
cfg2.sample_rate = sample_rate;
sineWave1.begin(cfg1);
sineWave2.begin(cfg2);
//Serial.println("started...");

}

// Arduino loop - copy sound to out
void loop() {
copier.copy();
stepp = 1.0; // the resample does not seem to see this change in stepp value from orig 2.0 to 1.0 ???
}

[HobmanX]

The "penny dropped"..........I need something like this in loop to change the amplitude and frequency of the resampled waves...
resample2.setStepSize(1.5);
mixer.setWeight(resample1, 0.9);
This compiles and appears to work ......will try touch control of this next..............

[pschatzmann]

Right, just as a final comment: if you just want to change the frequency of the the sine, you would use e.g. SineFromTable< T > or one of the other implementations and call the setFrequency() method. The resampling is mainly interesting if you have some recorded sounds (e.g. as files) and you want to vary the playback speed...

[HobmanX]

I have now looked at the SineFromTable as you suggested, and got it to work.
I was getting very slow response to changes in amplitude as though it ramped to the change. I looked at the SoundGenerator.h file to see if I could see what my cause delay and seen the amplitude change step is limited to Max of 50 ... so.......

In SoundGenerator.h I set the, " float max_amplitude_step = 32000; " (was set at 50....??)
Now getting quick sharp response to change in amplitude, But still gives a delay of 12 ms for each cycle of the loop when running the sketch shown below.......

I checked (using Audacity) the response for frequency by changing frequency at same time as amplitude change (in the loop) and see it changes quickly with it...OK. The time for change in one freq to next freq. or one ampl. to next ampl. is the loop time and for me works out at about 11 ms or 12ms.

Is there any advantage in using this SineFromTable library...???? for what I am doing...??
Is it only suitable for sinewave tables ??. I want to use arbitrary array random value type tables or arrays for copies or recording of "real" sounds. I see in the .h file the values array table is 181 points and only for half a cycle of a sine wave. A sine wave is symetrical though inverted for each half so the second half can be calculated, but "real sounds" are generally not symetrical so the "SineFromArray" for the full 1 cycle I thought was maybe better.
Is the 181 point array values[181] in SoundGenerator.h file just there for calculation of progress through a cycle or is it the actual array for the generated sound..... can I add my own array in the sketch...??

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#include <AudioConfig.h>
#include <AudioLogger.h>
#include <AudioTools.h>

int freqf = 0;
int volum = 0;
int channels = 1;
int sample_rate = 44100;
int bits_per_sample = 16;

SineFromTable<int16_t> sineWave(32000); // subclass of SoundGenerator
GeneratedSoundStream<int16_t> sound(sineWave); // Stream generated from sine wave

AnalogAudioStream out;
StreamCopy copier(out, sound);

void setup() {
Serial.begin(115200);

// start I2S
//Serial.println("starting I2S...");
auto config = out.defaultConfig(TX_MODE);
config.sample_rate = sample_rate;
config.channels = channels;
config.bits_per_sample = 16;
out.begin(config);

// Setup sine wave
auto cfgS = sineWave.defaultConfig();
cfgS.channels = channels;
cfgS.sample_rate = sample_rate;
cfgS.bits_per_sample = bits_per_sample;
sineWave.begin(cfgS, N_B4);

}

void loop() {
copier.copy();

freqf = freqf + 100;

sineWave.setFrequency(freqf);
if (freqf >= 1000) freqf = 0;

volum = volum + 3200;
sineWave.setAmplitude(volum);
if (volum >= 32000) volum = 0;

}

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx