Running RTSP server, but no audio

[josef2600]

1th, thank you for your impeccable work. it is jaw dropping!
i wanted to send audio from an analog microphone to web (i want to stream audio). with minimal delay. i don't care what esp32 i should use, but i can not use I2S or I2C or any digital mic.
i made an op-amp and tested the microphone with your sample streams-adc-serial and got the audio signal on plotter. it is very good.
then for few days i tried everything! but i cant get it to work. this is what i did:
code:

#include "AudioTools.h"
#include "AudioLibs/RTSP.h"
#include "AudioStreamer.h"
#include "RTSPServer.h"

int port = 554;
int channelss = 1;
int sample_ratee = 8000;
int bits_per_sample = 16;
const char* wifi = "Tenda_C70488";
const char* password = "Tenda_C70488";

AnalogAudioStream in; 
AudioInfo info(sample_ratee, channelss, bits_per_sample);
//MP3EncoderFDK encoder; // Add this line
RTSPFormatGSM formatt; // Add this line
RTSPSourceFromAudioStream source(in, formatt); // Modify this line
AudioStreamer streamer = AudioStreamer(&source);
RTSPServer rtsp = RTSPServer(&streamer, port);

void setup()
{
  Serial.begin(115200);
  delay(1000);
  AudioLogger::instance().begin(Serial, AudioLogger::Warning);// Info Debug Warning Error 

  // Setup microphone input
  auto cfgRx = in.defaultConfig(RX_MODE);
  cfgRx.adc_pin = 34; //gpio34-- optionally define pin
  cfgRx.copyFrom(info);
  cfgRx.channels = channelss;
  cfgRx.sample_rate = sample_ratee;
  in.begin(cfgRx);

  // Start Wifi & rtsp server
  rtsp.begin(wifi, password);
}

void loop()
{
  vTaskDelay(1000);
}

this is the output loge:

connect to rtsp://192.168.0.3:554
[  4666][I][RTSPServer.cpp:43] runAsync(): Running RTSP server on port 554
[  4666][I][AudioStreamer.cpp:180] InitAudioSource(): InitAudioSource
[  4670][I][AudioStreamer.cpp:191] InitAudioSource(): m_fragmentSize (bytes): 320
[  4678][I][RTSPServer.cpp:95] serverThread(): Server thread listening...
[ 18897][I][RTSPServer.cpp:103] serverThread(): Client connected. Client address: 192.168.0.2
[ 18897][I][RTSPSession.cpp:36] RtspSession(): RTSP session created
[ 18900][I][RTSPServer.cpp:135] sessionThread(): Session ready
[ 18906][I][WiFiClient.cpp:537] connected(): Unexpected: RES: 0, ERR: 0
[ 19912][I][RTSPSession.cpp:120] ParseRtspRequest(): RTSP received OPTIONS
[ 20964][I][RTSPSession.cpp:120] ParseRtspRequest(): RTSP received DESCRIBE
[ 20964][E][RTSPFormat.h:152] format(): unsupported audio type
[ 20965][E][RTSPFormat.h:136] payloadFormat(): unsupported audio type
[ 22024][I][RTSPSession.cpp:120] ParseRtspRequest(): RTSP received SETUP
[ 23078][I][RTSPSession.cpp:120] ParseRtspRequest(): RTSP received PLAY
[ 23079][I][AudioStreamer.cpp:196] Start(): Starting RTP Stream
[ 23080][I][AudioStreamer.cpp:180] InitAudioSource(): InitAudioSource
[ 23085][I][AudioStreamer.cpp:191] InitAudioSource(): m_fragmentSize (bytes): 320

i use VLC with this line:
rtsp://192.168.0.3:554
it is on default config (1000ms ...).
it connects, but no audio. as it says in the loge, unsupported audio type.
my goal is to send the audio to few client devices. it is an iphone that sends the audio to few apartments. they have android devices to receive the audio. usually there is only one device that can connect to audio. my most important thing is the speed! later i want to receive audio too, but now is not the time!
can you please help me with this?
and the sample communication-generator-rtsp works and makes an output audio. but it is very cut off audio.
and if you think i should use UDP, i will go for that. although i couldn't start the UDP server! i don't know why VLC says there is no server.
right now i am using esp32dev board with psram. if i have to, i have esp32s3 with 8mbyte psram too. whatever you say.
thanks.

[pschatzmann]

I could never get any decent sampling rates using RTSP.

If you want to play the audio on some android or ios devices you will need an app. Unfortunately there are not many apps that use a open source, documented protocol.

I recently found VBAN and added the support for this protocol today.

Some examples can be found here

Unfortunately multicast is not providing any good results in my environment, but with unicast I can go up to 44100 samples/second and stereo w/o problems.

[josef2600]

thank you very much. i have seen VBAN project, but i couldn't understand it much. i will work on your port. thank you for that.
for the app side, i don't care about it! if VLC can play it, it can be don. and since your codecs are open source, any good android programmer can do the job.
your project with web-server does the job. currently i was working on that. but the audio is at slow rate! i think it is because of the speed of ADC. i think i can fix it with creating a task only for reading and simple conversion of ADC value to audio file (like what you did with sin wave), but it would be on core 1 and i think since your code only uses core 0 , it would be perfect, because the speed can be up to 25k sample (base on my own test on the speed of ADC). but since the speed wouldn't be exact, there will be a need of a timer and delay of micro-sec for it to make the timing exact. i think it can be don with 16k sample per second. but the other main problem is the delay. the web server audio has a huge delay. i cant think of any way to fix that.
about 1.5 year ago i found your code for audio for Bluetooth and just fell in love with it! it was (still is) jaw dropping. thank you.

[pschatzmann]

I am not sure what you mean with speed: if it is a minimal latency (=delay/lag), then the server/browser solution looses because of the big buffers used by the browser.

If it's a high sample rate, then a solution using a codec would be preferred because of the limited network bandwidth.

The ADC uses DMA, so you can get very high sampling rates (e.g. 48000 samples/second) w/o any problems and you can minimize the lag by decreasing the I2S buffer_size and/or buffer_count

[josef2600]

i read the code for analog reading. as i understand, you are using value = analogRead(self->config.start_pin + j); for reading analog and using a timer for timing the time base. that is causing the problem. again, as i have worked so far, as the timing i am writing this(!) , there are many problems with Arduino.
first, analogRead is a very time consuming and a blocking operation. and when you are using it, it mess with all your code and timing. i have worked for few month on that thing, but man, chines designers. bad bad. it is impossible to use DMA with analog in a real scene, because it does not support it. forget what they say in datasheet, it is lies, or at best, only possible with the ESP-IDF itself and nothing else. the reason is you can not access the low level codes in Arduino. when you use the analogRead or raw or whatever code for reading the analog data, it does not access the analog module directly, instead, it has a huge code for it and the part that access the low level hardware, is code protected and is not open source (!!!). i couldn't even access the registers for analog module.
anyway, when you use internal analog, with whatever means, even DMA, it uses the shitty code it has. for example, if you direct it to DMA, instead of being direct and not use CPU, it uses all its code and CPU, then forward it to DMA. meaning, DMA is useless in this one. and when you are using timer interrupt for timing for reading 'n' times of value, it doesn't work. instead it might get 2 times faster or slower or ... times samples and mess everything up. and if you run a simulator, this wont happens. for seeing this problem, you have to run it on real hardware.
and just for the kick of it, did you know ESP32 all versions before ESP32-C3 ESP32-C4 ESP32-S3 , the analog can not read under 145 milli volt?!! it doesn't has zero. and when using WIFI , the ADC2 module wont work on any module!
this is the reason the analog audio is odd and is like the play speed is *0.5 . it is not understandable. but because you use the sine wave, you haven't seen it yet.
that is why i said we should work on using core 1 for analogread, so that code doesn't mess the timing for other parts and doesn't stop the main code.
here are some timing i measured. but know they are not accurate and timing changes about 20% time to time!

    my_static_readadc_cxX += analogReadMilliVolts(34); //10k reading is almost 1100ms (~ 1065 ms) ~ 100us or 10k sample per sec
    my_static_readadc_cxX += analogRead(34); //10k reading is almost 1100ms (~ 926 ms) ~ 100us
    my_static_readadc_cxX += adc1_get_raw(ADC1_CHANNEL_0); //10k reading is almost 500ms (~ 450 ms) ~ 50us or 20k sample per sec

as you can see, analogRead can not sample more than 10k sample per second. and it is while it only reads analog and does noting else.
thank you.

[pschatzmann]

That's nonsense: what makes you think I would do something like this on an ESP32 ?
Just have a look at the implementation!

Would you eventually volunteer to provide an implementation for the new API that comes with the latest IDF release?

[josef2600]

sorry, i didn't meant to make you angry! i don't work with IDF too. my brain cant handle that thing! i will move on to esp32-a1s (the audio kit). i have that too! i am just a little crazy!
thank you.

[pschatzmann]

Oh, I am not angry.

The thing is with Arduino ESP 3.0.0 which is alpha now, the old analog API which is based on I2S will not work any more and we need to switch to the new one!