VBAN stream from PC and recieve on ESP

[trackme518]

Hi, I am trying to receive VBAN stream from PC on ESP32. The code compiles but I am not getting any audio from ESP (output to i2s). I have tested the generator -> i2s which is working. All the VBAN examples are for streaming audio from ESP to PC not the other way round.

#include "AudioTools.h"
#include "AudioLibs/VBANStream.h"

AudioInfo info(44100, 2, 16);
VBANStream in;
I2SStream out;
StreamCopy copier(out, in);  // copies sound into i2s

 if (!out.begin(cfg)) stop();

  auto cfg_in = in.defaultConfig(RX_MODE);
  cfg_in.ssid = "mywifiname";
  cfg_in.password = "mywifipass";
  cfg_in.stream_name = "Stream1";
  //cfg_in.throttle_active = true;
  //cfg_in.throttle_correction_us = 0;  // optimize overload and underrun
  cfg_in.copyFrom(info);
  in.begin(cfg_in);

  Serial.println("VBAN WiFi receptor");
  Serial.print("Connected! IP address: ");
  Serial.println(WiFi.localIP());

void loop() {
  copier.copy();
}

I do get the message with my IP address so in the setup it seems the stream in is working - I can even do the VBAN ping but I still don't get any sound out...

Any help appreciated.

[pschatzmann]

Unfortunately I don't have any way to test this, so I suggest that you try to debug it yourself to find out what's going wrong.
Have a look what's happening in the receive_udp method in the VBANStream class:

• is it called at all ?
• does it fill the buffer ?

[pschatzmann]

I think in the beginRx method the connect is missing. Can you check if the following is working ?

    bool begin_rx(){
        start_wifi();
        if (WiFi.status() != WL_CONNECTED){
            LOGE("Wifi not connected");
            return false;
        }
        WiFi.setSleep(false); 
        // connect to target
        if (!udp.connect(cfg.target_ip, cfg.udp_port)){
            LOGE("Could not connect to target");
        }
        // handle data
        udp.onPacket([this](AsyncUDPPacket packet) {
             receive_udp(packet);
        });

        return true;
    }

[trackme518]

Hi, thank a lot for the help! I have defined the port as cfg_in.udp_port = 6980; not the IP - I am receiving the VBAN stream and outputting into i2s, I think that I even can not define the target IP for incoming stream...I am defining the IP inside the voicemeeter app - which I have tested with mobile app VBAN receptor to verify that I am indeed sending the audio. Do I need to rebuild the library somehow or just editing the .h file is enough?

[trackme518]

   bool begin_rx(){
    	LOGE("Logging into console");
        start_wifi();
        if (WiFi.status() != WL_CONNECTED){
            LOGE("Wifi not connected");
            return false;
        }
        WiFi.setSleep(false); 
        // connect to target
        if (!udp.connect(cfg.target_ip, cfg.udp_port)){
            LOGE("Could not connect to target");
        }
        // handle data
		udp.onPacket([this](AsyncUDPPacket packet) {
             LOGE("packet recieved");
			 receive_udp(packet);
        });

        return true;
    }

I am not seeing any print statements on udp recieve so I assume that the function is not even triggered.

[trackme518]

voicemeeter config

[trackme518]

I can ping the 192.168.1.111 and I get the reply in 6-11ms (expected - the device is indeed on the local network)

[trackme518]

Whole sketch:

#include "AudioTools.h"
#include "AudioLibs/VBANStream.h"

AudioInfo info(44100, 2, 16);
VBANStream in;
I2SStream out;
StreamCopy copier(out, in);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);

  Serial.println("configuring output");

  auto cfg = out.defaultConfig(TX_MODE); 
  cfg.copyFrom(info);
  cfg.pin_bck = 35;   //BCK - default 14
  cfg.pin_ws = 37;    //LCK - default 15
  cfg.pin_data = 39;  //DIN - default 22
  cfg.sample_rate = 44100;
  out.begin(cfg);  //starts i2s

  Serial.println("configuring input");

  auto cfg_in = in.defaultConfig(RX_MODE);
  cfg_in.copyFrom(info);
  cfg_in.ssid = "WIFINAME";
  cfg_in.password = "WIFIPASS";
  cfg_in.stream_name = "Stream1";
  cfg_in.udp_port = 6980;

  in.begin(cfg_in);
  Serial.println("VBAN WiFi receptor");
  Serial.print("Connected! IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  copier.copy();
}

[pschatzmann]

Just looked at the ESP32 UDP example. Maybe the following might work better:

   bool begin_rx(){
    	LOGE("Logging into console");
        start_wifi();
        if (WiFi.status() != WL_CONNECTED){
            LOGE("Wifi not connected");
            return false;
        }
        WiFi.setSleep(false); 
        // connect to target
        if (!udp.listen(cfg.udp_port)){
            LOGE("Could not connect to target");
        }
        // handle data
		udp.onPacket([this](AsyncUDPPacket packet) {
             LOGE("packet recieved");
			 receive_udp(packet);
        });

        return true;
    }

[trackme518]

Ok, now I can hear extremely loud high pitch sound that stop and starts when I stop or start the stream + can I hear difference when I lower the sampling frequency. However, it is not the sound I am sending so probably some format problem. It does not react on changing volume.

[pschatzmann]

The only supported format is PCM

[trackme518]

I think that you can not send anything else with VBAN anyway...from the screenshot in voicemeeter it shows 16bit PCM (when I switch to 24bit it is silent). I can also see that 44100Hz frequency is not listed in VBAN definition enum (enum VBanSampleRates) so I switched to 48000Hz. Now I am testing with AudioInfo info(48000, 2, 24); and that remains silent.

[pschatzmann]

No idea what this could be. Maybe VBAN is sending in network format (=big endian) and I am expecting little endian

[trackme518]

You have a note in the source code that reads And set samplesPerPacket to 64 for 11025, 128 for 22050, or 256 for all else. However, the "samplesPerPacket" is not a parameter inside the VBANStream class. Perhaps it means setting StreamCopy copier(out, in, 256); I assume that the last parameter is for buffer size? This yeilds following:

[I] StreamCopy.h : 143 - StreamCopy::copy  256 -> 256 -> 256 bytes - in 1 hops
[I] StreamCopy.h : 143 - StreamCopy::copy  256 -> 256 -> 256 bytes - in 1 hops
[I] StreamCopy.h : 143 - StreamCopy::copy  28 -> 28 -> 28 bytes - in 1 hops
[I] StreamCopy.h : 143 - StreamCopy::copy  256 -> 256 -> 256 bytes - in 1 hops
[I] StreamCopy.h : 143 - StreamCopy::copy  256 -> 256 -> 256 bytes - in 1 hops

But it is still just high pitched artifacts noise. I tried to set the channels to 1 as well but it remains more or less the same.

[pschatzmann]

Maybe it would make sense to have the buffer filled to 50% before switching it to active = available() reporting the available bytes

[trackme518]

To be clear I am using GY-PCM5102 I2S audio modul for i2s output (tested it's working).

[trackme518]

Also some time back I had a compilation error that I fixed by modifying this function:

        // limit output speed
        if (cfg.throttle_active){
        	//this method works
            throttle.delayFrames(samples / cfg.channels);
			//depraceted? dunno - but it does not work for esp32 s2 
			//throttle.delaySamples(samples / cfg.channels);
        }

I don't think it is related but noting it here for reference.

[pschatzmann]

I changed the logic a bit:

• Increased the buffer size to 1045
• available() is reporting a value only after the buffer has been filled to 50%
• issue error messages on buffer overflow

But I don't think that this will make a big difference...

[trackme518]

I am not sure where to look for this function. There is:

    size_t readBytes(uint8_t* data, size_t byteCount) override {
        return rx_buffer.readArray(data, byteCount);
    }

    int available() {
        return rx_buffer.available();
    }

From where is sourced the .available();?

Maybe it would make sense to have the buffer filled to 50% before switching it to active = available() reporting the available bytes

[pschatzmann]

just look at my latest commit

[trackme518]

I have copied the latest VBANStream.h file and replaced it in my local lib and reflashed the sketch. Unfortunately, It does not seem to change the behavior.

[pschatzmann]

but it should show copy 1024 now....

[trackme518]

Interestingly no - it shows [I] StreamCopy.h : 143 - StreamCopy::copy 540 -> 540 -> 540 bytes - in 1 hops even when i set StreamCopy copier(out, in, 1024); and AudioInfo info(48000, 2, 16);

[trackme518]

the same is true when i left the buffer size param empty like so StreamCopy copier(out, in);

[trackme518]

Even when I modify your line to:
if (bytes_received >= cfg.rx_buffer_count * DEFAULT_BUFFER_SIZE*2) insted of *0.5 (you are using int max_write_size = DEFAULT_BUFFER_SIZE * 2; at the top)
I only get StreamCopy::copy 540 -> 540 -> 540 bytes - in 1 hops

So I assume that a) check is not working b) check is always triggered once and later ignored (you set the flag to true than never reset it back). It does not seems to be triggered anyway (I tried to put LOGE fce there to see if it triggered and did not get anything out of it). I still don't understand how that should affect the function but I am just reporting back my findings.

[pschatzmann]

The idea was to report available() data only after the buffer has been filled by 50%. Before it will report 0, so nothing is copied.
As soon as the buffer had once more then 50%, available() will report the available data in the current buffer line of the n buffer which is DEFAULT_BUFFER_SIZE, that should be 1024 bytes, if you did not change this and the copier will limit the size to this..

[pschatzmann]

In order to reproduce this I had to buy the Talky Android app!
The worst error was, that the data was read from the wrong address. It is working much better now...
I also changed the buffer to use a thread save implementation and extended the API where you can define an output object, so that no copy is needed.

[trackme518]

VBAN reciever is free in lite version...https://play.google.com/store/apps/details?id=vbaudio.vbanreceptorlite&hl=en&gl=US you can use voicemeeter to stream from PC (and recieve on another PC or on mobile with mentioned lite version android app)

[trackme518]

Ok, I will try to download latest source and flash again and test it against esp32s2 - however, even at the start I was able to receive the stream on the mobile from voicemeeter without problem, so I am not sure what you have use that Talky Android App for.

[pschatzmann]

Yes, I know and I was testing my sender sketches with this.
But you wanted to receive vban Audio on the ESP32 and there is not free version for this.
Since I do not have access to any Windows machine, this was the only option.

[trackme518]

Ah I see, sorry to hear that. I plan to implement the VBAN stream in Processing Java so we can have build for any OS.

[trackme518]

Hi, I can confirm it is now working on esp32s2. Minor issue remains that it does not conform to changing sampleRate to anything elese than 44100Hz - if I go lower or higher I will get artifacts. I can see you have function in the source code for dynamically changing this from the VBAN header but somehow it is not working - the same is true when I try to change it in config when starting the stream (ie hardcode in the beginning).

 // update sample rate
      if (cfg.sample_rate != vbanSampleRate || cfg.channels != vbchannels) {
        cfg.sample_rate = vbanSampleRate;
        cfg.channels = vbchannels;
        setAudioInfo(cfg);
      }

[pschatzmann]

Strange, I did test this and it was working. Can you retry ? and mybe find out when it is not working...
I changed the sample rates and the number of channels...

[pschatzmann]

I updated the example to demo how the output can be informed about format changes in vban

[trackme518]

ok, thanks a lot i will test it in the evening!

[trackme518]

Hi, I just tested it and I can confirm it works - changing sample Rate is applied on the MCU side successfully. Thanks again!