Encoding 32bit I2S stream?

[jeffiel]

My I2S mems mic encodes at 18 bits and this requires at least 24bit storage, but only 32bit seems to be working. Which is fine. But I’m curious how to encode the raw stream. All the voice encoders seem to use 16bit streams as input. Can I bit shift the 18 bits into 16 losing a little precision? Any other ideas?

[pschatzmann]

Did you try to request 16 bits from the microphone ?

I was usually getting this to work with 1 channel with proper data and one channel with noise that can be just replaced with a copy of the proper data with the help of a ConverterFillLeftAndRight<int16_t>.

You can also convert 32 to 16 bits by wrapping the input stream into a NumberFormatConverterStream

[jeffiel]

Thanks Phil.

Channels are working ok. Just trying to encode this into something more efficient than raw PCM.

NumberFormatConverterStream doesn't seem to be working. Here's my chain:

I2S -> converter -> gain control -> encoder -> timedClient-> network client

WiFiClientSecure client;

/* MIC CHAIN */
I2SStream mic_in(I2S_MUTE_PIN);
WifiMeasuringStream outTimed(client);
EncodedAudioStream encoder(&outTimed, new RAWEncoder());          // encode and write to network
VolumeStream mic_gain(encoder);
NumberFormatConverterStream mic_converter(mic_gain);
StreamCopy mic_copier(mic_converter, mic_in);                           // copies sound into i2s

With the converter, I'm getting:

[E] AudioCopy.h : 250 - write to target has failed! (2 bytes)

I put some prints in the converter class to debug:

SAMPLES: 256
Result: 512
SAMPLES: 128
Result: 256
SAMPLES: 64
Result: 128
SAMPLES: 32
Result: 64
SAMPLES: 16
Result: 32
SAMPLES: 8
Result: 16
SAMPLES: 4
Result: 8
SAMPLES: 2
Result: 4
SAMPLES: 1
Result: 2
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
SAMPLES: 0
Result: 0
[E] AudioCopy.h : 250 - write to target has failed! (2 bytes)

Here's the relevant parts from my setup function:

const uint16_t sample_rate=8000;
const uint16_t channels = 1;
const uint16_t bits_per_sample = 16;                                    // or try with 24 or 32
const uint16_t bits_per_sample_mic = 32;                                    // or try with 24 or 32

  // configure MIC INPUT
  auto config_in = mic_in.defaultConfig(RX_MODE);
  config_in.sample_rate = sample_rate; 
  config_in.bits_per_sample = bits_per_sample_mic; 
  config_in.i2s_format = I2S_STD_FORMAT;
  config_in.channels = channels;
  config_in.is_master = true;
  config_in.port_no = 0;
  config_in.pin_bck = 14;
  config_in.pin_ws = 32;
  config_in.pin_data = 15;
  mic_in.begin(config_in);

  // setup converter stream to convert 32bit mic stream to 16bit for the rest of the chain 
  mic_converter.begin(bits_per_sample_mic, bits_per_sample);

  auto gain_config = mic_gain.defaultConfig();
  gain_config.sample_rate = sample_rate;
  gain_config.channels = channels;
  gain_config.bits_per_sample = bits_per_sample;
  gain_config.allow_boost = true;
  gain_config.volume = MIC_GAIN;
  mic_gain.begin(gain_config);

  // setup encoders/decoders
  auto encoderConfig = encoder.defaultConfig();
  encoderConfig.channels = channels;
  encoderConfig.sample_rate = sample_rate;
  encoderConfig.bits_per_sample = bits_per_sample;

  // setup to begin
  encoder.begin(encoderConfig);
  decoder.begin(encoderConfig);

  outTimed.begin();
  inTimed.begin();

Interestingly from the data sheet on this microphone:

The Data Format is I2S, 24 bit, 2’s compliment, MSB first. The Data Precision is 18 bits, unused bits are zeros.

https://www.adafruit.com/product/3421

When I try 24 bit, I get nothing. Only 32bit seems to produce any signal. So I have 18bit audio coming from the interface, stored in 32 bit, but I need 16.

[pschatzmann]

I committed some corrections, hoping that this might help. However I strongly recommend to start with the I2S to Csv sketch and add additional layers - step by step - only after you confirmed that you get the expected output. Otherwise it will get too difficult to locate the source of your issue.

[jeffiel]

Thanks Phil - will do.

I was thinking - if i've got an 18bit stream stored in 32bits, then the amplitude will be very small. I started playing with the ScalerConverter class to multiply the values up by 4096 before converting them back to 16 bit. Do you have any examples of using a converter class? I couldn't find any.

[pschatzmann]

Since you already have a VolumeStream in your chain you can configure it to allow amplification values > 1.0 (if the volume is too low) and handle it there, w/o the need of adding another concept.

[miroslavpetrov]

@jeffiel I am playing now with the same mic - SPH0645 and i cannot get it to work below 32bits. So far my only prove that its working at 32bits is the streams-i2s-serial example where I can see that the data read from the mic changes with the sound I make. I also cannot figure out how I can encode the 32bit mic sound to wav. Did you find a solution?

[pschatzmann]

Can you share the I2S settings that is working for the SPH0645 ? I was wondering if the SPH0645 is working with config.i2s_format = I2S_STD_FORMAT; or if it requires config.i2s_format = I2S_LSB_FORMAT; like the Adafruit DACs

[pschatzmann]

32 bit wav files: When I tried last, this was working w/o any issues: just make sure that you give the correct configuration to the webserver that has bits_per_sample set to 32

[miroslavpetrov]

Yes, i managed to encode a 32bit wav, but the problem with this microphone persists. As @jeffiel mentioned it outputs 24bit word but 18bit data and the LSBs are zeros. I cannot get any data from the mic with 24bit config, only 32bit seems to work but the serial output is showing around -120 000 000 at ambient noise. So I guess I should try to shift the bits.

[pschatzmann]

My gut feeling is that you selected the wrong i2s_format, if you still get noise with 32 bits

[miroslavpetrov]

I came up with this code that does not use audio-tools but the microphone works as expected:

/**
 * ESP32 I2S UDP Streamer
 *
 * This is influenced by maspetsberger's NoiseLevel at 
 * https://github.com/maspetsberger/esp32-i2s-mems/blob/master/examples/NoiseLevel/NoiseLevel.ino
 *
 * @author GrahamM
 */

#include <driver/i2s.h>
#include <soc/i2s_reg.h>
#include "WiFi.h"
#include "AsyncUDP.h"


// WiFi network name and password:
const char* ssidName = "xxx";
const char* ssidPswd = "xxx";

// UDP Destination
IPAddress udpAddress(192, 168, 6 , 190);
const int udpPort = 3333;
// Connection state
boolean connected = false;

//The udp library class
AsyncUDP udp;

const i2s_port_t I2S_PORT = I2S_NUM_0;
const int BLOCK_SIZE = 128;

void setup() {
    Serial.begin(115200);
    Serial.println(" * Configuring WiFi");
    setupWiFi();
    Serial.println(" * Configuring I2S");
    I2SSetup();
    
}

void setupWiFi() {
    WiFi.mode(WIFI_STA);
    WiFi.begin(ssidName, ssidPswd);
    if (WiFi.waitForConnectResult() != WL_CONNECTED) {
        Serial.println("WiFi Failed");
        while (1) {
            delay(1000);
        }
    }
}

void I2SSetup() {
    esp_err_t err;

    // The I2S config as per the example
    const i2s_config_t i2s_config = {
        .mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX), // We drive clk but only receive
                               // 112000 and below seem to be cleaner
        .sample_rate = 44100, //16000, 32000, 44100, 48000, 96000, 112000, 128000, 144000, 160000, 176000, 192000
        .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT, // WS signal must be BCLK/64 - this is how we manage it
        .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,  // Left by default
        .communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,     // Interrupt level 1
        .dma_buf_count = 4,                           // number of buffers
        .dma_buf_len = BLOCK_SIZE,                    // samples per buffer
    };

    // The pin config as per the setup
    const i2s_pin_config_t pin_config = {
        .bck_io_num = 14,   // Bit Clk
        .ws_io_num = 15,    // LR Clk
        .data_out_num = -1, // Data out
        .data_in_num = 32   // Data in
    };

    // Configuring the I2S driver and pins.
    err = i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL);
    if (err != ESP_OK) {
        Serial.printf(" ! Failed installing driver: %d\n", err);
        while (true);
    }

    // Alterations for SPH0645 to ensure we receive MSB correctly.
    REG_SET_BIT(I2S_TIMING_REG(I2S_PORT), BIT(9));   // I2S_RX_SD_IN_DELAY
    REG_SET_BIT(I2S_CONF_REG(I2S_PORT), I2S_RX_MSB_SHIFT);  // Phillips I2S - WS changes a cycle earlier

    err = i2s_set_pin(I2S_PORT, &pin_config);
    if (err != ESP_OK) {
        Serial.printf(" ! Failed setting pin: %d\n", err);
        while (true);
    }
    Serial.println(" + I2S driver installed.");
}

int32_t buffer[512];    // Effectively two 1024 byte buffers
volatile uint16_t rPtr = 0; // Yes, I should be using a pointer.

void mic_record()
{
    size_t bytes_read; // This will hold the number of bytes read
    i2s_read(I2S_PORT,
             buffer + rPtr,
             BLOCK_SIZE, 
             &bytes_read, 
             portMAX_DELAY);

    rPtr = rPtr + (bytes_read / sizeof(int32_t));
    // Wrap this when we get to the end of the buffer
    if (rPtr > (sizeof(buffer) / sizeof(int32_t)) - 1) rPtr = 0;
}

void loop() {
    static uint8_t state = 0;
    mic_record();

    if (!connected) {
        if (udp.connect(udpAddress, udpPort)) {
            connected = true;
            Serial.println(" * Connected to host");
        }
    }
    else {
        switch (state) {
            case 0: // wait for index to pass halfway
                if (rPtr > 255) {
                    state = 1;
                }
                break;
            case 1: // send the first half of the buffer
                state = 2;
                udp.write( (uint8_t *)buffer, 1024);
                break;
            case 2: // wait for index to wrap
                if (rPtr < 255) {
                    state = 3;
                }
                break;
            case 3: // send second half of the buffer
                state = 0;
                udp.write((uint8_t*)buffer+1024, 1024);
                break;
        }
    }   
}

Do you thnik I can implement this configuration in your library?

    // The I2S config as per the example
    const i2s_config_t i2s_config = {
        .mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX), // We drive clk but only receive
                               // 112000 and below seem to be cleaner
        .sample_rate = 44100, //16000, 32000, 44100, 48000, 96000, 112000, 128000, 144000, 160000, 176000, 192000
        .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT, // WS signal must be BCLK/64 - this is how we manage it
        .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,  // Left by default
        .communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_I2S | I2S_COMM_FORMAT_I2S_MSB),
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,     // Interrupt level 1
        .dma_buf_count = 4,                           // number of buffers
        .dma_buf_len = BLOCK_SIZE,                    // samples per buffer
    };

    // Alterations for SPH0645 to ensure we receive MSB correctly.
    REG_SET_BIT(I2S_TIMING_REG(I2S_PORT), BIT(9));   // I2S_RX_SD_IN_DELAY
    REG_SET_BIT(I2S_CONF_REG(I2S_PORT), I2S_RX_MSB_SHIFT);  // Phillips I2S - WS changes a cycle earlier

[pschatzmann]

I do not support I2S_CHANNEL_FMT_ONLY_RIGHT, but this might not be important.

It is definitely strange that this hardware does not support any I2S standard formats properly and that you need to set some registers to make things work.

You can try to add the REG_SET_BIT lines after calling i2s.begin(), just make sure to change the configuration to 32 bits.

If you leave channels to 2 you should get the audio on one channel and silence on the other. If you change channels to 1 you will get the 2 channels mixed together.

If you would manage to achieve the same with some bit shifting operations, this would lead to a more portable solution that would work on other processors as well.

[miroslavpetrov]

I have modified the streams-i2s-serial like this:

/**
 * @file streams-i2s-serial.ino
 * @author Phil Schatzmann
 * @brief see https://github.com/pschatzmann/arduino-audio-tools/blob/main/examples/examples-stream/streams-i2s-serial/README.md
 * 
 * @author Phil Schatzmann
 * @copyright GPLv3
 */


#include "AudioTools.h"
#include <soc/i2s_reg.h>
#include <driver/i2s.h>
int channels = 2;
I2SStream i2sStream; // Access I2S as stream
CsvOutput<int32_t> csvStream(Serial, channels);
StreamCopy copier(csvStream, i2sStream); // copy i2sStream to csvStream

// Arduino Setup
void setup(void) {
    Serial.begin(115200);
    AudioLogger::instance().begin(Serial, AudioLogger::Info);
    
    auto cfg = i2sStream.defaultConfig(RX_MODE);
    cfg.i2s_format = I2S_MSB_FORMAT; // or try with I2S_LSB_FORMAT
    cfg.bits_per_sample = 32;
    cfg.channels = channels;
    cfg.sample_rate = 44100;
    cfg.is_master = true;
    
     // this module nees a master clock if the ESP32 is master
    cfg.use_apll = false;  // try with yes
    //cfg.pin_mck = 3; 
    i2sStream.begin(cfg);
    REG_SET_BIT(I2S_TIMING_REG(I2S_NUM_0), BIT(9));
    REG_SET_BIT(I2S_CONF_REG(I2S_NUM_0), I2S_RX_MSB_SHIFT);

    // make sure that we have the correct channels set up
    csvStream.begin(cfg);

}

// Arduino loop - copy data
void loop() {
    copier.copy();
}

and this is the serial plotter output while playing music near the mic:

It doesn't seem ok. With working audio board the wave is around zero on the y axis.

[pschatzmann]

Yes, I would expect it to oscillate around 0 as well. But the shape looks promising: what happens if you record a sine wave e.g. with https://onlinetonegenerator.com/ ? Do you get a sine wave as output. In this case you could just shift the values up...

[pschatzmann]

Oh, cool. How does it look w/o the REG settings ?
You could try to use the ConverterAutoCenter or the ConverterScaler class to shift the data up.

Usually these kind of things are only needed for analog microphones. I have never seen any digital one that would do anything like this...

[miroslavpetrov]

This is without reg settings. I will try ConverterAutoCenter and see what happens.

[jeffiel]

Hey - just saw this thread again. Here's what I ended up doing to convert 32bit down to 16 bit.

I wrote a stream converter and put it in my chain:

class DownvertingStream : public AudioStreamX {

    public:

        DownvertingStream(Print &print) : AudioStreamX() {
            p_print = &print;
        }

        virtual int availableForWrite() { return p_print->availableForWrite(); }

        virtual size_t write(const uint8_t *buffer, size_t size) {

          // CONVERT FROM 32bit to 16bit
          int32_t sample_in;
          int16_t sample_out;
          uint8_t *buffPtr = (uint8_t*)buffer;
          size_t result_written = 0;

          // how many samples do we have here?
          int samples = size / sizeof(int32_t);

          int16_t outBuffer[samples];

          // if we got an invalid sample, ignore it?  (This may be wrong but let's try it for now)
          if(samples < 1)
            result_written = size;

          for(int i=0; i<samples; i++) {
            sample_in = *(int32_t*)buffPtr;
            outBuffer[i] = _convertFrom32To16(sample_in);
            buffPtr += sizeof(int32_t);
          }

          // note that this function is expected to return the bytes written OF THE INPUT BYTES, not output bytes
          // so multiply by two
          return 2 * p_print->write((uint8_t*)&outBuffer, 2*samples);


        }

        size_t readBytes(uint8_t *data, size_t length) override {
          return 0;
        }

        /// Returns the available bytes in the buffer: to be avoided
        virtual int available() override {
          return 0;
        }

    protected:
        Stream *p_stream;
        Print *p_print;
        int _channels = 1;

        int16_t _convertFrom32To16(int32_t value)  {
            return ((float)(value)) / INT32_MAX * INT16_MAX;
        }

};

[pschatzmann]

Did you read the Wiki: https://github.com/pschatzmann/arduino-audio-tools/wiki/Converting-the-Data-Format

[jeffiel]

I hadn't seen that but will take a look. Mine is working, but super specific to my need. Makes sense that you'd have a generic converter.

[pschatzmann]

Using google I found that the SPH0645 seems to have a DC-Offset Problem.