Dynamic Memory Allocation (making ESP32 external RAM work)

[DatanoiseTV]

I'm running Vult generated code on my ESP32, but internal RAM is limited to 520K (minus WiFi Stack etc), so in order to use more, I need to use dynamic memory allocation to use the external PSRAM.

This is how I do it in non-vult DSP code.

	buffer = (float*) ps_malloc((MAX_DELAY_LEN + 1) * sizeof(float));
	memset(buffer, 0, (MAX_DELAY_LEN + 1) * sizeof(float));

ps_malloc allocates memory on external PSRAM.

Do I need to write a custom generator?

[modlfo]

There are two ways that you can do it depending on your needs.

In situations when I have to allocate or reserve a big chunk of memory, for example when creating a delay, I usually access it with external functions. For example:

// Test.vult
// Declare two external function to read and write the data
external writeToBuffer(index:int, value:real) "writeToBuffer";
external readFromBuffer(index:int) : value "readFromBuffer";

// main.cpp
float* buffer;

#define DELAY_SIZE 1024

void initializeBuffer() {
   buffer = (float*)malloc(sizeof(float) * DELAY_SIZE); // use your custom allocator
}

extern "C" {
   void writeToBuffer(int index, float value) {
      buffer[index] = value;
   }

   float readFromBuffer(int index){
      return buffer[index];
   }
}

That way your Vult code can access the buffer.

The second option would be if you want to put all the memory used by your Vult code in the external memory. For example:

// Test.vult

// A function in Vult that requires memory
fun process(x:real) {
   mem count = count + 1;
   return count;
}

// main.cpp
#include "test.h"

int main(void) {
   // Allocate the memory used by the Vult code using your custom allocator
   Test_process_type* processor = (Test_process_type*) malloc(sizeof(Test_process_type));
   // then pass it to the Vult functions as follows.
   Test__ctx_type_2_init(*processor);

   return 0;
}

[DatanoiseTV]

@modlfo Thank you! Maybe it would make sense to make a template? I've started one for https://github.com/garygru/yummyDSP

yummyDSP.ml

(*
   The MIT License (MIT)

   Copyright (c) 2014 Leonardo Laguna Ruiz

   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
   in the Software without restriction, including without limitation the rights
   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
   copies of the Software, and to permit persons to whom the Software is
   furnished to do so, subject to the following conditions:

   The above copyright notice and this permission notice shall be included in
   all copies or substantial portions of the Software.

   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
   THE SOFTWARE.
*)

(** Template for the YummyDSP Audio library, based on the Teensy Template *)
open Config

let inputsArray n_inputs =
   if n_inputs > 0 then
      {pla|audio_block_t *inputQueueArray[<#n_inputs#i>];|pla}, {pla|inputQueueArray|pla}
   else
      Pla.unit, {pla|NULL|pla}


let tables (params : params) (code : Pla.t) : Pla.t =
   let file = String.uppercase params.output in
   {pla|
   /* Code automatically generated by Vult https://github.com/modlfo/vult */
   #ifndef <#file#s>_TABLES_H
   #define <#file#s>_TABLES_H


   <#code#>

   #endif // <#file#s>_TABLES_H
   |pla}


(** Header function *)
let header (params : params) (code : Pla.t) : Pla.t =
   let file = String.uppercase params.output in
   let tables = params.output in
   let output = params.output in
   let module_name = params.module_name in
   let n_inputs = Config.countInputsNoCtx params.config.process_inputs in
   let input_queue_delc, input_queue_name = inputsArray n_inputs in
   {pla|
#ifndef <#file#s>_H
#define <#file#s>_H

#include <stdint.h>
#include <math.h>
#include "vultin.h"
#include "<#tables#s>.tables.h"

#include "Arduino.h"
#include "Nodes/AudioNode.h"

<#code#>

class <#output#s> : public AudioNode
{
public:

  void begin(int fs, int channelCount) {
    <#module_name#s>_process_init(data);
    <#module_name#s>_default(data);
  }

  // Handles note on events
  void noteOn(int note, int velocity, int channel){
    // If the velocity is larger than zero, means that is turning on
    if(velocity) <#module_name#s>_noteOn(data, note, velocity, channel);
    else         <#module_name#s>_noteOff(data, note, channel);
  }

  // Handles note off events
  void noteOff(int note, int velocity, int channel) {
    <#module_name#s>_noteOff(data, note, channel);

  }

  // Handles control change events
  void controlChange(int control, int value, int channel) {
    <#module_name#s>_controlChange(data, control, value, channel);
  }

  float processSample(float sample, int channel);

private:
  int fs; // sample rate

  <#module_name#s>_process_type data;

};

#endif // <#file#s>_H
|pla}


let rec allocateBlocks (block : int) (inputs : int) (outputs : int) =
   match inputs, outputs with
   | 0, 0 -> []
   | 0, _ ->
      let t = {pla|audio_block_t * block<#block#i> = allocate(); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) inputs (outputs - 1)
   | _, 0 ->
      let t = {pla|audio_block_t * block<#block#i> = receiveReadOnly(<#block#i>); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) (inputs - 1) outputs
   | _, _ ->
      let t = {pla|audio_block_t * block<#block#i> = receiveWritable(<#block#i>); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) (inputs - 1) (outputs - 1)


let transmitBlocks (outputs : int) =
   CCList.init outputs (fun i -> {pla|transmit(block<#i#i>, <#i#i>);|pla}) |> Pla.join_sep Pla.newline


let releaseBlocks (blocks : int) =
   CCList.init blocks (fun i -> {pla|release(block<#i#i>);|pla}) |> Pla.join_sep Pla.newline


let castInput params typ i acc =
   match typ with
   | IReal _ when params.real = "fixed" -> i + 1, {pla|fix16_t in<#i#i> = short_to_fix(block<#i#i>->data[i]);|pla} :: acc
   | IReal _ -> i + 1, {pla|float in<#i#i> = short_to_float(block<#i#i>->data[i]);|pla} :: acc
   | IBool _ -> i + 1, {pla|uint8_t in<#i#i> = block<#i#i>->data[i] != 0;|pla} :: acc
   | IInt _ -> i + 1, {pla|int in<#i#i> = block<#i#i>->data[i];|pla} :: acc
   | IContext -> i, acc


let castOutput params typ value =
   match typ with
   | OReal when params.real = "fixed" -> {pla|fix_to_short(<#value#>)|pla}
   | OReal -> {pla|<#value#>|pla}
   | OFix16 -> {pla|fix_to_short(<#value#>)|pla}
   | OBool -> {pla|<#value#>|pla}
   | OInt -> {pla|<#value#>|pla}


let declareInputs params =
   List.fold_left (fun (i, acc) a -> castInput params a i acc) (0, []) params.config.process_inputs
   |> snd
   |> Pla.join_sep Pla.newline
   |> Pla.indent
   |> Pla.indent


let declReturn params =
   let module_name = params.module_name in
   match params.config.process_outputs with
   | [] -> Pla.unit, Pla.unit
   | [ o ] ->
      let current_typ = Replacements.getType params.repl (Config.outputTypeString o) in
      let decl = {pla|<#current_typ#s> out;|pla} in
      let value = castOutput params o (Pla.string "out") in
      let copy = {pla|block0->data[i] = <#value#>; |pla} in
      decl, copy
   | o ->
      let copy =
         List.mapi
            (fun i o ->
                let value = castOutput params o {pla|<#module_name#s>_process_ret_<#i#i>(data)|pla} in
                {pla|block<#i#i>->data[i] = <#value#>; |pla})
            o
         |> Pla.join_sep_all Pla.newline
         |> Pla.indent
      in
      Pla.unit, copy


(** Implementation function *)
let implementation (params : params) (code : Pla.t) : Pla.t =
   let output = params.output in
   let module_name = params.module_name in
   let n_inputs = Config.countInputsNoCtx params.config.process_inputs in
   let n_outputs = Config.countOutputs params.config.process_outputs in
   let allocate_blocks = allocateBlocks 0 n_inputs n_outputs |> Pla.join_sep Pla.newline in
   let transmit_blocks = transmitBlocks n_outputs in
   let release_blocks = releaseBlocks (max n_inputs n_outputs) in
   let inputs = declareInputs params in
   let decl_out, copy_out = declReturn params in
   {pla|
#include "<#output#s>.h"

<#code#>

float <#output#s>::processSample(float sample, int channel)
{
   return <#module_name#s>_process(data, sample);
}

|pla}


let get (params : params) (header_code : Pla.t) (impl_code : Pla.t) (tables_code : Pla.t) : (Pla.t * FileKind.t) list =
   [ header params header_code, FileKind.ExtOnly "h"
   ; tables params tables_code, FileKind.ExtOnly "tables.h"
   ; implementation params impl_code, FileKind.ExtOnly "cpp"
   ]

[DatanoiseTV]

Also, how would this adaptation look for examples/effects/short_delay.vult?

[DatanoiseTV]

Ok, got it working and my template is working. How do i add process arguments to my C++ process wrapper?

(*
   The MIT License (MIT)

   Copyright (c) 2014 Leonardo Laguna Ruiz

   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
   in the Software without restriction, including without limitation the rights
   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
   copies of the Software, and to permit persons to whom the Software is
   furnished to do so, subject to the following conditions:

   The above copyright notice and this permission notice shall be included in
   all copies or substantial portions of the Software.

   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
   THE SOFTWARE.
*)

(** Template for the YummyDSP Audio library, based on the Teensy Template *)
open Config

let inputsArray n_inputs =
   if n_inputs > 0 then
      {pla|audio_block_t *inputQueueArray[<#n_inputs#i>];|pla}, {pla|inputQueueArray|pla}
   else
      Pla.unit, {pla|NULL|pla}


let tables (params : params) (code : Pla.t) : Pla.t =
   let file = String.uppercase params.output in
   {pla|
   /* Code automatically generated by Vult https://github.com/modlfo/vult */
   #ifndef <#file#s>_TABLES_H
   #define <#file#s>_TABLES_H


   <#code#>

   #endif // <#file#s>_TABLES_H
   |pla}


(** Header function *)
let header (params : params) (code : Pla.t) : Pla.t =
   let file = String.uppercase params.output in
   let tables = params.output in
   let output = params.output in
   let module_name = params.module_name in
   let n_inputs = Config.countInputsNoCtx params.config.process_inputs in
   let input_queue_delc, input_queue_name = inputsArray n_inputs in
   {pla|
#ifndef <#file#s>_H
#define <#file#s>_H

#include <stdint.h>
#include <math.h>
#include "vultin.h"
#include "<#tables#s>.tables.h"

#include "Arduino.h"
#include "Nodes/AudioNode.h"

<#code#>

class <#output#s> : public AudioNode
{
public:

  void begin(int fs, int channelCount) {
   data = (<#module_name#s>_process_type*) ps_malloc(sizeof(<#module_name#s>_process_type));
    <#module_name#s>_process_init(*data);
    <#module_name#s>_default(*data);
  }

  // Handles note on events
  void noteOn(int note, int velocity, int channel){
    // If the velocity is larger than zero, means that is turning on
    if(velocity) <#module_name#s>_noteOn(*data, note, velocity, channel);
    else         <#module_name#s>_noteOff(*data, note, channel);
  }

  // Handles note off events
  void noteOff(int note, int velocity, int channel) {
    <#module_name#s>_noteOff(*data, note, channel);

  }

  // Handles control change events
  void controlChange(int control, int value, int channel) {
    <#module_name#s>_controlChange(*data, control, value, channel);
  }

  float processSample(float sample, int channel);

private:
  int fs; // sample rate

  <#module_name#s>_process_type *data;

};

#endif // <#file#s>_H
|pla}


let rec allocateBlocks (block : int) (inputs : int) (outputs : int) =
   match inputs, outputs with
   | 0, 0 -> []
   | 0, _ ->
      let t = {pla|audio_block_t * block<#block#i> = allocate(); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) inputs (outputs - 1)
   | _, 0 ->
      let t = {pla|audio_block_t * block<#block#i> = receiveReadOnly(<#block#i>); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) (inputs - 1) outputs
   | _, _ ->
      let t = {pla|audio_block_t * block<#block#i> = receiveWritable(<#block#i>); if(!block<#block#i>) return;|pla} in
      t :: allocateBlocks (block + 1) (inputs - 1) (outputs - 1)


let transmitBlocks (outputs : int) =
   CCList.init outputs (fun i -> {pla|transmit(block<#i#i>, <#i#i>);|pla}) |> Pla.join_sep Pla.newline


let releaseBlocks (blocks : int) =
   CCList.init blocks (fun i -> {pla|release(block<#i#i>);|pla}) |> Pla.join_sep Pla.newline


let castInput params typ i acc =
   match typ with
   | IReal _ when params.real = "fixed" -> i + 1, {pla|fix16_t in<#i#i> = short_to_fix(block<#i#i>->data[i]);|pla} :: acc
   | IReal _ -> i + 1, {pla|float in<#i#i> = short_to_float(block<#i#i>->data[i]);|pla} :: acc
   | IBool _ -> i + 1, {pla|uint8_t in<#i#i> = block<#i#i>->data[i] != 0;|pla} :: acc
   | IInt _ -> i + 1, {pla|int in<#i#i> = block<#i#i>->data[i];|pla} :: acc
   | IContext -> i, acc


let castOutput params typ value =
   match typ with
   | OReal when params.real = "fixed" -> {pla|fix_to_short(<#value#>)|pla}
   | OReal -> {pla|<#value#>|pla}
   | OFix16 -> {pla|fix_to_short(<#value#>)|pla}
   | OBool -> {pla|<#value#>|pla}
   | OInt -> {pla|<#value#>|pla}


let declareInputs params =
   List.fold_left (fun (i, acc) a -> castInput params a i acc) (0, []) params.config.process_inputs
   |> snd
   |> Pla.join_sep Pla.newline
   |> Pla.indent
   |> Pla.indent


let declReturn params =
   let module_name = params.module_name in
   match params.config.process_outputs with
   | [] -> Pla.unit, Pla.unit
   | [ o ] ->
      let current_typ = Replacements.getType params.repl (Config.outputTypeString o) in
      let decl = {pla|<#current_typ#s> out;|pla} in
      let value = castOutput params o (Pla.string "out") in
      let copy = {pla|block0->data[i] = <#value#>; |pla} in
      decl, copy
   | o ->
      let copy =
         List.mapi
            (fun i o ->
                let value = castOutput params o {pla|<#module_name#s>_process_ret_<#i#i>(data)|pla} in
                {pla|block<#i#i>->data[i] = <#value#>; |pla})
            o
         |> Pla.join_sep_all Pla.newline
         |> Pla.indent
      in
      Pla.unit, copy


(** Implementation function *)
let implementation (params : params) (code : Pla.t) : Pla.t =
   let output = params.output in
   let module_name = params.module_name in
   let n_inputs = Config.countInputsNoCtx params.config.process_inputs in
   let n_outputs = Config.countOutputs params.config.process_outputs in
   let allocate_blocks = allocateBlocks 0 n_inputs n_outputs |> Pla.join_sep Pla.newline in
   let transmit_blocks = transmitBlocks n_outputs in
   let release_blocks = releaseBlocks (max n_inputs n_outputs) in
   let inputs = declareInputs params in
   let decl_out, copy_out = declReturn params in
   {pla|
#include "<#output#s>.h"

<#code#>

float <#output#s>::processSample(float sample, int channel)
{
   return <#module_name#s>_process(*data, sample);
}

|pla}


let get (params : params) (header_code : Pla.t) (impl_code : Pla.t) (tables_code : Pla.t) : (Pla.t * FileKind.t) list =
   [ header params header_code, FileKind.ExtOnly "h"
   ; tables params tables_code, FileKind.ExtOnly "tables.h"
   ; implementation params impl_code, FileKind.ExtOnly "cpp"
   ]

[DatanoiseTV]

@modlfo I am thinking of following additions / todo.

• Benchmark floating point / fixed point on ESP32
• Add class prefix to generated functions (to be closer to C++)
• Add optimized routines from Espressif ESP DSP
• Dot-product - Calculates dot-product of two vectors
• FFT - Fast Fourier Transform functionality
• DCT - Discrete Cosine Transform functionality
• IIR - IIR filter functionality
• FIR - FIR filter functionality
• Math - Basic vector operations
• Conv - Convolution/correlation functionality
• Support - Support functions
• Window functions - FFT window generation functions

[modlfo]

Some of the things that you mention could be achieved by creating a Vult library (a file) that declares the functions and makes it available to your code. One possible issue could be the strictness in Vult Language when working with arrays.

Add class prefix to generated functions (to be closer to C++)

What do you mean here? I recently added a feature to prefix all the generated code. Maybe that's what you mean.

Have you tested the template? If it works, I can integrate it.

[DatanoiseTV]

@modlfo Regarding the prefix, I get the following if generating code with:
./vultc.js -template yummydsp -ccode -real fixed -i examples/util -i examples/osc -o oscNode examples/effects/short_delay.vult

.....
void Short_delay_controlChange(Short_delay__ctx_type_12 &_ctx, int control, int value, int channel){
   if(control == 0){
      _ctx.time = fix_mul(0x204 /* 0.007874 */,int_to_fix(value));
   }
   if(control == 1){
      _ctx.feedback = fix_mul(0x204 /* 0.007874 */,int_to_fix(value));
   }
   if(control == 2){
      _ctx.flutter = fix_mul(0x204 /* 0.007874 */,int_to_fix(value));
   }
}

int32_t oscNode::processSample(int32_t sample, int channel)
{
   return Short_delay_process(*data, sample);
}

So Short_delay_controlChange etc. is missing the oscNode:: class prefix.
My template works so far, but it probably needs some cleanup and I am unsure about some template tags.

[modlfo]

Finally I got a ESP32. I will test how thing work here when I have some time.

[DatanoiseTV]

Hi @modlfo: Did you get it working? It is working quite nice like this on my platform.