Differentiable audio processors w/ the Faust programming language

[DBraun]

I made a transpiler from Faust code to JAX. Faust is a powerful language for sound synthesis, so connecting it to JAX while supporting differentiability and learnable parameters is a huge research opportunity.

You can explore Faust to JAX in two Colabs. The first Colab uses gradient descent to optimize the cutoff frequency of a low-pass filter. It also shows how to make a differentiable synthesizer like the ones used in Google's DDSP and ByteDance's Differentiable Wavetable Synthesis. DDSP provided many differentiable modules, but now we can convert the amazing functions in the Faust Libraries to differentiable JAX code. These libraries cover physical modeling, ambisonics, filters, reverbs, envelopes, compressors, wave digital filters, and much more. Further research could use these processors for adaptive filters, audio mastering/FX, and much more. These differentiable operators can also be integrated with pre-existing JAX model architectures.

The second Faust to JAX Colab uses QDax for quality-diversity learning. In a toy problem, we have a 3-section parametric equalizer with 10 total parameters. We achieve near-perfect fitness in a short amount of time.

If you're in need of business-related use-case involving this project, please let me know 📫 braun$AT$ccrma.stanford.edu