[REQ] AI audio libraries "port"

[MarcoRavich]

Hi there,
many interesting projects are growing around "AI" audio, it would be great to have an OpenVINO version too:

• @asteroid-team's torch-audiomentations - Audio data augmentation in PyTorch, inspired by audiomentations;
• @iver56's audiomentations - A Python library for audio data augmentation, inspired by albumentations:

  Transforms:

  • AddBackgroundNoise: Mixes in another sound to add background noise
  • AddColorNoise: Adds noise with specific color
  • AddGaussianNoise: Adds gaussian noise to the audio samples
  • AddGaussianSNR: Injects gaussian noise using a randomly chosen signal-to-noise ratio
  • AddShortNoises: Mixes in various short noise sounds
  • AdjustDuration: Trims or pads the audio to fit a target duration
  • AirAbsorption: Applies frequency-dependent attenuation simulating air absorption
  • Aliasing: Produces aliasing artifacts by downsampling without low-pass filtering and then upsampling
  • ApplyImpulseResponse: Convolves the audio with a randomly chosen impulse response
  • BandPassFilter: Applies band-pass filtering within randomized parameters
  • BandStopFilter: Applies band-stop (notch) filtering within randomized parameters
  • BitCrush: Applies bit reduction without dithering
  • Clip: Clips audio samples to specified minimum and maximum values
  • ClippingDistortion: Distorts the signal by clipping a random percentage of samples
  • Gain: Multiplies the audio by a random gain factor
  • GainTransition: Gradually changes the gain over a random time span
  • HighPassFilter: Applies high-pass filtering within randomized parameters
  • HighShelfFilter: Applies a high shelf filter with randomized parameters
  • Lambda: Applies a user-defined transform
  • Limiter: Applies dynamic range compression limiting the audio signal
  • LoudnessNormalization: Applies gain to match a target loudness
  • LowPassFilter: Applies low-pass filtering within randomized parameters
  • LowShelfFilter: Applies a low shelf filter with randomized parameters
  • Mp3Compression: Compresses the audio to lower the quality
  • Normalize: Applies gain so that the highest signal level becomes 0 dBFS
  • Padding: Replaces a random part of the beginning or end with padding
  • PeakingFilter: Applies a peaking filter with randomized parameters
  • PitchShift: Shifts the pitch up or down without changing the tempo
  • PolarityInversion: Flips the audio samples upside down, reversing their polarity
  • RepeatPart: Repeats a subsection of the audio a number of times
  • Resample: Resamples the signal to a randomly chosen sampling rate
  • Reverse: Reverses the audio along its time axis
  • RoomSimulator: Simulates the effect of a room on an audio source
  • SevenBandParametricEQ: Adjusts the volume of 7 frequency bands
  • Shift: Shifts the samples forwards or backwards
  • SpecChannelShuffle: Shuffles channels in the spectrogram
  • SpecFrequencyMask: Applies a frequency mask to the spectrogram
  • TanhDistortion: Applies tanh distortion to distort the signal
  • TimeMask: Makes a random part of the audio silent
  • TimeStretch: Changes the speed without changing the pitch
  • Trim: Trims leading and trailing silence from the audio
• @FedericoDiMarzo's torch_utilities - a collection of tools and utilities for working with PyTorch in the audio domain
• @ritik8801's Audio Data Analysis Using Deep Learning with Python - audio data analysis and extracted necessary features from a sound/audio file

More to come.

[MarcoRavich]

• @ckx2001's AudioCraft:
  • MusicGen: A state-of-the-art controllable text-to-music model.
  • AudioGen: A state-of-the-art text-to-sound model.
  • EnCodec: A state-of-the-art high fidelity neural audio codec.
  • Multi Band Diffusion: An EnCodec compatible decoder using diffusion.
  • MAGNeT: A state-of-the-art non-autoregressive model for text-to-music and text-to-sound.
  • AudioSeal: A state-of-the-art audio watermarking.

[RyanMetcalfeInt8]

Just to point out, the Music Generation feature in these plugjns is currently based on this MusicGen model (and uses EnCodec also for conversions).

[RyanMetcalfeInt8]

btw, it's not really clear what the difference is between @ckx2001's repo, and the official AudioCraft repo: https://github.com/facebookresearch/audiocraft

[MarcoRavich]

btw, it's not really clear what the difference is between @ckx2001's repo, and the official AudioCraft repo: https://github.com/facebookresearch/audiocraft

I honestly don't know since it don't reports as a fork... bus seems pretty fresh committed so maybe a mod ?

While @ckx2001 cuold certainly explain better, the @facebookresearch's one is the obvious way to go.

[MarcoRavich]

Forgot to mention @HENDRIX-ZT2's Python Audio Restoration Suite which implements very useful (even if "non-AI"):

• Wow & Flutter Removal
• Speed matching to hum frequency
• EQ matching with differential EQ
• Sub-sample accurate Spectral Temporal Alignment
• Automatic Dropout Restoration
• Spectral Expander / Decompressor

Hope that inspires !

[MarcoRavich]

Bump.

Just discovered the cool @mikeoliphant's NeuralAudio library, a C++ library designed to make it easy to use neural network machine learning models (ie: guitar amplifier captures/profiles) in real-time audio applications.