PANAMA: PArametric Neural Amp Modeling with Active learning

This is the implementation of our parametric amp modelling framework. Our code is based on NAM. Demos can be found here.

Setup

Create a conda environment from environments/environment_gpu.yml.

Config JSONs

There are three types of config JSONs:

• Model config, which defines the model architecture as well as the loss function. Defaults ones are provided in default_config_files/models/. wavenet-mel-mrstft.json and lstm-mel-mrstft.json are recommended. The global condition size (i.e. number of amp knobs) is by default 6, but you can change it using the json keys "global_condition_size" for WaveNet and "conditioning_dim" for LSTM.
• Learning config, which provides the batch size, number of epochs and other training hyperparams. default_config_files/learning/default.json shows the default learning config. We observed that in general, a larger batch size works better with a larger dataset, and vice versa.
• Data config, which supplies the paths of audio files in a dataset. The dataset format requires supplying a fixed input signal, and wet signals with different amp settings. We provide a mini example in default_config_files/data/skeleton.json with 2 training points and 1 validation point. The json key "ny" corresponds to the length of each audio clip in a batch. The training input signal from NAM is provided in training_data/input.wav for convenience.

You can always use --help to see all commandline options!

Inference

Inference can be done with inference_w_ckpt.py. We provide a demo checkpoint demo_ckpt.ckpt.

Usage example:

python3 inference_w_ckpt.py --ckpt-path "demo_ckpt.ckpt"  --input-path "my_input.wav" --g-vector 0.5 0.2 0.5 0.7 0.5 0.8 --output-dir .

Single Model Training

Training is done with custom_train_full.py. The results will be in a subdirectory inside of --base-outdir.

Usage example:

python3 custom_train_full.py --base-outdir "outputs" --data-config "my_dataset.json" --model-config "default_config_files/models/wavenet-mel-mrstft.json" --learning-config "default_config_files/learning/default.json"

Active Learning

active_learner_multi_gpu.py performs one round of our active learning method. We recommend making a output directory specifically for active learning. This same directory, for example my_active_learning_folder, should be supplied each round along with the current round number. Round indices start at 0. The 0-th round needs a starter dataset supplied in --starter-data-config-path; starting from round 1 this is no longer needed. 10 random datapoints should work fine for the starter. The signal x used during optimization of cross-model disagreement with regard to g is supplied via --x-path-for-g-opt. By default, the ensemble size is 4, but can be adjusted via --ensemble-size. Make sure you have a multi-GPU setup, and that the number of GPUS is at least ensemble size + 1.

When the script terminates, it logs the g-vectors of the new datapoints to be gathered. After recording and supplying the new audio clips in active_learning_inputs, rerun it with the same output directory and the new round index. Rinse and repeat.

Usage example:

python3 active_learner_multi_gpu.py --starter-data-config-path "my_starter_dataset.json" --current-round-idx 0 --x-path-for-g-opt "my_input_for_g_optimization.wav" --output-dir "my_active_learning_folder" --ensemble-size 4 --model-config-path "default_config_files/models/lstm-mel-mrstft.json" --learning-config-path "default_config_files/learning/active_learning_LSTM.json"
# Record new datapoints
python3 active_learner_multi_gpu.py --current-round-idx 1 --x-path-for-g-opt "my_input_for_g_optimization.wav" --output-dir "my_active_learning_folder" --ensemble-size 4 --model-config-path "default_config_files/models/lstm-mel-mrstft.json" --learning-config-path "default_config_files/learning/active_learning_LSTM.json"
# Record new datapoints
...

Testing

Test a model with test_model.py. This (ideally) requires a separate data config from the training one. The config should still be the same JSON format, but only the "validation" split of it will be used. The result metrics will be stored as a JSON file and visualized as a plot.

Usage example:

python3 test_model.py --data-config-path "my_test_data_config.json" --ckpt-path "model_to_be_tested.ckpt" --metrics-path "results.json" --plot-path "results.png"