Merge pull request #67 from hyperion-ml/lachesis

Lachesis

Author: jesus-villalba

.gitignore

@@ -29,14 +29,15 @@ tests/data_out/*
 
 # developer environments
 #.idea
-tools/anaconda/anaconda3.5
-tools/anaconda/anaconda3
-tools/anaconda/apex
-tools/ibm_art/adversarial-robustness-toolbox/
-tools/cudnn/cudnn-*
-tools/kaldi/kaldi
-tools/nccl/nccl_*
+tools/anaconda3.5
+tools/anaconda3
+tools/apex
+tools/adversarial-robustness-toolbox
+tools/cudnn-*
+tools/kaldi
+tools/nccl_*
 tools/path.sh
+tools/sph2pipe_*
 
 rirs_noises.zip
 RIRS_NOISES

README.md

@@ -2,47 +2,140 @@
 
 Speaker recognition toolkit
 
-## Cloning the repo
+## Installation Instructions
 
-- To clone the repo execute
+### Prerequisites
+
+    We use anaconda or miniconda, though you should be able to make it work in other python distributions
+    To start, you should create a new enviroment and install pytorch>=1.6, e.g.:
+```
+conda create --name ${your_env} python=3.8
+conda activate ${your_env}
+conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
+```
+
+### Installing Hyperion
+
+- First, clone the repo:
 ```bash
 git clone https://github.com/hyperion-ml/hyperion.git
 ```
 
-## Dependencies:
-  - Anaconda3.5:
-     - Make a link to your anaconda installation in the tools directory:
-     ```bash
-     cd hyperion/tools/anaconda
-     ln -s <your-anaconda-3.5> anaconda3.5
-     ```
-     - or follow instructions in hyperion/tools/anaconda/full_install.sh to install anaconda from scratch
-  - Kaldi speech recognition toolkit
-     - Make link to an existing kaldi installation
-     ```bash
-     cd hyperion/tools/kaldi
-     ln -s <your-kaldi> kaldi
-     ```
-     - or follow instructions in hyperion/tools/anaconda/install_kaldi.sh to install kaldi from scratch
-
-  - CuDNN: tensorflow and pytorch will need some version of cudnn
-     - Make a link to some existing cudnn version that matches the requirements of your tf or pytorch, e.g.:
-     ```bash
-     cd hyperion/tools/cudnn
-     #cudnn v7.4 for cuda 9.0 needed by pytorch 1.0 
-     ln -s /home/janto/usr/local/cudnn-9.0-v7.4 cudnn-9.0-v7.4
-     ```
-  - In the CLSP grid: you can use some preinstalled versions of anaconda and kaldi in the grid to avoid each person having its own.
-     - To create links to preinstalled kaldi, anaconda and cudnn, run:
-     ```bash
-     cd hyperion/
-     ./make_clsp_links.sh
-     ```
-     - The anaconda that you will link with this has several environments:
-        - base: numpy, h5py, pandas, etc.
-	- tensorflow1.8g_cpu: tensorflow 1.8 for cpu
-	- tensorflow1.8g_gpu: tensorflow 1.8 for gpu
-	- pytorch1.0_cuda9.0: pytorch 1.0 with cuda 9.0
+- You can choolse to install hyperion in the environment
+```bash
+cd hyperion
+pip install -e .
+```
+
+- Or add the hyperion toolkit to the PYTHONPATH envirnoment variable
+  This option will allow you to share the same environment if you are working with several hyperion branches
+  at the same time, while installing it requires to have an enviroment per branch.
+  For this, you need to install the requirements
+```bash
+cd hyperion
+pip install -r requirements.txt
+```
+Then add these lines to your `~/.bashrc` or to each script that uses hyperion
+```bash
+HYP_ROOT= #substitute this by your hyperion location
+export PYTHONPATH=${HYP_ROOT}:$PYTHONPATH
+export PATH=${HYP_ROOT}/bin:$PATH
+```
+
+## Recipes
+
+There are recipes for several tasks in the `./egs` directory.
+
+### Prerequistes to run the recipes
+
+These recipes require some extra tools (e.g. sph2pipe), which need to be installed first:
+```bash
+./install_egs_requirements.sh 
+```
+
+Most recipes do not require Kaldi, only the older ones using Kaldi x-vectors,
+so we do not install it by default. If you are going to need it install it 
+yourself. Then make a link in `./tools` to your kaldi installation
+```bash
+cd tools
+ln -s ${your_kaldi_path} kaldi
+cd -
+```
+
+Finally configure the python and environment name that you intend to use to run the recipes.
+For that run
+```bash
+./prepare_egs_paths.sh
+```
+This script will ask for the path to your anaconda installation and enviromentment name.
+It will also detect if hyperion is already installed in the environment,
+otherwise it will add hyperion to your python path.
+This will create the file
+```
+tools/path.sh
+```
+which sets all the enviroment variables required to run the recipes.
+This has been tested only on JHU computer grids, so you may need to 
+modify this file manually to adapt it to your grid.
+
+## Recipes structure
+
+The structure of the recipes is very similar to Kaldi, so if should be
+familiar for most people.
+Data preparation is also similar to Kaldi. Each dataset has
+a directory with files like
+```
+wav.scp
+utt2spk
+spk2utt
+...
+```
+
+### Running the recipes
+
+Contrary to other toolkits, the recipes do not contain a single `run.sh` script 
+to run all the steps of the recipe.
+Since some recipes have many steps and most times you don't want to run all of then
+from the beginning, we have split the recipe in several run scripts.
+The scripts have a number indicating the order in the sequence.
+For example,
+```bash
+run_001_prepare_data.sh
+run_002_compute_vad.sh
+run_010_prepare_audios_to_train_xvector.sh
+run_011_train_xvector.sh
+run_030_extract_xvectors.sh
+run_040_evaluate_plda_backend.sh
+```
+will evaluate the recipe with the default configuration.
+The default configuration is in the file `default_config.sh`
+
+We also include extra configurations, which may change 
+the hyperparamters of the recipe. For example:
+ - Acoustic features
+ - Type of the x-vector neural netwok
+ - Hyper-parameters of the models
+ - etc.
+
+Extra configs are in the `global_conf` directory of the recipe.
+Then you can run the recipe with the alternate config as:
+```bash
+run_001_prepare_data.sh --config-file global_conf/alternative_conf.sh
+run_002_compute_vad.sh --config-file global_conf/alternative_conf.sh
+run_010_prepare_audios_to_train_xvector.sh --config-file global_conf/alternative_conf.sh
+run_011_train_xvector.sh --config-file global_conf/alternative_conf.sh
+run_030_extract_xvectors.sh --config-file global_conf/alternative_conf.sh
+run_040_evaluate_plda_backend.sh --config-file global_conf/alternative_conf.sh
+```
+Note that many alternative configus share hyperparameters with the default configs.
+That means that you may not need to rerun all the steps to evaluate a new configuration.
+It mast cases you just need to re-run the steps from the neural network training to the end.
+
+
+## Citing
+
+Each recipe README.md file contains the bibtex to the works that should be cited if you 
+use that recipe in your research
 
 ## Directory structure:
  - The directory structure of the repo looks like this:
@@ -53,14 +146,10 @@ hyperion/hyperion
 hyperion/resources
 hyperion/tests
 hyperion/tools
-hyperion/tools/anaconda
-hyperion/tools/cudnn
-hyperion/tools/kaldi
-hyperion/tools/keras
 ```
  - Directories:
     - hyperion: python classes with utilities for speaker and language recognition
-    - egs: recipes for sevareal tasks: SRE18, voices, ...
+    - egs: recipes for sevaral tasks: VoxCeleb, SRE18/19/20, voices, ...
     - tools: contains external repos and tools like kaldi, python, cudnn, etc.
     - tests: unit tests for the classes in hyperion
     - resources: data files required by unittest or recipes

apps.txt

@@ -1,23 +1,14 @@
 apply-mvn-select-frames.py
-ark2hyp.py
-arkvad2nist.py
 compute-energy-vad.py
-compute-gmm-post.py
 compute-mfcc-feats.py
 copy-feats.py
-eval-2class-performance.py
 eval-cos-1vs1.py
-eval-elbo-ubm.py
-eval-linear-gbe.py
 eval-linear-gbe-up.py
+eval-linear-gbe.py
 eval-linear-svmc.py
 eval-logistic-regression.py
 eval-plda-1vs1.py
 eval-plda-nvs1.py
-eval-q-scoring-homo-gbe.py
-eval-score-norm.py
-h5vad2nist.py
-init-ubm.py
 make-babble-noise-audio-files.py
 merge-h5-files.py
 pack-audio-files.py
@@ -26,27 +17,42 @@ plot-vector-hist.py
 plot-vector-tsne.py
 preprocess-audio-files.py
 rttm-to-bin-vad.py
-scores2lre_format.py
 segments-to-bin-vad.py
 torch-adv-finetune-xvec-from-wav.py
+torch-adv-finetune-xvec.py
 torch-compute-mfcc-feats.py
 torch-eval-vae.py
+torch-eval-xvec-cosine-scoring-from-adv-test-wav-wavegan.py
+torch-eval-xvec-cosine-scoring-from-adv-test-wav.py
+torch-eval-xvec-cosine-scoring-from-art-test-wav.py
+torch-eval-xvec-cosine-scoring-from-test-wav.py
+torch-eval-xvec-cosine-scoring-from-transfer-adv-test-wav.py
+torch-eval-xvec-cosine-scoring-from-transfer-art-test-wav.py
+torch-eval-xvec-logits-from-wav.py
+torch-extract-xvectors-from-wav-with-rttm.py
 torch-extract-xvectors-from-wav.py
+torch-extract-xvectors-slidwin-from-wav.py
+torch-extract-xvectors-slidwin.py
+torch-extract-xvectors-vae-preproc.py
 torch-extract-xvectors.py
 torch-finetune-xvec-dfr-from-wav.py
 torch-finetune-xvec-dfr.py
 torch-finetune-xvec-from-wav.py
 torch-finetune-xvec.py
-torch-madry-adv-finetune-xvec.py
+torch-generate-adv-attacks-xvector-classif.py
+torch-generate-adv-attacks-xvector-verif.py
 torch-train-conformer-enc-v1-vq-dvae.py
 torch-train-conformer-enc-v1-vq-vae.py
 torch-train-dc1d-ae.py
 torch-train-dc1d-dvae.py
 torch-train-dc1d-vae.py
 torch-train-dc2d-dvae.py
 torch-train-dc2d-vae.py
+torch-train-dvae.py
 torch-train-efficientnet-xvec-from-wav.py
 torch-train-efficientnet-xvec.py
+torch-train-resnet-xvec-from-wav.py
+torch-train-resnet-xvec.py
 torch-train-resnet1d-dvae.py
 torch-train-resnet1d-vae.py
 torch-train-resnet1d-vq-dvae.py
@@ -55,8 +61,7 @@ torch-train-resnet2d-dvae.py
 torch-train-resnet2d-vae.py
 torch-train-resnet2d-vq-dvae.py
 torch-train-resnet2d-vq-vae.py
-torch-train-resnet-xvec-from-wav.py
-torch-train-resnet-xvec.py
+torch-train-spinenet-xvec-from-wav.py
 torch-train-tdnn-xvec-from-wav.py
 torch-train-tdnn-xvec.py
 torch-train-transformer-enc-v1-dvae.py
@@ -65,18 +70,19 @@ torch-train-transformer-enc-v1-vq-dvae.py
 torch-train-transformer-enc-v1-vq-vae.py
 torch-train-transformer-xvec-v1-from-wav.py
 torch-train-transformer-xvec-v1.py
-torch-train-xvector.py
-train-cw.py
+torch-train-vae.py
+torch-train-vq-dvae.py
+torch-train-vq-vae.py
+torch-train-xvec-from-wav.py
 train-cw-up.py
+train-cw.py
 train-gaussianizer.py
 train-lda.py
-train-linear-gbe.py
 train-linear-gbe-up.py
+train-linear-gbe.py
 train-linear-svmc.py
 train-logistic-regression.py
 train-mvn.py
 train-nda.py
 train-pca.py
 train-plda.py
-train-q-scoring-homo-gbe.py
-vectors2scores.py

egs/chime5_spkdet/v1/path.sh

@@ -2,62 +2,4 @@
 export HYP_ROOT=$(readlink -f `pwd -P`/../../..)
 export TOOLS_ROOT=$HYP_ROOT/tools
 
-export KALDI_ROOT=$TOOLS_ROOT/kaldi/kaldi
-export PATH=$PWD/utils/:$KALDI_ROOT/tools/openfst/bin:$KALDI_ROOT/tools/sph2pipe_v2.5:$PWD:$PATH
-[ ! -f $KALDI_ROOT/tools/config/common_path.sh ] && echo >&2 "The standard file $KALDI_ROOT/tools/config/common_path.sh is not present -> Exit!" && exit 1
-. $KALDI_ROOT/tools/config/common_path.sh
-export LC_ALL=C
-
-#Anaconda env
-CONDA_ROOT=$TOOLS_ROOT/anaconda/anaconda3
-if [ -f "$CONDA_ROOT/etc/profile.d/conda.sh" ]; then
-    #for conda version >=4.4 do    
-    . $CONDA_ROOT/etc/profile.d/conda.sh
-    conda activate
-else
-    #for conda version <4.4 do 
-    PATH=$CONDA_ROOT/bin:$PATH
-fi
-
-if [ "$(hostname --domain)" == "cm.gemini" ];then
-    module load ffmpeg
-    HYP_ENV="pytorch1.6_cuda10.1"
-    module load cuda10.1/toolkit/10.1.105
-    module load cudnn/7.6.3_cuda10.1
-else
-    #CUDA_ROOT=/home/janto/usr/local/cuda-10.1
-    CUDA_ROOT=/usr/local/cuda
-    LD_LIBRARY_PATH=$CUDA_ROOT/lib64:$LD_LIBRARY_PATH
-    LD_LIBRARY_PATH=$CUDA_ROOT/lib:$LD_LIBRARY_PATH
-    if [ ! -d $CUDA_ROOT/lib64 ]; then
-	LD_LIBRARY_PATH=$HOME/cuda/lib64:$LD_LIBRARY_PATH
-    fi
-
-    HYP_ENV="pytorch1.6_cuda10.2"
-    # #CuDNN env
-    # CUDNN_ROOT=$TOOLS_ROOT/cudnn/cudnn-10.1-v7.6
-    # LD_LIBRARY_PATH=$CUDNN_ROOT/lib64:$LD_LIBRARY_PATH
-    # LIBRARY_PATH=$CUDNN_ROOT/lib64:$LIBRARY_PATH
-    # CPATH=$CUDNN_ROOT/include:$CPATH
-fi
-
-export LRU_CACHE_CAPACITY=1 #this will avoid crazy ram memory when using pytorch with cpu, it controls cache of MKLDNN
-export HDF5_USE_FILE_LOCKING=FALSE
-
-export MPLBACKEND="agg"
-export PATH=$HYP_ROOT/hyperion/bin:$CUDA_ROOT/bin:$PATH
-export PYTHONPATH=$HYP_ROOT:$PYTHONPATH
-export LD_LIBRARY_PATH
-export LC_ALL=C
-
-wait_file() {
-    local file="$1"; shift
-    local wait_seconds="${2:-30}"; shift # 10 seconds as default timeout
-    for((i=0; i<$wait_seconds; i++)); do
-	[ -f $file ] && return 1
-	sleep 1s
-    done
-    return 0
-}
-
-export -f wait_file
+. $TOOLS_ROOT/path.sh