Merge pull request #19 from marco-willi/add_models

add pre-trained models for download

Author: marco-willi

README.md

@@ -19,6 +19,7 @@ This repository contains code and documentation to train and apply convolutional
 3. Multi-output modelling: model species, behaviors, and any other label at the same time.
 4. A large variety of options: models, data augmentation, installation, data-prep, transfer-learning, and more.
 5. Tested approach: This code is currently in use and is being developed further.
+6. Trained models for download: [Trained Models](docs/trained_models.md)
 
 ## Pre-Requisites
 
@@ -340,6 +341,10 @@ This will apply the following transformations to each image during model trainin
 
 This feature is experimental in the sense that it has not yet been thoroughly tested. We think that this option may benefit models which have to identify the presence of animals -- especially if there are small animals that are difficult to see without the motion information.
 
+## Downloading Trained Models
+
+See the following documentation about trained models available for download: [Trained Models](docs/trained_models.md)
+
 
 ## Testing the Code
 

docs/figures/ENO_S1_model_evaluation.pdf

@@ -0,0 +1,61 @@
+"SER_S11#B04#1#62":
+  {"predictions_top":
+    {"standing": "0",
+     "interacting": "0",
+     "young_present": "0",
+     "moving": "1",
+     "eating": "0",
+     "count": "1",
+     "species": "zebra",
+     "resting": "0"},
+"aggregated_pred":
+  {"count":
+    {"7": "0.0000", "2": "0.0317", "10": "0.0000", "4": "0.0005",
+     "1": "0.9650", "5": "0.0001", "3": "0.0026", "11-50": "0.0000",
+     "8": "0.0000", "51+": "0.0000", "6": "0.0000", "9": "0.0000"},
+   "interacting": {"0": "0.9999", "1": "0.0001"},
+   "young_present": {"0": "0.9993", "1": "0.0007"},
+   "moving": {"0": "0.1518", "1": "0.8482"},
+   "eating": {"0": "0.9830", "1": "0.0170"},
+   "standing": {"0": "0.8392", "1": "0.1608"},
+   "species": {
+     "baboon": "0.0000", "civet": "0.0000", "buffalo":"0.0000",
+     "kudu": "0.0000", "rodents": "0.0000", "waterbuck": "0.0000",
+     "vulture": "0.0000", "impala": "0.0000", "human": "0.0000",
+     "lion": "0.0000", "genet": "0.0000", "mongoose": "0.0000",
+     "caracal": "0.0000", "rhinoceros": "0.0000", "klipspringer": "0.0000",
+     "topi": "0.0000", "secretarybird": "0.0000", "sitatunga": "0.0000",
+     "springbok": "0.0000", "duiker": "0.0000", "hartebeest": "0.0000",
+     "ottercapeclawless": "0.0000", "hyenabrown": "0.0000",
+     "bushpig": "0.0000", "domesticanimal": "0.0000", "pangolin": "0.0000",
+     "bontebok": "0.0000", "gemsbokoryx": "0.0000", "wilddog": "0.0000",
+     "rabbitriverine": "0.0000", "serval": "0.0000", "fallowdeer": "0.0000",
+     "wildcat": "0.0000", "groundsquirrel": "0.0000", "hippopotamus": "0.0000",
+     "warthog": "0.0000", "reedbuck": "0.0000", "meerkatsuricate": "0.0000",
+     "dikdik": "0.0000", "craneblue": "0.0000", "reptiles": "0.0000",
+     "birdother": "0.0000", "zebra": "0.9999", "cattle": "0.0000",
+     "foxbateared": "0.0000", "bushbaby": "0.0000", "steenbok": "0.0000",
+     "tsessebe": "0.0000", "grysbok": "0.0000", "rhebokgrey": "0.0000",
+     "wildebeest": "0.0000", "hyenastriped": "0.0000", "bat": "0.0000",
+     "ostrich": "0.0000", "tortoise": "0.0000", "aardwolf": "0.0000",
+     "eland": "0.0000", "rabbitredrock": "0.0000", "honeybadger": "0.0000",
+     "roan": "0.0000", "giraffe": "0.0000", "hare": "0.0000",
+     "foxcape":"0.0000", "nyala": "0.0000", "blesbok": "0.0000",
+     "monkeysamango": "0.0000", "insectspider": "0.0000",
+     "sable": "0.0000", "jackal": "0.0000", "gazellethomsons": "0.0000",
+     "zorilla": "0.0000", "bustardludwigs": "0.0000",
+     "gazellegrants": "0.0000", "monkeyvervet": "0.0000",
+     "aardvark": "0.0000", "hyenaspotted": "0.0000", "bushbuck": "0.0000",
+     "suni": "0.0000", "bustardkori": "0.0000", "porcupine": "0.0000",
+     "cheetah": "0.0000", "leopard": "0.0000", "fire": "0.0000",
+     "elephant": "0.0000", "guineafowl":"0.0000"},
+   "resting": {"0": "0.9995", "1": "0.0005"}},
+"confidences_top": {
+   "standing": "0.8392",
+   "interacting": "0.9999",
+   "young_present": "0.9993",
+   "moving": "0.8482",
+   "eating": "0.9830",
+   "count": "0.9650",
+   "species": "0.9999",
+   "resting": "0.9995"}}

docs/figures/Evaluation_SnapshotSafariModels.pdf

@@ -0,0 +1,107 @@
+# Trained Models
+
+The following trained models are available for downloading. The models were trained on AWS using a p3.2xlarge EC2  instance and Docker as described here [AWS example](../docs/Docker_GPU.md). The models can be apply as described here [Model Application](../README.md#5-model-use).
+
+## Empty or Not Model
+
+Download from: https://s3.msi.umn.edu/snapshotsafari/models/empty_or_not/ResNet18_448_v1.zip
+
+* Number of capture events in training set: 3.66 million
+* Architecture: ResNet18
+* Input Resolution: 448x448
+* Images: mainly from Tanzania (Serengeti National Park) and South Africa
+* Output: is_blank - binary label indicating whether the capture is empty ('blank'), i.e. no animal or human
+
+
+### Training Parameters
+
+The model was trained using the following command:
+
+```
+sudo docker exec ctc ctc.train \
+  -train_tfr_path $TFR_FILES \
+  -val_tfr_path $TFR_FILES \
+  -test_tfr_path $TFR_FILES \
+  -class_mapping_json  $LABEL_MAPPINGS \
+  -run_outputs_dir $RUN_OUTPUT \
+  -model_save_dir $MODEL_SAVE \
+  -model ResNet18 \
+  -labels is_blank \
+  -batch_size 64 \
+  -n_cpus 8 \
+  -n_gpus 1 \
+  -buffer_size 512 \
+  -max_epochs 70 \
+  -reduce_lr_on_plateau_patience 4 \
+  -early_stopping_patience 6 \
+  -output_width 448 \
+  -output_height 448 \
+  -n_batches_per_epoch_train 5000 \
+  -rotate_by_angle 0 \
+  -zoom_factor 0
+```
+
+## Species Model
+
+Download from: https://s3.msi.umn.edu/snapshotsafari/models/species/Xception_v1.zip
+
+* Number of captures in training set: 1.84 million
+* Architecture: Xception
+* Input Resolution: 299x299
+* Images: mainly from Tanzania (Serengeti National Part) and South Africa
+* Output: species, counts, standing, resting, moving, eating, interacting, young_present
+* Number of species: 85
+
+<img src="https://github.com/marco-willi/camera-trap-classifier/blob/master/docs/figures/zebra_example.png"/>
+
+*This figure shows a capture of a Zebra (see corresponding model output below).*
+
+Example Model Output for the image above: [Xception_v1 Output](../docs/figures/example_pred_Xception_v1.json). This file also shows the complete list of species and all the other labels that the model was trained to classify.
+
+### Training Parameters
+
+The model was trained using the following command:
+
+```
+sudo docker exec ctc ctc.train \
+  -train_tfr_path $TFR_FILES \
+  -val_tfr_path $TFR_FILES \
+  -test_tfr_path $TFR_FILES \
+  -class_mapping_json  $LABEL_MAPPINGS \
+  -run_outputs_dir $RUN_OUTPUT \
+  -model_save_dir $MODEL_SAVE \
+  -model Xception \
+  -labels species count standing resting moving eating interacting young_present \
+  -batch_size 32 \
+  -n_cpus 8 \
+  -n_gpus 1 \
+  -buffer_size 512 \
+  -max_epochs 70 \
+  -reduce_lr_on_plateau_patience 6 \
+  -early_stopping_patience 10 \
+  -rotate_by_angle 0 \
+  -zoom_factor 0 \
+  -n_batches_per_epoch_train 10000 \
+  -max_epochs 70
+```
+
+## Evaluation Results
+
+### Results on Test-Data (in sample data)
+
+The following report shows evaluation results on the test data: [Evaluation Results](../docs/figures/Evaluation_SnapshotSafariModels.pdf).
+
+Shown are results separated by different locations / parks. The overwhelming majority of the images is from 'SER' the Serengeti national park.
+
+The data is refered to as 'in sample data' because it is based on data from the same locations as used in training the model. This may introduce a positive bias to the results. Furthermore, the species model is evaluated on species images only (i.e. the model could not make the error of predicting a species when there is none).
+
+
+### Results on Independent Data (out of sample data)
+
+The following reports show model evaluations using independent test data:
+
+* Grumeti: [Evaluation Results Grumeti](../docs/figures/GRU_S1_model_evaluation.pdf)
+* Enonkishu: [Evaluation Results Enonkishu](../docs/figures/ENO_S1_model_evaluation.pdf)
+* Kruger: [Evaluation Results Kruger](../docs/figures/KRU_S1_model_evaluation.pdf)
+
+The evaluation from these locations is based on a realistic scenario, on data collected from locations (Enonkishu and Kruger) the model has not seen or mostly not seen (Grumeti), as well as by applying a two-stage process: 1) identify images with animals and 2) identify species on images considered as containing an animal. Note that some (rare) species from Kruger / Enonkishu are completely unknown to the model and thus not predictable.