README: Clarify models and dataset structures, model instantiation and parameters (#149)

Author: Robert Muchsel

README.md

@@ -1,6 +1,6 @@
 # MAX78000 Model Training and Synthesis
 
-_July 14, 2021_
+_July 16, 2021_
 
 The Maxim Integrated AI project is comprised of five repositories:
 
@@ -1231,7 +1231,17 @@ For both approaches, the `quantize.py` software quantizes an existing PyTorch ch
 
 #### Quantization-Aware Training (QAT)
 
-Quantization-aware training is the better performing approach. It is enabled by default. QAT learns additional parameters during training that help with quantization (see [Weights: Quantization-Aware Training (QAT)](#Weights: Quantization-Aware Training (QAT)). No additional arguments are needed for `quantize.py`.
+Quantization-aware training is the better performing approach. It is enabled by default. QAT learns additional parameters during training that help with quantization (see [Weights: Quantization-Aware Training (QAT)](#Weights: Quantization-Aware Training (QAT)). No additional arguments (other than input, output, and device) are needed for `quantize.py`.
+
+The input checkpoint to `quantize.py` is either `qat_best.pth.tar`, the best QAT epoch’s checkpoint, or `qat_checkpoint.pth.tar`, the final QAT epoch’s checkpoint.
+
+Example:
+
+```shell
+(ai8x-synthesis) $ ./quantize.py proj/qat_best.pth.tar proj/proj_q8.pth.tar --device MAX78000
+```
+
+
 
 #### Post-Training Quantization
 
@@ -1241,6 +1251,17 @@ While several approaches for clipping are implemented in `quantize.py`, clipping
 
 Note that the “optimum” scale factor for simple clipping is highly dependent on the model and weight data. For the MNIST example, a `--scale 0.85` works well. For the CIFAR-100 example on the other hand, Top-1 performance is 30 points better with `--scale 1.0`.
 
+The input checkpoint to `quantize.py` for post-training quantization is typically `best.pth.tar`, the best epoch’s checkpoint, or `checkpoint.pth.tar`, the final epoch’s checkpoint.
+
+Example:
+
+```shell
+(ai8x-synthesis) $ ./quantize.py proj2/best.pth.tar proj2/proj2_q8.pth.tar \
+--device MAX78000 --scale 0.85 --clip-method SCALE
+```
+
+
+
 #### Command Line Arguments
 
 The `quantize.py` software has the following important command line arguments:
@@ -1259,7 +1280,7 @@ The `quantize.py` software has the following important command line arguments:
 
 *Note: The syntax for the optional YAML file is described below. The same file can be used for both `quantize.py` and `ai8xize.py`.*
 
-`quantize.py` does not need access to the dataset.
+*Note:* `quantize.py` does <u>not</u> need access to the dataset.
 
 #### Example and Evaluation
 
@@ -1295,18 +1316,47 @@ In all cases, ensure that the quantizer writes out a checkpoint file that the Ne
 
 The following step is needed to add new network models:
 
-Implement a new network model based on the constraints described earlier, see [Custom nn.Modules](#custom-nnmodules) (and `models/ai85net.py` for an example). 
+Implement a new network model based on the constraints described earlier, see [Custom nn.Modules](#custom-nnmodules) (and `models/ai85net.py` for an example).
+
+***Note:*** *When re-using existing models, please note that some of the example models are designed to be used with [Neural Architecture Search (NAS)](#Neural Architecture Search (NAS)). These models will typically not perform well, or not fit into hardware without the NAS steps. These models have “nas” as part of their name.*
+
+##### Model Instantiation and Initialization
+
+To support *evaluation* of the quantized model using PyTorch, the model must be instantiated and initialized using all parameters supplied by `train.py`, and the parameters must be passed to the individual *nn.Modules*.
+
+Example:
+
+```python
+class NewModel(nn.Module):
+    def __init__(self, num_classes=10, num_channels=3, dimensions=(64, 64), bias=False, **kwargs):
+        super().__init__()
+        self.conv1 = ai8x.FusedConv2dReLU(..., bias=bias, **kwargs)
+				...
+
+    def forward(self, x):
+      	...
+        
+def newmodel(pretrained=False, **kwargs):
+    ...
+    return NewModel(**kwargs)
+```
+
+Note the  `__init(...)__` function signature, the extra arguments to `ai8x.FusedConv2dReLU(...)` and the `NewModel(**kwargs)` instantiation.
 
 ##### `models` Data Structure
 
 The file must include the `models` data structure that describes the model. `models` can list multiple models in the same file.
 
 For each model, three fields are required in the data structure:
 
-* The `name` field assigns a name to the model for discovery by `train.py`, for example “`resnet5`”.
+* The `name` field assigns a name to the model for discovery by `train.py`, for example “`resnet5`”. *Note: The `name` must be unique.*
 * The `min_input` field describes the minimum width for 2D models, it is typically `1` *(when the input `W` dimension is smaller than `min_input`, it is padded to `min_input`)*.
 * The `dim` field is either `1` (the model handles 1D inputs) or `2` (the model handles 2D inputs).
 
+##### Model File Location
+
+Place the new model file (with its unique model name as specified by `name` in the data structure described above) into the `models` folder. `train.py` will now be able to discover and use the new model by specifying `--model modelname`.
+
 #### Data Loader
 
 The following steps are needed for new data formats and datasets:
@@ -1333,10 +1383,9 @@ On the other hand, a different sensor may produce unsigned data values in the fu
 
 Add the new data loader to a new file in the `datasets` directory (for example `datasets/mnist.py`). The file must include the `datasets` data structure that describes the dataset and points to the new loader. `datasets` can list multiple datasets in the same file.
 
-* The `input` field describes the dimensionality of the data, and the first dimension is passed as `num_channels` to the model, whereas the remaining dimensions are passed as `dimension`. For example, `'input': (1, 28, 28)` will be passed to the model as `num_channels=1` and `dimensions=(28,28)`.
-
+* The `name` field assigns a name to the dataset for discovery by `train.py`, for example “`MNIST`”. *Note: The `name` must be unique.*
+* The `input` field describes the dimensionality of the data, and the first dimension is passed as `num_channels` to the model, whereas the remaining dimensions are passed as `dimension`. For example, `'input': (1, 28, 28)` will be passed to the model as `num_channels=1` and `dimensions=(28, 28)`. One-dimensional input uses a single “dimension”, for example `'input': (2, 512)` will be passed to the model as `num_channels=2` and `dimensions=(512, )`.
 * The optional `regression` field in the structure can be set to `True` to automatically select the `--regression` command line argument. `regression` defaults to `False`.
-
 * The optional `visualize` field can point to a custom visualization function used when creating `--embedding`. The input to the function (format NCHW for 2D data, or NCL for 1D data) is a batch of data (with N ≤ 100). The default handles square RGB or monochrome images. For any other data, a custom function must be supplied.
 
 #### Training and Verification Data