Check that sample input is int64; add conversion script (#151)

Author: Robert Muchsel

README.md

@@ -1,6 +1,6 @@
 # MAX78000 Model Training and Synthesis
 
-_July 16, 2021_
+_July 19, 2021_
 
 The Maxim Integrated AI project is comprised of five repositories:
 
@@ -2131,9 +2131,13 @@ Adding new datasets to the Network Loader is implemented as follows:
    fc.linear.bias   
    ```
 
-3. Provide a sample input. The sample input is used to generate a known-answer test (self test). The sample input is provided as a NumPy “pickle” — add `sample_dset.npy` for the dataset named `dset` to the `tests` directory. This file can be generated by saving a sample in CHW format (no batch dimension) using `numpy.save()`, see below.
+3. Provide a sample input. The sample input is used to generate a known-answer test (self test) against the predicted label. The purpose of the sample input is to ensure that the generated code matches the model — it does <u>*not*</u> ensure that the model is of good quality. However, it can help finding issues in the YAML description of the model.
 
-   For example, the MNIST 1×28×28 image sample would be stored in `tests/sample_mnist.npy` in an `np.array` with shape `[1, 28, 28]` and datatype `<i8`. The file can be random, or can be obtained from the `train.py` software.
+   The sample input is provided as a NumPy “pickle” — add `sample_dset.npy` for the dataset named `dset` to the `tests` directory. This file can be generated by saving a sample in CHW format (no batch dimension) using `numpy.save()`, see below.
+   
+   For example, the MNIST 1×28×28 image sample would be stored in `tests/sample_mnist.npy` in an `np.array` with shape `[1, 28, 28]` and datatype `>i8` (`np.int64`). The file can contain random integers, or it can be obtained from the `train.py` software.
+   
+   *Note: To convert an existing sample input file to `np.int64`, use the `tests/convert_sample.py` script.*
 
 #### Generating a Random Sample Input
 
@@ -2147,7 +2151,8 @@ a = np.random.randint(-128, 127, size=(1, 28, 28), dtype=np.int64)
 np.save(os.path.join('tests', 'sample_mnist'), a, allow_pickle=False, fix_imports=False)
 ```
 
-For RGB image inputs, there are three channels. For example, a 3×80×60 (C×H×W) input is created using `size=(3, 80, 60)`.
+For RGB image inputs, there are three channels. For example, a 3×80×60 (C×H×W) input is created using `size=(3, 80, 60)`. 
+**Note:** The array must be of data type `np.int64`.
 
 #### Saving a Sample Input from Training Data
 

README.pdf

@@ -28,6 +28,10 @@ def get(
     #         allow_pickle=False, fix_imports=False)
 
     data = np.load(filename)
+    if data.dtype.type is not np.dtype('int64').type:
+        eprint(f'The sample data array in {filename} is of type {data.dtype}, rather than '
+               'int64!')
+
     if synthesize_input is not None:
         # Every 8 (or synthesize_words) words, add data to the
         # combined 32-bit word for up to 4 channels

izer/sampledata.py

@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+###################################################################################################
+# Copyright (C) Maxim Integrated Products, Inc. All Rights Reserved.
+#
+# Maxim Integrated Products, Inc. Default Copyright Notice:
+# https://www.maximintegrated.com/en/aboutus/legal/copyrights.html
+###################################################################################################
+"""
+Convert sample input data to int64.
+"""
+import argparse
+
+import numpy as np
+
+parser = argparse.ArgumentParser(description='Convert sample input to int64')
+parser.add_argument('input', help='path to the input sample data')
+parser.add_argument('output', help='path to the output file (converted data)')
+args = parser.parse_args()
+
+d = np.int64(np.load(args.input))
+np.save(args.output, d, allow_pickle=False, fix_imports=False)