Trouble changing dynamic input in onnx file (outdated tutorial)

[gingjehli]

Hi,
I've trained a LAN on a new model and I'm trying to embed the resulting onnx file into HSSM.
For doing so, I've followed this tutorial: https://lnccbrown.github.io/HSSM/tutorials/blackbox_contribution_onnx_example/#build-hssm-models-starting-from-onnx-files

I can embed the onnx file and simulate data from the new model.
However, I've troubles to use the snippet below in the tutorial to actually make the network load dynamic input dimensions.
Specifically, this code snippet from the tutorial below is not working:

"
import onnx
import onnxruntime as ort

onnx_model = onnx.load(network_path)

for input_tensor in onnx_model.graph.input:
dim_proto = input_tensor.type.tensor_type.shape.dim[0]
if not dim_proto.dim_param == "None":
dim_proto.dim_param = "None"

for output_tensor in onnx_model.graph.output:
dim_proto = output_tensor.type.tensor_type.shape.dim[0]
if not dim_proto.dim_param == "None":
dim_proto.dim_param = "None"

input_name = onnx_model.graph.input[0].name

onnx.save(onnx_model, "test_files/race_3_no_bias_lan_batch.onnx")
"

I've tried to attach the corresponding onnx file and jupyter notebook to this thread.

In the jupyter notebook, you can see that I can compile the model but when I sample from it, it throws an error saying that the onnx file has dimension 1 but because of the simulated data, it is expecting 500.

what changes do I have to make?

[AlexanderFengler]

@gingjehli was this resolved?

Need to double check what is going on, but this code snippet is in fact specifically for allowing input and output dim's to be batchable (dynamic).

[gingjehli]

Hi Alex,
Yes - this is resolved. I had to make a few changes in the tutorial and I'm summarizing them below just so you have them:

1. In the tutorial, the corrected network is put into the "test_files" subdirectory (this is the last commented line), but then it is reloading the original file from the "data" directory. When I instead made the ort_session like this:

Load model batch ready model

ort_session = ort.InferenceSession(network_test_path)
...where "network_test_path" is the the new onnx file in the test_files subdirectory, suddenly things started working.

2. To get rid off the annoying thread error messages:
   The cell starting with "# Load model batch ready model" needs to have 3 extra lines at the top:
   sess_options = ort.SessionOptions()
   sess_options.intra_op_num_threads = 4
   sess_options.inter_op_num_threads = 4
   and a modified InferenceSession constructor:
   ort_session = ort.InferenceSession(network_test_path, sess_options). Note that the "= 4" is kind of arbitrary, but it gets rid of the errors and doesn't seem to change the processing speed.

3. when compiling the model in HSSM, I had to set p_outlier=0.1, (any value bigger than zero)