Export Models to ONNX (Feature Request)

[Christopher-Thornton]

As mentioned in #5 , the repository currently doesn't have a class for inferencing new data, and requires tweaking the model definitions to make this possible. It would greatly benefit the users if the repo had this out-of-the-box functionality.

I suggest a function be created to export a checkpoint model to ONNX, as is standard in pytorch-lightning for inferencing in production (from docs), along with any required compatibility changes to the model definitions.

torch.onnx.export(model,
                  args=example_input,
                  f=outname,
                  input_names=['input_ids', 'attention_mask', 'seq_len'],
                  output_names=['label'],
                  export_params=True)

Ideally a separate module e.g. inference.py can be written with optimized imports and a class to preprocess (tokenize) and run inferences from a collection of dialog strings.

[macabdul9]

@Christopher-Thornton you can create a PR if you have inferece.py and you have tested it else I will work on it asap.

[Christopher-Thornton]

Thanks, I will make a PR if I can get it working.

Edit:
I started working on the Inference module, which can be found here.
One of the issues I'm having is tracing some of the operations in the model.

[noshad-vida]

@Christopher-Thornton have you been able to get the inference module work with the saved checkpoints?

[glicerico]

@Christopher-Thornton , I have been trying to run your inference module. I know you're still working on it, but I wonder if you also get the same error:
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:seq_len
The suggestion is to check if the onnx model also contains the same input_names, and it does contain seq_len... so I am not certain what the problem may be.

[Christopher-Thornton]

@glicerico, I've encountered the same issue while exporting, and am unsure how to trace it back to the source of the problem. I'm hoping there is a reasonable solution for a similar issue posted elsewhere but haven't found anything.

[glicerico]

@Christopher-Thornton , I was unable to fix the ONNX problems, but I managed to perform inference without it. Check PR #12 ;)