How is model.ts generated from model-zoo

[liqunfu]

MONAI bundle from model zoo can be exported:
python -m monai.bundle trt_export --net_id network_def --filepath wholeBody_ct_segmentation/models/model_trt.ts --ckpt_file wholeBody_ct_segmentation/models/model.pt --meta_file wholeBody_ct_segmentation/configs/metadata.json --config_file wholeBody_ct_segmentation/configs/inference.json --precision fp32 --use_trace "True"

In this case weights are stored in a constants folder in model_trt.ts.

However, deploy sdk expects weights in a data folder:

monai-deploy-app-sdk/monai/deploy/core/models/torch_model.py

Line 103 in 2ce415f

if pivot != -1 and not prefix_data and file_name[pivot:].startswith("/data/"):

this means the exported model_trt.ts is not usable with deploy sdk.

So my question is:
Is there a process to export bundles from model-zoo to torch script models that can be used in the monai deploy sdk?

[gigony]

Thanks @liqunfu for the report!

It seems that the MONAI Deploy SDK doesn't yet support the new TorchScript file format.
PR #546 updates the identification logic, and we hope the patch resolves the issue once it's released.

As a workaround, you can manually patch the code in the installation directory, which you can locate with:
python -c "import monai.deploy; print(monai.deploy.__file__)"

For example:
.pyenv/versions/monai-3.10/lib/python3.10/site-packages/monai/deploy/core/models/torch_model.py

[MMelQin]

Thanks @liqunfu for raising the question and @gigony for adding the support for the new TorchScript format.

Most of the example application and Notebooks are using TrochScript models from Model Zoo that were created a while back, so they continued to work, but the newly export TorchScript models would have issues as reported in this issue.

Also, the SDK tries to load the models from the given path with its Model Factory object, and if none of the supported ones could be found or loaded, an app or a custom inference operator can still rely on the model path to locate and load the model network itself and continue with the workflow. This should work for say PyTorch models.

[MMelQin]

TorchScript itself does not save constants in a constants folder in the archive, see here, and the model file mentioned in the issue was exported with --trt_export flag, so it is the MONAI Bundle that uses the constants folder instead of constants.pkl for TRT optimized model. TorchScript does mention about an optional tensors folder.