You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
From my experience with interacting with Triton Server using Python client, I haven't been able to find a way to get a model's configuration without loading it first, when the server is running in EXPLICIT mode. The reason for that is I have a logic to load a specific version of a model, that updates the runtime model config upon loading. I had to use a context that loads and unloads a model to perform this operation.
While loading and unloading an ONNX model can be quite fast, it causes a lot of latency to do the same for Python backend models. If there was an API/method to simply read the model configuration for EXPLICIT mode that I didn't know about, I'll be happy to hear details about them. Any other opinions/sharing are welcome too.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Dear Triton Community and Maintainers,
From my experience with interacting with Triton Server using Python client, I haven't been able to find a way to get a model's configuration without loading it first, when the server is running in EXPLICIT mode. The reason for that is I have a logic to load a specific version of a model, that updates the runtime model config upon loading. I had to use a context that loads and unloads a model to perform this operation.
While loading and unloading an ONNX model can be quite fast, it causes a lot of latency to do the same for Python backend models. If there was an API/method to simply read the model configuration for EXPLICIT mode that I didn't know about, I'll be happy to hear details about them. Any other opinions/sharing are welcome too.
Thank you in advance.
Scot
Beta Was this translation helpful? Give feedback.
All reactions