How to get model configuration from HTTP API without first loading the model in EXPLICIT mode?

[scotgopal]

Dear Triton Community and Maintainers,

From my experience with interacting with Triton Server using Python client, I haven't been able to find a way to get a model's configuration without loading it first, when the server is running in EXPLICIT mode. The reason for that is I have a logic to load a specific version of a model, that updates the runtime model config upon loading. I had to use a context that loads and unloads a model to perform this operation.

with self._model_config_context(model_name) as model_cfg:
    version_policy = (
        {"latest": {"num_versions": 1}}
        if model_version == "latest"
        else {"specific": {"versions": [model_version]}}
    )
    
    model_cfg.update({"version_policy": version_policy})
    if init_params:
        model_cfg["parameters"].update(
            {k: {"string_value": str(v)} for k, v in init_params.items()}
        )
    self.logger.debug(f"Model config: {model_cfg}")
    new_model_cfg = model_cfg.copy()
    
    self.client.load_model(model_name, config=json.dumps(new_model_cfg))

While loading and unloading an ONNX model can be quite fast, it causes a lot of latency to do the same for Python backend models. If there was an API/method to simply read the model configuration for EXPLICIT mode that I didn't know about, I'll be happy to hear details about them. Any other opinions/sharing are welcome too.

Thank you in advance.
Scot

[indrajit96]

Hi @scotgopal ,
You can get all your model meta data directly from the config.pbtxt of the model which is part of the model repo.
You can do this outside of triton apis and should be quick as would not require loading and unloading of models.

[scotgopal]

That could be a little tricky to pull off from the client side, in which the models are not available via filesystem.

Even if I were to create a model with python backend, e.g. config_reader that reads the model's protobuf config file, how do I get the information of where in the file system does the requested model live? Considering the fact that multiple local and remote model repositories may be included during the startup of a single triton server.

I also would prefer having an auto completed config, rather than only what's available in the config.pbtxt.