how can i use triton_python_backend_utils?

[YooSungHyun]

Description
in trtionserver:23.05. and git clone https://github.com/triton-inference-server/python_backend -b r23.05

root@4248wefe0f480:~# python3

import triton_python_backend_utils
ModuleNotFoundError: No module named 'triton_python_backend_utils'

how can i use it? how can i doing pip install?
it just cp {workspace}/python_backend/src/resources/triton_python_backend_utils.py {model_path}/1/triton_python_backend_utils.py and use in model.py?

Triton Information
What version of Triton are you using?
23.05

Are you using the Triton container or did you build it yourself?
using tritonserver container

Expected behavior
A clear and concise description of what you expected to happen.

import done well

[kthui]

I think the triton_python_backend_utils will be available when the Python script is ran by Triton. Would you be able to try the quick start? The model.py on the quick start is a good example on importing triton_python_backend_utils.

[YooSungHyun]

@kthui thx for comment!

wow that is so amazing...!
in my python code, warning like this, and i don't have triton_python_backend_utils.py anywhere.

but, when running tritonserver, it work good...!

how is this possible?

[nnshah1]

triton_python_back_utils is added to the sys.path when the model is launched by Triton (in its own process) - that's why it is available during execution.

It is not installed in the path of your local python environment - but it could be copied out for linting purposes (note however it is not designed to be used outside of the Triton environment)

[kthui]

triton_python_backend_utils is a utility library provided by the Triton Python backend, see this comment for its purpose.

[Galoev]

How can I install this module (triton_python_backend_utils) so that I can have code auto-complete during development?

[nnshah1]

@rmccorm4 , @Tabrizian - Did we have a recipe for code complete during development?

[Tabrizian]

We don't have a recipe right now but we do have a few items to improve this. Related GH issue: #5961

[MatteoPagliani]

Hi @Tabrizian. Can you share your progress on this, please?

[mshajarrazip]

If you have stumbled upon this discussion because you are trying to figure out how to debug TritonPythonModel, then one way to add a lot of logging messages while running the tritonserver, and start from there. Or, you could create a mock triton_python_backend_utils module to debug alongside a model.py file, something like this:

├── mock
│   ├── main.py # attach a debugger 
│   ├── model.py # contains TritonPythonModel
│   └── triton_python_backend_utils.py # define the functionalities of pb_utils, minimally

Personally, I like the mock environment idea. I haven't gone through the rest of Triton's documentation yet so maybe the answer is more obvious to some of you 🥲 But I hope it helps for somebody who's kinda stuck at this.