How to make batch inference in pipeline Transformers #3614
Unanswered
khongtrunght
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm new to BentoML. When I read this example : https://github.com/bentoml/BentoML/tree/main/examples/inference_graph . I see that the model is not batched.
When I follow the tutorial Adaptive Batching, I do somthing like that:
But it not work and yield error when run, can anybody with experiments helps me?
Beta Was this translation helpful? Give feedback.
All reactions