Skip to content

“用 TensorRT-LLM & Triton Server 部署2B模型”请问这样部署是同步接口吗？貌似不能异步调用？ #132

New issue
New issue
Open
Open
“用 TensorRT-LLM & Triton Server 部署2B模型”请问这样部署是同步接口吗？貌似不能异步调用？#132
@dongteng

Description

@dongteng
dongteng
opened on Mar 19, 2024

同时请求会发生堵塞，一个请求结束之后另一个才出答案

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions