Skip to content

Commit 402f574

Browse files
fix: fix the bug that openai api frontend not able to start (#8163)
1 parent 1c7aaaa commit 402f574

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

src/python/tritonfrontend/_api/_kservegrpc.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ class Options:
7979
infer_compression_level: Union[
8080
int, Grpc_compression_level
8181
] = Grpc_compression_level.NONE
82+
infer_thread_count: int = Field(2, ge=0)
8283
infer_allocation_pool_size: int = Field(8, ge=0)
8384
max_response_pool_size: int = Field(2_147_483_647, ge=0)
8485
forward_header_pattern: str = ""

src/python/tritonfrontend/_api/_kservegrpc.pyi

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ class KServeGrpc:
5656
max_connection_age_ms: int
5757
max_connection_age_grace_ms: int
5858
infer_compression_level: int | Grpc_compression_level
59+
infer_thread_count: int
5960
infer_allocation_pool_size: int
6061
max_response_pool_size: int
6162
forward_header_pattern: str

0 commit comments

Comments
 (0)