Skip to content

Commit e9e8443

Browse files
authored
fix num_blocks_local when small size model in TP2 running mode (#2792)
1 parent 749b2e9 commit e9e8443

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

fastdeploy/worker/worker_process.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -360,11 +360,11 @@ def determine_num_available_blocks(self) -> None:
360360
model_block_memory_used)
361361
# NOTE(liuzichang): Too many block will lead to illegal memory access
362362
# We will develop dynamic limits in future.
363-
if num_blocks_local > 20000:
363+
if num_blocks_local > 40000:
364364
logger.info(
365-
f"------- Reset num_blocks_local {num_blocks_local} to 20000"
365+
f"------- Reset num_blocks_local {num_blocks_local} to 40000"
366366
)
367-
num_blocks_local = min(20000, num_blocks_local)
367+
num_blocks_local = min(40000, num_blocks_local)
368368
logger.info(
369369
f"------- model_block_memory_used:{model_block_memory_used} --------"
370370
)

0 commit comments

Comments
 (0)