Skip to content

Commit 5668c79

Browse files
committed
server: bench: enable flash_attn param
1 parent 4053857 commit 5668c79

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

examples/server/bench/bench.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,7 @@ def start_server_background(args):
268268
server_args.extend(['--defrag-thold', "0.1"])
269269
server_args.append('--cont-batching')
270270
server_args.append('--metrics')
271+
server_args.append('--flash-attn')
271272
server_args.extend(['--log-format', "text"])
272273
args = [str(arg) for arg in [server_path, *server_args]]
273274
print(f"bench: starting server with: {' '.join(args)}")

0 commit comments

Comments
 (0)