20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Remove max_num_seqs in latency benchmark script (#1855)

Browse Source
This commit is contained in:
Woosuk Kwon 2023-11-30 00:00:32 -08:00 committed by GitHub
parent e74b1736a1
commit 51d3cb951d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
benchmarks/benchmark_latency.py
View File

@ -19,7 +19,6 @@ def main(args: argparse.Namespace):
tokenizer=args.tokenizer, tokenizer=args.tokenizer,
quantization=args.quantization, quantization=args.quantization,
tensor_parallel_size=args.tensor_parallel_size, tensor_parallel_size=args.tensor_parallel_size,
max_num_seqs=args.batch_size,
trust_remote_code=args.trust_remote_code, trust_remote_code=args.trust_remote_code,
dtype=args.dtype, dtype=args.dtype,
) )