20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[benchmark] Update TGI version (#7917)

Browse Source
This commit is contained in:
Philipp Schmid 2024-08-28 00:07:53 +02:00 committed by GitHub
parent fc911880cc
commit 345be0e244
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
benchmarks/launch_tgi_server.sh
View File

@ -6,7 +6,7 @@ TOKENS=$2
docker run -e HF_TOKEN=$HF_TOKEN --gpus all --shm-size 1g -p $PORT:80 \ docker run -e HF_TOKEN=$HF_TOKEN --gpus all --shm-size 1g -p $PORT:80 \
-v $PWD/data:/data \ -v $PWD/data:/data \
ghcr.io/huggingface/text-generation-inference:1.4.0 \ ghcr.io/huggingface/text-generation-inference:2.2.0 \
--model-id $MODEL \ --model-id $MODEL \
--sharded false \ --sharded false \
--max-input-length 1024 \ --max-input-length 1024 \