20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Bugfix] Fix a path bug in disaggregated prefill example script. (#12121)

Browse Source 
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
This commit is contained in:
Kuntai Du 2025-01-17 11:12:41 +08:00 committed by GitHub
parent d06e824006
commit ebc73f2828
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
examples/online_serving/disaggregated_prefill.sh
View File

@ -3,6 +3,8 @@
# We will launch 2 vllm instances (1 for prefill and 1 for decode), # We will launch 2 vllm instances (1 for prefill and 1 for decode),
# and then transfer the KV cache between them. # and then transfer the KV cache between them.
set -xe
echo "🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧" echo "🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧"
sleep 1 sleep 1
@ -69,7 +71,7 @@ wait_for_server 8200
# instance # instance
# NOTE: the usage of this API is subject to change --- in the future we will # NOTE: the usage of this API is subject to change --- in the future we will
# introduce "vllm connect" to connect between prefill and decode instances # introduce "vllm connect" to connect between prefill and decode instances
python3 ../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py & python3 ../../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
sleep 1 sleep 1
# serve two example requests # serve two example requests