Explain where the engine args go when using Docker (#12041)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor 2025-01-14 16:05:50 +00:00 committed by GitHub
parent a2d2acb4c8
commit c9d6ff530b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -19,6 +19,8 @@ $ docker run --runtime nvidia --gpus all \
--model mistralai/Mistral-7B-v0.1
```
You can add any other <project:#engine-args> you need after the image tag (`vllm/vllm-openai:latest`).
```{note}
You can either use the `ipc=host` flag or `--shm-size` flag to allow the
container to access the host's shared memory. vLLM uses PyTorch, which uses shared