6dd55af6c9
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Roger Wang <ywang@roblox.com> Co-authored-by: Roger Wang <ywang@roblox.com>
876 B
876 B
(engine-args)=
Engine Arguments
Engine arguments control the behavior of the vLLM engine.
- For offline inference, they are part of the arguments to
LLMclass. - For online serving, they are part of the arguments to
vllm serve.
Below, you can find an explanation of every engine argument:
.. argparse::
:module: vllm.engine.arg_utils
:func: _engine_args_parser
:prog: vllm serve
:nodefaultconst:
Async Engine Arguments
Additional arguments are available to the asynchronous engine which is used for online serving:
.. argparse::
:module: vllm.engine.arg_utils
:func: _async_engine_args_parser
:prog: vllm serve
:nodefaultconst: