2024-12-23 17:35:38 -05:00
|
|
|
(engine-args)=
|
2023-11-22 21:31:27 +01:00
|
|
|
|
2024-12-23 17:35:38 -05:00
|
|
|
# Engine Arguments
|
2023-11-22 21:31:27 +01:00
|
|
|
|
|
|
|
Below, you can find an explanation of every engine argument for vLLM:
|
|
|
|
|
2024-12-23 17:35:38 -05:00
|
|
|
```{eval-rst}
|
2024-04-20 04:51:33 +01:00
|
|
|
.. argparse::
|
|
|
|
:module: vllm.engine.arg_utils
|
|
|
|
:func: _engine_args_parser
|
2024-07-17 15:43:21 +08:00
|
|
|
:prog: vllm serve
|
2024-04-21 17:15:28 +01:00
|
|
|
:nodefaultconst:
|
2024-12-23 17:35:38 -05:00
|
|
|
```
|
2024-04-04 23:52:01 -05:00
|
|
|
|
2024-12-23 17:35:38 -05:00
|
|
|
## Async Engine Arguments
|
2024-04-04 23:52:01 -05:00
|
|
|
|
2024-04-20 04:51:33 +01:00
|
|
|
Below are the additional arguments related to the asynchronous engine:
|
2024-04-04 23:52:01 -05:00
|
|
|
|
2024-12-23 17:35:38 -05:00
|
|
|
```{eval-rst}
|
2024-04-20 04:51:33 +01:00
|
|
|
.. argparse::
|
|
|
|
:module: vllm.engine.arg_utils
|
|
|
|
:func: _async_engine_args_parser
|
2024-07-17 15:43:21 +08:00
|
|
|
:prog: vllm serve
|
2024-12-23 17:35:38 -05:00
|
|
|
:nodefaultconst:
|
|
|
|
```
|