20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
4,040 Commits 1 Branch 0 Tags
Commit Graph

582 Commits

Author SHA1 Message Date
Cyrus Leung
c0efe92d8b
[Doc] Add note to gte-Qwen2 models (#11808) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-07 21:50:58 +08:00
youkaichao
d9fa1c05ad
[doc] update how pip can install nightly wheels (#11806) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-07 21:42:58 +08:00
Roger Wang
2de197bdd4
[V1] Support audio language models on V1 (#11733) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-07 19:47:36 +08:00
youkaichao
869e829b85
[doc] add doc to explain how to use uv (#11773) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-07 18:41:17 +08:00
Roger Wang
8082ad7950
[V1][Doc] Update V1 support for LLaVa-NeXT-Video (#11798) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-01-07 09:55:39 +00:00
Russell Bryant
ce1917fcf2
[Doc] Create a vulnerability management team (#9925) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-01-06 22:57:32 -08:00
Cyrus Leung
8ceffbf315
[Doc][3/N] Reorganize Serving section (#11766) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-07 11:20:01 +08:00
Roger Wang
91b361ae89
[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 19:58:16 +00:00
youkaichao
4ca5d40adc
[doc] explain how to add interleaving sliding window support (#11771) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-06 21:57:44 +08:00
Cyrus Leung
ee77fdb5de
[Doc][2/N] Reorganize Models and Usage sections (#11755) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 21:40:31 +08:00
Suraj Deshmukh
2a622d704a
k8s-config: Update the secret to use stringData (#11679) 
Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
 2025-01-06 08:01:22 +00:00
Cyrus Leung
402d378360
[Doc] [1/N] Reorganize Getting Started section (#11645) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 02:18:33 +00:00
Alberto Ferrer
d1d49397e7
Update bnb.md with example for OpenAI (#11718) 2025-01-04 06:29:02 +00:00
Hust_YangXian
9c93636d84
Update tool_calling.md (#11701) 2025-01-04 06:16:30 +00:00
Sachin Varghese
2f1e8e8f54
Update default max_num_batch_tokens for chunked prefill (#11694) 2025-01-03 00:25:53 +00:00
Chunyang Wen
84c35c374a
According to vllm.EngineArgs, the name should be distributed_executor_backend (#11689) 2025-01-02 18:14:16 +00:00
Cyrus Leung
365801fedd
[VLM] Add max-count checking in data parser for single image models (#11661) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-31 22:15:21 -08:00
Roger Wang
e7c7c5e822
[V1][VLM] V1 support for selected single-image models. (#11632) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-12-31 21:17:22 +00:00
Matthias Vogler
a2a40bcd0d
[Model][LoRA]LoRA support added for MolmoForCausalLM (#11439) 
Signed-off-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Matthias Vogler <matthias.vogler@joesecurity.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-30 17:33:06 -08:00
youkaichao
b12e87f942
[platforms] enable platform plugins (#11602) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 20:24:45 +08:00
Cyrus Leung
32b4c63f02
[Doc] Convert list tables to MyST (#11594) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-29 15:56:22 +08:00
youkaichao
328841d002
[bugfix] interleaving sliding window for cohere2 model (#11583) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-28 16:55:42 +00:00
Cyrus Leung
d427e5cfda
[Doc] Minor documentation fixes (#11580) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-28 21:53:59 +08:00
Isotr0py
d34be24bb1
[Model] Support InternLM2 Reward models (#11571) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-28 06:14:10 +00:00
Cyrus Leung
101418096f
[VLM] Support caching in merged multi-modal processor (#11396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-27 17:22:48 +00:00
Chen1022
5ce4627a7e
[Doc] Add xgrammar in doc (#11549) 
Signed-off-by: ccjincong <chenjincong11@gmail.com>
 2024-12-27 13:05:10 +00:00
AlexHe99
d003f3ea39
Update deploying_with_k8s.md with AMD ROCm GPU example (#11465) 
Signed-off-by: Alex He <alehe@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-27 10:00:04 +00:00
Robert Shaw
0c0c2015c5
Update openai_compatible_server.md (#11536) 
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2024-12-26 16:26:18 -08:00
Simon Mo
82d24f7aac
[Docs] Document Deepseek V3 support (#11535) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2024-12-26 16:21:56 -08:00
Isotr0py
b85a977822
[Doc] Add video example to openai client for multimodal (#11521) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-26 17:31:29 +00:00
Roger Wang
7492a36207
[Doc] Add QVQ and QwQ to the list of supported models (#11509) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-12-26 09:44:32 +00:00
Cyrus Leung
6ad909fdda
[Doc] Improve GitHub links (#11491) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-25 14:49:26 -08:00
Cyrus Leung
3f3e92e1f2
[Model] Automatic conversion of classification and reward models (#11469) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 18:22:22 +00:00
Cyrus Leung
9edca6bf8f
[Frontend] Online Pooling API (#11457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 17:54:30 +08:00
Rafael Vasquez
32aa2059ad
[Docs] Convert rST to MyST (Markdown) (#11145) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-23 22:35:38 +00:00
Yuan Tang
2e726680b3
[Bugfix] torch nightly version in ROCm installation guide (#11423) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2024-12-23 17:20:22 +00:00
youkaichao
5d2248d81a
[doc] explain nccl requirements for rlhf (#11381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-20 13:00:56 -08:00
omer-dayan
995f56236b
[Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) 
Signed-off-by: OmerD <omer@run.ai>
 2024-12-20 16:46:24 +00:00
youkaichao
1ecc645b8f
[doc] backward compatibility for 0.6.4 (#11359) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-19 21:33:53 -08:00
youkaichao
7801f56ed7
[ci][gh200] dockerfile clean up (#11351) 
Signed-off-by: drikster80 <ed.sealing@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: cenzhiyao <2523403608@qq.com>
 2024-12-19 18:13:06 -08:00
Yehoshua Cohen
6c7f881541
[Model] Add JambaForSequenceClassification model (#10860) 
Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 22:48:06 +08:00
Travis Johnson
17ca964273
[Model] IBM Granite 3.1 (#11307) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-12-19 11:27:24 +08:00
kYLe
66d4b16724
[Frontend] Add OpenAI API support for input_audio (#11027) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-16 22:09:58 -08:00
youkaichao
35bae114a8
fix gh200 tests on main (#11246) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-16 17:22:38 -08:00
bk-TurbaAI
35ffa682b1
[Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (#11235) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-12-16 22:20:39 +00:00
Jani Monoses
bddbbcb132
[Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203) 2024-12-16 09:56:19 +00:00
cennn
b3b1526f03
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (#11212) 
Signed-off-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
 2024-12-16 09:20:49 +00:00
AlexHe99
da6f409246
Update deploying_with_k8s.rst (#10922) 2024-12-15 16:33:58 -08:00
Kuntai Du
38e599d6a8
[Doc] add documentation for disaggregated prefilling (#11197) 
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
 2024-12-15 13:31:16 -06:00
Jee Jee Li
15859f2357
[[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) 2024-12-15 03:03:06 +00:00