20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
5,769 Commits 1 Branch 0 Tags
Commit Graph

1711 Commits

Author SHA1 Message Date
Nicolò Lucchesi
3cc9af88ff
[TPU][V1] Disable per-request seed/Generator (#16172) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-10 17:05:44 -04:00
Nick Hill
dd143ef541
[V1] Zero-copy tensor/ndarray serialization/transmission (#13790) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-10 19:23:14 +00:00
Lily Liu
e8224f3dca
[V1][Spec Decode] Eagle Model loading (#16035) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-04-10 11:21:48 -07:00
Russell Bryant
9665313c39
[V1] Set structured output backend to auto by default (#15724) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-10 17:53:26 +00:00
Cyrus Leung
83b824c8b4
[VLM] Remove BaseProcessingInfo.get_mm_max_tokens_per_item (#16408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 09:06:58 -07:00
Cyrus Leung
3d4c87758e
[Misc] Update transformers version limits of multi-modal tests (#16381) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-09 23:03:33 -07:00
Yuxuan Zhang
1e44ffc3ff
Add GLM-4-0414 support (#16338) 
Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Ajay Vohra <ajayvohr@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
Co-authored-by: Accelerator1996 <lvfei.lv@alibaba-inc.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: ajayvohra2005 <ajayvohr@amazon.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-04-10 09:19:42 +08:00
Chengji Yao
a454748544
[TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-04-09 18:51:51 -06:00
Guillaume Calmettes
98d01d3ce2
[Bugfix][Frontend] respect provided default guided decoding backend (#15476) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-04-09 05:11:10 -07:00
Accelerator1996
24f6b9a713
[Misc] Fix test_sharded_state_loader.py(#16004) (#16005) 
Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
 2025-04-09 14:47:30 +08:00
Luka Govedič
9cdde47289
[BugFix] Fix fusion test and add them to CI (#16287) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-04-08 23:46:45 -07:00
Chengji Yao
b1eb4ca152
[TPU] Update PyTorch/XLA (#16288) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-04-09 14:46:32 +08:00
Michael Goin
87b4ac56c2
[CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-09 04:14:46 +00:00
rongfu.leng
4716377fbc
[Feature] Estimate max-model-len use available KV cache memory (#16168) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-04-08 19:12:51 -07:00
Chauncey
102bf967f0
[Model] Add smolvlm support (#16017) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-08 19:12:17 -07:00
Jee Jee Li
86c3369eb8
[CI/Build] Fix CI LoRA failure (#16270) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-09 09:13:56 +08:00
Cyrus Leung
4ebc0b9640
[Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-08 09:45:21 -07:00
wang.yuqi
1f5d13ab9f
[New Model]: jinaai/jina-embeddings-v3 (#16120) 2025-04-08 08:39:12 -07:00
Kebe
e11880deea
[Bugfix] Remove triton do_bench fast_flush arg (#16256) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-04-08 13:51:06 +00:00
Michael Goin
8e5314a468
[V1] Add disable_chunked_mm_input arg to disable partial mm input prefill (#15837) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-07 23:24:07 -07:00
Isotr0py
f6b32efb7f
[Bugfix] Fix and reorganize broken GGUF tests and bump gguf version (#16194) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-08 13:38:13 +08:00
Michael Goin
b99733d092
[Bugfix] Do not skip "empty" parts of chats that are parsable (#16219) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-08 05:14:15 +00:00
Roger Wang
f2ebb6f541
[V1] Scatter and gather placeholders in the model runner (#16076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-04-08 10:43:41 +08:00
Driss Guessous
652907b354
Torchao (#14231) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-04-07 19:39:28 -04:00
leon-seidel
24f1c01e0f
[Bugfix][V0] XGrammar structured output supports Enum (#15878) 
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
 2025-04-07 22:38:25 +00:00
Nick Hill
7f6d47c1a2
[V1][BugFix] Exit properly if engine core fails during startup (#16137) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-07 15:30:15 -07:00
Nicolò Lucchesi
090c856d76
[Misc] Human-readable max-model-len cli arg (#16181) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-04-07 14:40:58 -04:00
Cyrus Leung
66d433b94f
[V1] Revert the default max_num_seqs to V0 values for most hardware (#16158) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 13:54:36 -04:00
Cyrus Leung
027b204ff1
[Bugfix] Re-enable support for ChatGLMForConditionalGeneration (#16187) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 23:15:58 +08:00
Lu Fang
55dcce91df
Upstream Llama4 Support to Main (#16113) 
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 08:06:27 -07:00
YamPengLi
7699258ef0
[Model] Add Qwen3 and Qwen3MoE (#15289) 
Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-07 04:06:41 -07:00
Roger Wang
bb8dab821e
[CI] Set max transformers version for Ultravox model test (#16149) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-04-07 04:37:58 +00:00
Isotr0py
fc0f87768a
[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-07 04:07:15 +00:00
Tristan Leclercq
4285e423a6
[Misc] Auto detect bitsandbytes pre-quantized models (#16027) 
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
 2025-04-04 23:30:45 -07:00
Roger Wang
af51d80fa1
Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus Leung
f5722a5052
[V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Mark McLoughlin
a35a8a8392
[V1][Spec Decode] Avoid logging useless nan metrics (#16023) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-04 08:52:41 -07:00
bnellnm
dcc56d62da
[Bugfix] Fix function names in test_block_fp8.py (#16033) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-04-03 23:01:34 +00:00
Robert Shaw
f15e70d906
[TPU] Switch Test to Non-Sliding Window (#15981) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-04-03 14:28:45 -07:00
iefgnoix
b6be6f8d1e
[TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) 
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
 2025-04-03 14:23:28 -07:00
bnellnm
15ba07ef25
[Minor] Fused experts refactor (#15914) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-04-03 10:19:38 -07:00
Liangfu Chen
d2b58ca203
[Neuron][kernel] Fuse kv cache into a single tensor (#15911) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-04-03 09:51:32 -07:00
Aleksandr Malyshev
e73ff24e31
[ROCM][KERNEL] Paged attention for V1 (#15720) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
 2025-04-02 19:48:00 -07:00
Hyesoo Yang
1b84eff03a
[V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
 2025-04-02 17:18:08 -07:00
Matthias Matt
cefb9e5a28
[Frontend] Implement Tool Calling with tool_choice='required' (#13483) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Matt, Matthias <matthias.matt@tuwien.ac.at>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
 2025-04-02 07:45:45 -07:00
Mark McLoughlin
98d7367b61
[Metrics] Hide deprecated metrics (#15458) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-02 07:37:19 -07:00
Chauncey
594a8b9030
[Bugfix] Fix the issue where the model name is empty string, causing no response with the model name. (#15938) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-02 06:33:52 -07:00
Russell Bryant
14e53ed11f
[V1] Fix json_object support with xgrammar (#15488) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-02 02:00:08 -07:00
Eric Tang
ddb94c2605
[core] Add tags parameter to wake_up() (#15500) 
Signed-off-by: Eric <erictang000@gmail.com>
 2025-04-02 01:59:27 -07:00
LukasBluebaum
90969fb39a
[Kernel] Add more dtype support for GGUF dequantization (#15879) 
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
 2025-04-02 01:58:48 -07:00