20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
5,662 Commits 1 Branch 0 Tags
Commit Graph

1678 Commits

Author SHA1 Message Date
Tristan Leclercq
4285e423a6
[Misc] Auto detect bitsandbytes pre-quantized models (#16027) 
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
 2025-04-04 23:30:45 -07:00
Roger Wang
af51d80fa1
Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus Leung
f5722a5052
[V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Mark McLoughlin
a35a8a8392
[V1][Spec Decode] Avoid logging useless nan metrics (#16023) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-04 08:52:41 -07:00
bnellnm
dcc56d62da
[Bugfix] Fix function names in test_block_fp8.py (#16033) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-04-03 23:01:34 +00:00
Robert Shaw
f15e70d906
[TPU] Switch Test to Non-Sliding Window (#15981) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-04-03 14:28:45 -07:00
iefgnoix
b6be6f8d1e
[TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) 
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
 2025-04-03 14:23:28 -07:00
bnellnm
15ba07ef25
[Minor] Fused experts refactor (#15914) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-04-03 10:19:38 -07:00
Liangfu Chen
d2b58ca203
[Neuron][kernel] Fuse kv cache into a single tensor (#15911) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-04-03 09:51:32 -07:00
Aleksandr Malyshev
e73ff24e31
[ROCM][KERNEL] Paged attention for V1 (#15720) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
 2025-04-02 19:48:00 -07:00
Hyesoo Yang
1b84eff03a
[V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
 2025-04-02 17:18:08 -07:00
Matthias Matt
cefb9e5a28
[Frontend] Implement Tool Calling with tool_choice='required' (#13483) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Matt, Matthias <matthias.matt@tuwien.ac.at>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
 2025-04-02 07:45:45 -07:00
Mark McLoughlin
98d7367b61
[Metrics] Hide deprecated metrics (#15458) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-02 07:37:19 -07:00
Chauncey
594a8b9030
[Bugfix] Fix the issue where the model name is empty string, causing no response with the model name. (#15938) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-02 06:33:52 -07:00
Russell Bryant
14e53ed11f
[V1] Fix json_object support with xgrammar (#15488) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-02 02:00:08 -07:00
Eric Tang
ddb94c2605
[core] Add tags parameter to wake_up() (#15500) 
Signed-off-by: Eric <erictang000@gmail.com>
 2025-04-02 01:59:27 -07:00
LukasBluebaum
90969fb39a
[Kernel] Add more dtype support for GGUF dequantization (#15879) 
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
 2025-04-02 01:58:48 -07:00
Jee Jee Li
4203926f10
[CI/Build] Further clean up LoRA tests (#15920) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-02 01:39:09 -07:00
Gerald
9ef98d527e
[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454) 
Signed-off-by: qscqesze <475517977@qq.com>
Co-authored-by: qingjun <qingjun@minimaxi.com>
Co-authored-by: qscqesze <475517977@qq.com>
 2025-04-01 16:23:55 -04:00
Mark McLoughlin
a79cc68b3a
[V1][Metrics] Initial speculative decoding metrics (#15151) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-01 10:45:04 -07:00
Roger Wang
7e3f7a4ee7
[CI] Disable flaky structure decoding test temporarily. (#15892) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-04-01 17:42:34 +00:00
Jennifer Zhao
38327cf454
[Model] Aya Vision (#15441) 
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-01 16:30:43 +00:00
Jee Jee Li
dfa82e2a3d
[CI/Build] Clean up LoRA tests (#15867) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-01 16:28:50 +00:00
bnellnm
e59ca942f5
Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-04-01 12:07:43 -04:00
wang.yuqi
085cbc4f9f
[New Model]: jinaai/jina-reranker-v2-base-multilingual (#15876) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-01 08:32:26 -07:00
Michael Goin
51d7c6a2b2
[Model] Support Mistral3 in the HF Transformers format (#15505) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-01 06:10:05 -07:00
Varun Sundar Rabindranath
79455cf421
[Misc] Enable V1 LoRA by default (#15320) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-04-01 16:53:56 +08:00
Wei Zeng
30d6a015e0
[Feature] specify model in config.yaml (#15798) 
Signed-off-by: weizeng <weizeng@roblox.com>
 2025-04-01 01:20:06 -07:00
Chen Zhang
3a5f0afcd2
[V1] Implement sliding window attention in kv_cache_manager (#14097) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-04-01 00:33:17 -07:00
Yan Ma
ff6473980d
[Bugfix][Model] fix mllama multi-image (#14883) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2025-03-31 22:53:37 -07:00
Harry Mellor
a76f547e11
Rename fallback model and refactor supported models section (#15829) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-31 22:49:41 -07:00
Ilya Markov
b7b7676d67
[Distributed] Add custom allreduce support for ROCM (#14125) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-03-31 22:49:12 -07:00
Mark McLoughlin
f98a4920f9
[V1][Core] Remove unused speculative config from scheduler (#15818) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-03-31 19:15:21 +00:00
Alexander Matveev
9a2160fa55
[V1] TPU CI - Add basic perf regression test (#15414) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-03-31 13:25:20 -04:00
shangmingc
239b7befdd
[V1][Spec Decode] Remove deprecated spec decode config params (#15466) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-31 09:19:35 -07:00
Cyrus Leung
09e974d483
[Bugfix] Check dimensions of multimodal embeddings in V1 (#15816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-31 09:01:35 -07:00
Harry Mellor
e5ef4fa99a
Upgrade transformers to v4.50.3 (#13905) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-31 08:59:37 -07:00
Alex Brooks
c2e7507ad4
[Bugfix] Fix Crashing When Loading Modules With Batchnorm Stats (#15813) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-03-31 13:23:53 +00:00
Naveassaf
3aa2b6a637
[Model] Update support for NemotronNAS models (#15008) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
 2025-03-31 20:35:14 +08:00
youkaichao
555aa21905
[V1] Fully Transparent Implementation of CPU Offloading (#15354) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-31 20:22:34 +08:00
Charlie Fu
e85829450d
[Feature][ROCm]Enable fusion pass for torch.compile on ROCm (#15050) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-03-31 04:42:18 -07:00
yihong
248e76c4df
fix: lint fix a ruff checkout syntax error (#15767) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-03-30 03:36:02 -07:00
Cyrus Leung
803d5c35f3
[V1] Override mm_counts for dummy data creation (#15703) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-30 03:20:42 -07:00
pansicheng
7fd8c0f85c
fix test_phi3v (#15321) 
Signed-off-by: pansicheng <sicheng.pan.chn@gmail.com>
 2025-03-30 02:01:34 -07:00
Julien Denize
6909a76201
[Bugfix] Fix Mistral guided generation using xgrammar (#15704) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-03-29 20:20:19 -07:00
Chauncey
045533716b
[CI] xgrammar structured output supports Enum. (#15757) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-03-29 20:20:02 -07:00
Roger Wang
c67abd614f
[V1] Support interleaved modality items (#15605) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-29 06:30:09 -07:00
TJian
4965ec42d2
[FEAT] [ROCm] Add AITER int8 scaled gemm kernel (#15433) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-29 03:33:56 -07:00
Russell Bryant
7a7992085b
[CI] Speed up V1 structured output tests (#15718) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-28 21:10:45 -07:00
Varun Sundar Rabindranath
1286211f57
[Bugfix] LoRA V1: add and fix entrypoints tests (#15715) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-28 21:10:41 -07:00