20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
4,808 Commits 1 Branch 0 Tags
Commit Graph

271 Commits

Author SHA1 Message Date
Harry Mellor
aba8d6ee00
[Doc] Move examples into categories (#11840) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-08 13:09:53 +00:00
Cyrus Leung
91445c7bc8
[Bugfix] Fix image input for Pixtral-HF (#11741) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 10:17:16 +08:00
Harry Mellor
5950f555a1
[Doc] Group examples into categories (#11782) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-08 09:20:12 +08:00
Liangfu Chen
898cdf033e
[CI] Fix neuron CI and run offline tests (#11779) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-01-06 21:36:10 -08:00
Aurick Qiao
e1a5c2f0a1
[Model] Whisper model implementation (#11280) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-01-03 16:39:19 +08:00
Nathan Azrak
68d37809b9
[Misc] Minimum requirements for SageMaker compatibility (#11576) 2025-01-02 15:59:25 -08:00
Roger Wang
e7c7c5e822
[V1][VLM] V1 support for selected single-image models. (#11632) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-12-31 21:17:22 +00:00
Chen Zhang
a60731247f
[Doc] Update mllama example based on official doc (#11567) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2024-12-28 00:31:10 +00:00
Isotr0py
b85a977822
[Doc] Add video example to openai client for multimodal (#11521) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-26 17:31:29 +00:00
Cyrus Leung
9edca6bf8f
[Frontend] Online Pooling API (#11457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 17:54:30 +08:00
Isotr0py
e24113a8fe
[Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 16:28:00 +00:00
Yanyi Liu
5aef49806d
[Feature] Add load generation config from model (#11164) 
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-12-19 10:50:38 +00:00
Cyrus Leung
6142ef0ada
[VLM] Merged multimodal processor for Qwen2-Audio (#11303) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 06:14:17 +00:00
Alexander Matveev
fdea8ec167
[V1] VLM - enable processor cache by default (#11305) 
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
 2024-12-18 18:54:46 -05:00
kYLe
66d4b16724
[Frontend] Add OpenAI API support for input_audio (#11027) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-16 22:09:58 -08:00
Varun Sundar Rabindranath
efbce85f4d
[misc] Layerwise profile updates (#10242) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-12-16 18:14:57 +00:00
Isotr0py
2ca830dbaa
[Doc] Reorder vision language examples in alphabet order (#11228) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-12-16 11:23:33 +00:00
Isotr0py
d927dbcd88
[Model] Refactor Ultravox to use merged input processor (#11198) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-16 10:09:53 +00:00
Cyrus Leung
b10609e6a1
[Misc] Clean up multi-modal processor (#11207) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-15 06:30:28 +00:00
Cyrus Leung
93abf23a64
[VLM] Fully dynamic prompt replacement in merged input processor (#11199) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 17:52:18 +00:00
Cyrus Leung
0920ab9131
[Doc] Reorganize online pooling APIs (#11172) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 00:22:22 +08:00
Cyrus Leung
eeec9e3390
[Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
Jani Monoses
7cd7409142
PaliGemma 2 support (#11142) 2024-12-13 07:40:07 +00:00
Roger Wang
4816d20aa4
[V1] Fix torch profiling for offline inference (#11125) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-12 15:51:53 +00:00
Alexander Matveev
4e11683368
[V1] VLM preprocessor hashing (#11020) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-12 00:55:30 +00:00
Cyrus Leung
8f10d5e393
[Misc] Split up pooling tasks (#10820) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 01:28:00 -08:00
Maxime Fournioux
fe2e10c71b
Add example of helm chart for vllm deployment on k8s (#9199) 
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
 2024-12-10 09:19:27 +00:00
Cyrus Leung
39e227c7ae
[Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 17:10:05 +00:00
Cyrus Leung
1c768fe537
[Doc] Explicitly state that InternVL 2.5 is supported (#10978) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 16:58:02 +00:00
Travis Johnson
39c89e71a8
[Misc] Update llama 3.2 template to support system prompt with images (#10901) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-12-05 05:54:06 +00:00
Kuntai Du
0590ec3fd9
[Core] Implement disagg prefill by StatelessProcessGroup (#10502) 
This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>
 2024-12-01 19:01:00 -06:00
Cyrus Leung
d2f058e76c
[Misc] Rename embedding classes to pooling (#10801) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 14:36:51 +08:00
Sanket Kale
a6760f6456
[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) 
Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
 2024-11-25 18:32:39 -08:00
zhou fan
b1d920531f
[Model]: Add support for Aria model (#10514) 
Signed-off-by: xffxff <1247714429@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-11-25 18:10:55 +00:00
Maximilien de Bayser
214efc2c3c
Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
Russell Bryant
ebda51968b
[Core] Fix broken log configuration (#10458) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-23 10:23:51 +08:00
Travis Johnson
9195dbdbca
[Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-11-23 10:17:38 +08:00
Woosuk Kwon
46fe9b46d8
[Minor] Revert change in offline inference example (#10545) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-11-21 21:28:16 +00:00
Maximilien de Bayser
a324d3a1a7
Change granite chat template to keep json list formatting for tool calls (#10452) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
 2024-11-19 18:16:54 -07:00
ismael-dm
31894a2155
[Doc] Add documentation for Structured Outputs (#9943) 
Signed-off-by: ismael-dm <ismaeldm99@gmail.com>
 2024-11-18 09:52:12 -08:00
wchen61
d1557e66d3
[Misc] Enhance offline_inference to support user-configurable paramet… (#10392) 
Signed-off-by: wchen61 <wchen61@foxmail.com>
 2024-11-17 11:32:40 +00:00
Mike Depinet
f67ce05d0b
[Frontend] Pythonic tool parser (#9859) 
Signed-off-by: Mike Depinet <mike@fixie.ai>
 2024-11-14 04:14:34 +00:00
Austin Veselka
1b886aa104
[Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
harrywu
874f551b36
[Metrics] add more metrics (#4464) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-12 00:17:38 +08:00
Isotr0py
1ff4aed5bd
[Model] Expose size to Idefics3 as mm_processor_kwargs (#10146) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-08 09:56:58 +00:00
Russell Bryant
3be5b26a76
[CI/Build] Add shell script linting using shellcheck (#7925) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-07 18:17:29 +00:00
Maximilien de Bayser
ae62fd17c0
[Frontend] Tool calling parser for Granite 3.0 models (#9027) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 07:09:02 -08:00
Flávia Béo
aa9078fa03
Adds method to read the pooling types from model's files (#9506) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 08:42:40 +00:00
Jee Jee Li
a5bba7d234
[Model] Add Idefics3 support (#9767) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
 2024-11-06 11:41:17 +00:00
shanshan wang
54597724f4
[Model] Add support for H2OVL-Mississippi models (#9747) 
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-04 00:15:36 +00:00