20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636)

Browse Source 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
This commit is contained in:
Bryan Lu 2025-04-16 19:47:26 -07:00 committed by GitHub
parent 3092375e27
commit 2cbd4d2999
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/config.py
View File

@ -2341,7 +2341,9 @@ class SpeculativeConfig:
) )
# Automatically detect the method # Automatically detect the method
if "eagle-" in self.draft_model_config.model.lower(): if self.method == 'eagle':
pass
elif "eagle-" in self.draft_model_config.model.lower():
self.method = "eagle" self.method = "eagle"
elif self.draft_model_config.hf_config.model_type == "medusa": elif self.draft_model_config.hf_config.model_type == "medusa":
self.method = "medusa" self.method = "medusa"