20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[TPU][V1] Use language_model interface for getting text backbone in MM (#16410)

Browse Source 
Signed-off-by: NickLucche <nlucches@redhat.com>
This commit is contained in:
Nicolò Lucchesi 2025-04-10 19:32:04 +02:00 committed by GitHub
parent 83b824c8b4
commit c1b57855ec
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/v1/worker/tpu_model_runner.py
View File

@ -972,8 +972,7 @@ class TPUModelRunner:
def reset_dynamo_cache(self):
if self.is_multimodal_model:
assert hasattr(self.model, "language_model")
compiled_model = self.model.language_model.model
compiled_model = self.model.get_language_model().model
else:
compiled_model = self.model.model
if isinstance(compiled_model, TorchCompileWrapperWithCustomDispatcher):