20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Misc] SpecDecodeWorker supports profiling (#9719)

Browse Source 
Signed-off-by: Abatom <abatom@163.com>
This commit is contained in:
科英 2024-10-27 12:18:03 +08:00 committed by GitHub
parent 6650e6a930
commit 67a6882da4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
vllm/spec_decode/spec_decode_worker.py
View File

@ -1038,6 +1038,14 @@ class SpecDecodeWorker(LoraNotSupportedWorkerBase):
"""
raise NotImplementedError
def start_profile(self):
if isinstance(self.scorer_worker, Worker):
self.scorer_worker.start_profile()
def stop_profile(self):
if isinstance(self.scorer_worker, Worker):
self.scorer_worker.stop_profile()
def split_num_cache_blocks_evenly(scorer_cache_block_size_bytes: int,
proposer_cache_block_size_bytes: int,