20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Doc] Fix typo (#11666)

Browse Source 
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
This commit is contained in:
Kazuhiro Serizawa 2025-01-01 17:10:10 +09:00 committed by GitHub
parent f962f426bc
commit 6d70198b17
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/model_executor/layers/rejection_sampler.py
View File

@ -39,7 +39,7 @@ class RejectionSampler(SpecDecodeStochasticBaseSampler):
strict_mode: Whether or not to perform shape/device/dtype checks strict_mode: Whether or not to perform shape/device/dtype checks
during sampling. This catches correctness issues but adds during sampling. This catches correctness issues but adds
nontrivial latency. nontrivial latency.
use_falshinfer: We will use this parameter to determine whether use_flashinfer: We will use this parameter to determine whether
to use the FlashInfer rejection sampling kernel or not. If it's to use the FlashInfer rejection sampling kernel or not. If it's
None, we will use the default value from the environment variable. None, we will use the default value from the environment variable.
This parameter is only used for testing purposes. This parameter is only used for testing purposes.

2
vllm/v1/sample/ops/topk_topp_sampler.py
View File

@ -44,7 +44,7 @@ class TopKTopPSampler(nn.Module):
logger.warning( logger.warning(
"FlashInfer is not available. Falling back to the PyTorch-" "FlashInfer is not available. Falling back to the PyTorch-"
"native implementation of top-p & top-k sampling. For the " "native implementation of top-p & top-k sampling. For the "
"best performance, please install FalshInfer.") "best performance, please install FlashInfer.")
self.forward = self.forward_native self.forward = self.forward_native
else: else:
self.forward = self.forward_native self.forward = self.forward_native