[FrontEnd][Perf] merge_async_iterators fast-path for single-prompt requests (#15150)

Signed-off-by: Nick Hill <nhill@redhat.com>
2025-03-19 14:04:41 -07:00 · 2025-03-19 14:04:41 -07:00 · 22d33baca2
commit 22d33baca2
parent b0e96aaebb
1 changed files with 5 additions and 0 deletions
--- a/vllm/utils.py
+++ b/vllm/utils.py
@ -411,6 +411,11 @@ async def merge_async_iterators(
    When it yields, it yields a tuple (i, item) where i is the index of the
    iterator that yields the item.
    """
+    if len(iterators) == 1:
+        # Fast-path single iterator case.
+        async for item in iterators[0]:
+            yield 0, item
+        return

    loop = asyncio.get_running_loop()