20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Abort when coroutine is cancelled (#1020)

Browse Source
This commit is contained in:
Jerry Yang 2023-09-15 08:40:18 +08:00 committed by GitHub
parent 64ca424e75
commit b9fe4616f9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/engine/async_llm_engine.py
View File

@ -395,8 +395,9 @@ class AsyncLLMEngine:
async for request_output in stream: async for request_output in stream:
yield request_output yield request_output
except Exception as e: except (Exception, asyncio.CancelledError) as e:
# If there is an exception, abort the request. # If there is an exception or coroutine is cancelled, abort the
# request.
self._abort(request_id) self._abort(request_id)
raise e raise e