6 Commits

Author	SHA1	Message	Date
Joe Runde	6e0c9d6bd0	[Bugfix] Use heartbeats instead of health checks (#8583)	2024-09-24 20:37:38 -07:00
Simon Mo	3185fb0cca	Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (#8750)	2024-09-24 05:45:20 +00:00
Alexander Matveev	1a2aef3e59	Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335)	2024-09-23 15:38:04 -07:00
Cyrus Leung	0057894ef7	[Core] Rename PromptInputs and inputs(#8673)	2024-09-20 19:00:54 -07:00
Nick Hill	76515f303b	[Frontend] Use MQLLMEngine for embeddings models too (#8584)	2024-09-19 12:51:06 -04:00
Alexander Matveev	7c7714d856	[Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157) ... Co-authored-by: Nick Hill <nickhill@us.ibm.com> Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com> Co-authored-by: Simon Mo <simon.mo@hey.com>	2024-09-18 13:56:58 +00:00