20231088/vllm - vllm - Luminance Code Repo

20231088/vllm

Fork 0

Commit Graph

Author	SHA1	Message	Date
Kuntai Du	81ede99ca4	[Core] Deprecating block manager v1 and make block manager v2 default (#8704 ) Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).	2024-10-17 11:38:15 -05:00
Nick Hill	2110557dab	[BugFix] Fix cuda graph for MLPSpeculator (#5875 ) Co-authored-by: Abhinav Goyal <abhinav.goyal@flipkart.com>	2024-06-27 04:12:10 +00:00
Joshua Rosenkranz	b12518d3cf	[Model] MLPSpeculator speculative decoding support (#4947 ) Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com> Co-authored-by: Nick Hill <nickhill@us.ibm.com> Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>	2024-06-20 20:23:12 -04:00

Author

SHA1

Message

Date

Kuntai Du

81ede99ca4

[Core] Deprecating block manager v1 and make block manager v2 default (#8704 )

Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).

2024-10-17 11:38:15 -05:00

Nick Hill

2110557dab

[BugFix] Fix cuda graph for MLPSpeculator (#5875 )

Co-authored-by: Abhinav Goyal <abhinav.goyal@flipkart.com>

2024-06-27 04:12:10 +00:00

Joshua Rosenkranz

b12518d3cf

[Model] MLPSpeculator speculative decoding support (#4947 )

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

2024-06-20 20:23:12 -04:00

3 Commits