Woosuk Kwon
|
7a7929abe8
|
Implement preemption via recomputation & Refactor scheduling logic (#12)
|
2023-03-30 14:51:46 -07:00 |
|
Zhuohan Li
|
721fa3df15
|
FastAPI-based working frontend (#10)
|
2023-03-29 14:48:56 +08:00 |
|
Woosuk Kwon
|
cfae35b861
|
Add miscellaneous updates (#8)
|
2023-03-13 13:48:38 -07:00 |
|
Woosuk Kwon
|
e9d3f2ff77
|
Add memory analyzer & utomatically configure KV cache size (#6)
|
2023-03-11 23:23:14 -08:00 |
|
Woosuk Kwon
|
1a7eb7da61
|
Support beam search & parallel generation (#7)
|
2023-03-10 09:58:21 -08:00 |
|
Woosuk Kwon
|
1132fae0ca
|
Add Frontend
|
2023-02-24 11:46:43 +00:00 |
|
Woosuk Kwon
|
ef6098ec51
|
Merge pre_step and step
|
2023-02-24 10:36:08 +00:00 |
|
Woosuk Kwon
|
53f70e7334
|
Reduce the number of states in scheduler
|
2023-02-24 10:22:39 +00:00 |
|
Woosuk Kwon
|
afdbe5d373
|
[WIP] Add server script
|
2023-02-24 01:33:37 +00:00 |
|
Woosuk Kwon
|
fdd0f2f472
|
Minor
|
2023-02-23 20:23:47 +00:00 |
|
Woosuk Kwon
|
331fa0b042
|
Implement scheduler.step & Add a threshold for batch size
|
2023-02-23 07:54:20 +00:00 |
|
Woosuk Kwon
|
7e5f604e68
|
Fix bugs in scheduler
|
2023-02-14 02:25:32 +00:00 |
|
Woosuk Kwon
|
531e1c74e8
|
Fix typo
|
2023-02-13 18:51:33 +00:00 |
|
Woosuk Kwon
|
d3d317665b
|
Fix scheduler
|
2023-02-13 09:37:00 +00:00 |
|
Woosuk Kwon
|
5a309bb588
|
Add scheduler
|
2023-02-13 02:39:53 +00:00 |
|