Wang, Yi
|
8a93a598d9
|
fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524)
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
|
2024-11-21 11:15:36 +00:00 |
|
Cyrus Leung
|
f4c2187e29
|
[Misc] Fix typo in #5895 (#10145)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-08 09:07:01 +00:00 |
|
DearPlanet
|
ad39bd640c
|
[Bugfix] Add error handling when server cannot respond any valid tokens (#5895)
|
2024-11-08 04:58:37 +00:00 |
|
Aaron Pham
|
21063c11c7
|
[CI/Build] drop support for Python 3.8 EOL (#8464)
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
|
2024-11-06 07:11:55 +00:00 |
|
Guillaume Calmettes
|
abbfb6134d
|
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
|
2024-10-30 18:15:56 -07:00 |
|
youkaichao
|
18b296fdb2
|
[core] remove beam search from the core (#9105)
|
2024-10-07 05:47:04 +00:00 |
|
Kuntai Du
|
fbb74420e7
|
[CI] Update performance benchmark: upgrade trt-llm to r24.07, and add SGLang (#7412)
|
2024-10-04 14:01:44 -07:00 |
|
Isotr0py
|
1b6de8352b
|
[Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495)
|
2024-09-17 07:34:27 +00:00 |
|
afeldman-nm
|
e5cab71531
|
[Frontend] Add --logprobs argument to benchmark_serving.py (#8191)
|
2024-09-06 09:01:14 -07:00 |
|
William Lin
|
dd53c4b023
|
[misc] Add Torch profiler support (#7451)
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-08-21 15:39:26 -07:00 |
|
Roger Wang
|
70d268a399
|
[Bugfix] Fix ITL recording in serving benchmark (#7372)
|
2024-08-09 10:00:00 -07:00 |
|
Roger Wang
|
c4774eb841
|
[Bugfix] Fix snapshot download in serving benchmark (#6318)
|
2024-07-11 07:04:05 +00:00 |
|
James Whedbee
|
e373853e12
|
[Frontend] Relax api url assertion for openai benchmarking (#6046)
|
2024-07-01 23:39:10 +00:00 |
|
zhyncs
|
bb60326836
|
[Misc] update benchmark backend for scalellm (#6018)
|
2024-07-01 10:20:33 -07:00 |
|
mcalman
|
c4bca740e8
|
[Bugfix] fix missing last itl in openai completions benchmark (#5926)
|
2024-06-29 10:34:42 +08:00 |
|
zhyncs
|
1f12122b17
|
[Misc] use AutoTokenizer for benchmark serving when vLLM not installed (#5588)
|
2024-06-17 09:40:35 -07:00 |
|
Wang, Yi
|
88407532e7
|
[Bugfix]if the content is started with ":"(response of ping), client should i… (#5303)
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-06-12 20:16:41 -07:00 |
|
Roger Wang
|
f17a1a8f96
|
[Misc] Make Serving Benchmark More User-friendly (#5044)
|
2024-05-25 17:28:16 +00:00 |
|
Elinx
|
fe3b5bbc23
|
[Bugfix] fix output parsing error for trtllm backend (#4137)
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-04-17 11:07:23 +00:00 |
|
Michael Feil
|
c2b4a1bce9
|
[Doc] Add typing hints / mypy types cleanup (#3816)
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-04-11 17:17:21 -07:00 |
|
Roger Wang
|
ccb58b23e6
|
[Misc] Fix Benchmark TTFT Calculation for Chat Completions (#3768)
|
2024-04-01 15:24:30 -07:00 |
|
Roger Wang
|
45b6ef6513
|
feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277)
|
2024-03-27 13:39:26 -07:00 |
|
Simon Mo
|
8e67598aa6
|
[Misc] fix line length for entire codebase (#3444)
|
2024-03-16 00:36:29 -07:00 |
|
Ronen Schaffer
|
14e3f9a1b2
|
Replace lstrip() with removeprefix() to fix Ruff linter warning (#2958)
|
2024-03-15 21:01:30 -07:00 |
|
Chen Wang
|
9a4548bae7
|
Fix the openai benchmarking requests to work with latest OpenAI apis (#2992)
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-03-04 15:51:56 -08:00 |
|
Roger Wang
|
a4211a4dc3
|
Serving Benchmark Refactoring (#2433)
|
2024-02-12 22:53:00 -08:00 |
|