2 results found Sort:

4.5k
29.7k
apache-2.0
242
A high-throughput and memory-efficient inference and serving engine for LLMs
Created 2023-02-09
3,290 commits to main branch, last one 7 hours ago
Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
Created 2024-01-09
1,129 commits to main branch, last one 9 hours ago