1 result found Sort:

vllm-project
vllm vllm-project
2.9k
20.9k
apache-2.0
197
A high-throughput and memory-efficient inference and serving engine for LLMs
amd gpt llm cuda rocm llama mlops llmops pytorch trainium inference inferentia llm-serving transformer model-serving
Created 2023-02-09
1,568 commits to main branch, last one 22 hours ago