1 result found Sort:

vllm-project
vllm vllm-project
3.1k
21.8k
apache-2.0
196
A high-throughput and memory-efficient inference and serving engine for LLMs
amd gpt llm tpu xpu cuda rocm llama mlops llmops pytorch trainium inference inferentia llm-serving transformer model-serving
Created 2023-02-09
1,748 commits to main branch, last one 15 hours ago