1 result found Sort:

vectorch-ai
ScaleLLM vectorch-ai
23
311
apache-2.0
15
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
543 commits to main branch, last one 5 days ago