1 result found Sort:

vectorch-ai
ScaleLLM vectorch-ai
34
430
apache-2.0
16
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
703 commits to main branch, last one 14 hours ago