1 result found Sort:

vectorch-ai
ScaleLLM vectorch-ai
30
397
apache-2.0
17
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
631 commits to main branch, last one about a month ago