1 result found Sort:

vectorch-ai
ScaleLLM vectorch-ai
30
405
apache-2.0
16
A high-performance inference system for large language models, designed for production environments.
gpu llm cuda llama model llama3 serving inference efficiency production performance speculative transformer llm-inference
Created 2023-07-24
670 commits to main branch, last one 3 days ago