Search Results - RepositoryStats

2 results found Sort:

411

7.9k

mit

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

llm llama falcon bamboo-7b llm-inference local-inference large-language-models

Created 2023-12-15

1,584 commits to main branch, last one 2 months ago

170

apache-2.0

Fast Inference of MoE Models with CPU-GPU Orchestration

llm mixtral-8x7b llm-inference local-inference mixture-of-experts

Created 2024-02-05

49 commits to main branch, last one 6 months ago