1 result found Sort:

inferflow
inferflow inferflow
24
236
mit
8
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
moe qwen bloom gemma phi-2 falcon llama2 m2m100 minicpm mistral mixtral deepseek internlm llamacpp baichuan2 llm-inference mixture-of-experts model-quantization multi-gpu-inference
Created 2023-12-26
116 commits to main branch, last one 8 months ago