4 results found Sort:

DefTruth
Awesome-LLM-Inference DefTruth
120
1.6k
gpl-3.0
62
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
llm llms sora vllm deepseek open-sora awesome-llm tensorrt-llm llm-inference streaming-llm flash-attention paged-attention flash-attention-2
Created 2023-08-27
279 commits to main branch, last one 2 days ago
inferflow
inferflow inferflow
21
227
mit
7
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
moe qwen bloom gemma phi-2 falcon llama2 m2m100 minicpm mistral mixtral deepseek internlm llamacpp baichuan2 llm-inference mixture-of-experts model-quantization multi-gpu-inference
Created 2023-12-26
116 commits to main branch, last one 2 months ago
LLM-Red-Team
deepseek-free-api LLM-Red-Team
41
137
gpl-3.0
3
🚀 DeepSeek-V2大模型逆向API白嫖测试【特长：GPT4平替】，支持高速流式输出、多轮对话，零配置部署，多路token支持。
llm chatbot chat-api deepseek chatgpt-api deepseek-chat
Created 2024-05-07
18 commits to master branch, last one 24 days ago
open-kf
rag-gpt open-kf
13
128
apache-2.0
6
RAG-GPT, leveraging LLM and RAG technology, learns from user-customized knowledge bases to provide contextually relevant answers for a wide range of queries, ensuring rapid and accurate information re...
gpt llm rag gpt-4o nextjs ollama openai python deepseek zhipu-api
Created 2024-04-09
162 commits to main branch, last one 3 days ago