Statistics for topic llm-inference

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

本项目旨在分享大模型相关技术原理以及实战经验。

gpt4all: run open-source LLMs anywhere

The open-source serverless GPU container runtime.

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

The open-source serverless GPU container runtime.

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inferenc...

Sequence Parallel Attention for Long Context LLM Model Training and Inference

LLM in Godot

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

本项目旨在分享大模型相关技术原理以及实战经验。

gpt4all: run open-source LLMs anywhere

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

The open-source serverless GPU container runtime.

LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

LLM in Godot

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inferenc...

Sequence Parallel Attention for Long Context LLM Model Training and Inference

LLM in Godot

gpt4all: run open-source LLMs anywhere

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

本项目旨在分享大模型相关技术原理以及实战经验。

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

A collection of hand on notebook for LLMs practitioner

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.

Local LLM inference Library

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

Reference implementation of Mistral AI 7B v0.1 model.

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

本项目旨在分享大模型相关技术原理以及实战经验。

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

gpt4all: run open-source LLMs anywhere

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.

Reference implementation of Mistral AI 7B v0.1 model.

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalab...

LLMs as Copilots for Theorem Proving in Lean

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

irresponsible innovation. Try now at https://chat.dev/

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.