Search Results - RepositoryStats

73

1.2k

apache-2.0

25

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

llm cuda mlsys triton attention quantization video-generation efficient-attention inference-acceleration

Created 2024-10-03

84 commits to main branch, last one 2 days ago

TeaCache ali-vilab

20

547

apache-2.0

7

Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

latte cogvideox open-sora hunyuan-video open-sora-plan diffusion-models video-generation inference-acceleration

Created 2024-11-27

97 commits to main branch, last one 5 days ago

SpargeAttn thu-ml

16

324

apache-2.0

4

SpargeAttention: A training-free sparse attention that can accelerate any model inference.

llm mlsys ai-infra attention quantization sageattention sparse-attention vision-transformer inference-acceleration

Created 2025-02-25

26 commits to main branch, last one 7 days ago

AsyncDiff czg1225

12

191

apache-2.0

3

[NeurIPS 2024] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising

text-to-image text-to-video training-free diffusion-models stable-diffusion efficient-inference distributed-computing inference-acceleration

Created 2024-05-31

64 commits to main branch, last one 28 days ago

nos autonomi-ai

12

138

apache-2.0

1

⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.

inference generative-ai llm-inference computer-vision machine-learning inference-acceleration

Created 2023-04-16

358 commits to main branch, last one 9 months ago

Q-LLM dvlab-research

2

45

unknown

0

This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"

long-context fast-inference kv-cache-compression large-language-models inference-acceleration

Created 2024-06-11

8 commits to master branch, last one 8 months ago