Search Results - RepositoryStats

35

709

apache-2.0

18

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

llm cuda triton attention quantization video-generation inference-acceleration

Created 2024-10-03

64 commits to main branch, last one 22 hours ago

AsyncDiff czg1225

11

169

apache-2.0

4

[NeurIPS 2024] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising

text-to-image text-to-video training-free diffusion-models stable-diffusion efficient-inference distributed-computing inference-acceleration

Created 2024-05-31

63 commits to main branch, last one 2 months ago

nos autonomi-ai

11

132

apache-2.0

1

⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.

inference generative-ai llm-inference computer-vision machine-learning inference-acceleration

Created 2023-04-16

358 commits to main branch, last one 6 months ago

Q-LLM dvlab-research

1

42

unknown

1

This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"

long-context fast-inference kv-cache-compression large-language-models inference-acceleration

Created 2024-06-11

8 commits to master branch, last one 5 months ago