Search Results - RepositoryStats

20

201

apache-2.0

6

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

fp8 flux pytorch diffusion quantization fast-inference

Created 2024-08-05

64 commits to main branch, last one 25 days ago

BigLittleDecoder kssteven418

10

85

apache-2.0

6

[NeurIPS'23] Speculative Decoding with Big Little Decoder

llm decoding fast-inference efficient-inference speculative-decoding speculative-execution

Created 2023-02-10

11,217 commits to main branch, last one 9 months ago

Q-LLM dvlab-research

1

38

unknown

1

This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"

long-context fast-inference kv-cache-compression large-language-models inference-acceleration

Created 2024-06-11

8 commits to master branch, last one 3 months ago