5 results found Sort:

DefTruth
Awesome-LLM-Inference DefTruth
193
2.8k
gpl-3.0
91
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
llm llms sora vllm deepseek open-sora awesome-llm tensorrt-llm llm-inference flash-attention paged-attention flash-attention-2 flash-attention-3
Created 2023-08-27
405 commits to main branch, last one 7 days ago
DefTruth
CUDA-Learn-Notes DefTruth
152
1.4k
gpl-3.0
13
🎉 Modern CUDA Learn Notes with PyTorch: CUDA Cores, Tensor Cores, fp32/tf32, fp16/bf16, fp8/int8, flash_attn, rope, sgemm, hgemm, sgemv, warp/block reduce, elementwise, softmax, layernorm, rmsnorm.
cuda gemm gemv triton pytorch rmsnorm softmax layernorm elementwise warp-reduce block-reduce flash-attention cuda-programming flash-attention-2 flash-attention-3
Created 2022-12-17
299 commits to main branch, last one 2 days ago
arihanv
Shush arihanv
31
187
mit
4
Shush is an app that deploys a WhisperV3 model with Flash Attention v2 on Modal and makes requests to it via a NextJS app
modal whisper shadcn-ui transcription machine-learning flash-attention-2 huggingface-transformers
Created 2023-11-18
64 commits to main branch, last one 5 months ago
alexzhang13
flashattention2-custom-mask alexzhang13
6
72
apache-2.0
5
Triton implementation of FlashAttention2 that adds Custom Masks.
triton attention triton-lang cuda-kernels deep-learning flash-attention flash-attention-2 attention-mechanism
Created 2024-07-20
18 commits to main branch, last one 2 months ago
Bruce-Lee-LY
flash_attention_inference Bruce-Lee-LY
3
25
bsd-3-clause
1
Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
gpu llm mha cuda nvidia cutlass inference tensor-core flash-attention flash-attention-2 large-language-model multi-head-attention
Created 2023-08-16
1 commits to master branch, last one 2 months ago