2 results found Sort:

Bruce-Lee-LY
cuda_hgemv Bruce-Lee-LY
4
50
mit
4
Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.
gpu cuda gemm gemv hgemm hgemv cublas nvidia cuda-core tensor-core matrix-multiply
Created 2023-10-09
1 commits to master branch, last one 2 months ago
Bruce-Lee-LY
decoding_attention Bruce-Lee-LY
1
26
bsd-3-clause
2
Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.
gpu llm mha cuda nvidia cuda-core inference flashinfer flash-attention decoding-attention large-language-model multi-head-attention
Created 2024-08-14
1 commits to master branch, last one 18 days ago