Search Results - RepositoryStats

81

806

gpl-3.0

10

🎉CUDA 笔记 / 大模型手撕CUDA / C++笔记，更新随缘: flash_attn、sgemm、sgemv、warp reduce、block reduce、dot product、elementwise、softmax、layernorm、rmsnorm、hist etc.

cuda gemm gemv rmsnorm softmax layernorm elementwise warp-reduce block-reduce cuda-kernels flash-attention cuda-programming flash-attention-2

Created 2022-12-17

110 commits to main branch, last one a day ago

11

187

bsd-3-clause

4

Root Mean Square Layer Normalization

rmsnorm layernorm

Created 2019-09-24

7 commits to master branch, last one about a year ago

3

44

unknown

6

Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)

attention layernorm transformers layer-normalization

Created 2023-05-03

3 commits to main branch, last one 11 months ago