4 results found Sort:

lucidrains
ring-attention-pytorch lucidrains
22
397
mit
8
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
long-context attention-mechanism efficient-attention distributed-attention
Created 2024-02-14
196 commits to main branch, last one 2 months ago
lucidrains
CoLT5-attention lucidrains
12
218
mit
8
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
routing deep-learning efficient-attention attention-mechanisms artificial-intelligence
Created 2023-03-20
103 commits to main branch, last one 14 days ago
jlamprou
Infini-Attention jlamprou
5
49
unknown
3
Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval
llm qwen infinite attention transformer efficient-attention
Created 2024-04-13
32 commits to main branch, last one about a month ago
davidsvy
cosformer-pytorch davidsvy
8
43
mit
2
Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
iclr pytorch iclr2022 transformer deep-learning neural-network attention-mechanism efficient-attention artificial-intelligence
Created 2021-10-20
20 commits to main branch, last one 2 years ago