Search Results - RepositoryStats

ring-attention-pytorch lucidrains

29

484

mit

11

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

long-context attention-mechanism efficient-attention distributed-attention

Created 2024-02-14

221 commits to main branch, last one about a month ago

CoLT5-attention lucidrains

13

225

mit

8

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

routing deep-learning efficient-attention attention-mechanisms artificial-intelligence

Created 2023-03-20

104 commits to main branch, last one 3 months ago

Infini-Attention jlamprou

5

68

unknown

3

Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval

llm qwen infinite attention transformer efficient-attention

Created 2024-04-13

32 commits to main branch, last one 7 months ago

cosformer-pytorch davidsvy

8

44

mit

2

Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".

iclr pytorch iclr2022 transformer deep-learning neural-network attention-mechanism efficient-attention artificial-intelligence

Created 2021-10-20

20 commits to main branch, last one 3 years ago

CascadedGaze Ascend-Research

1

41

mit

3

The official PyTorch implementation for CascadedGaze: Efficiency in Global Context Extraction for Image Restoration, TMLR'24.

denoising deblurring efficiency transformer image-restoration efficient-attention

Created 2024-05-07

8 commits to main branch, last one 2 months ago