Search Results - RepositoryStats

InternLM InternLM

400

5.5k

apache-2.0

49

Official release of InternLM2 7B and 20B base and chat models. 200K context support

gpt llm rlhf chatbot chinese long-context fine-tuning-llm flash-attention pretrained-models large-language-model

Created 2023-07-06

220 commits to main branch, last one 11 days ago

LongLoRA dvlab-research

257

2.5k

apache-2.0

13

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

llm lora long-context fine-tuning-llm large-language-models

Created 2023-09-21

171 commits to main branch, last one 5 months ago

MEGABYTE-pytorch lucidrains

51

596

mit

10

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

long-context transformers deep-learning attention-mechanisms learned-tokenization artificial-intelligence

Created 2023-05-15

28 commits to main branch, last one about a month ago

ringattention lhao499

43

558

apache-2.0

5

Transformers with Arbitrarily Large Context

long-context transformers memory-efficient large-language-models

Created 2023-06-01

47 commits to main branch, last one 7 days ago

LongBench THUDM

34

530

mit

6

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

llm longtext benchmark long-context

Created 2023-07-29

59 commits to main branch, last one 4 months ago

ring-attention-pytorch lucidrains

22

397

mit

8

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

long-context attention-mechanism efficient-attention distributed-attention

Created 2024-02-14

196 commits to main branch, last one 2 months ago

recurrent-memory-transformer-pytorch lucidrains

15

386

mit

13

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

memory recurrence long-context transformers deep-learning attention-mechanisms artificial-intelligence

Created 2023-04-24

67 commits to main branch, last one 4 months ago

infini-transformer dingo-actual

20

243

mit

6

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

pytorch long-context transformers deep-learning infini-attention mixture-of-depths attention-mechanism

Created 2024-04-15

81 commits to main branch, last one about a month ago

InfLLM thunlp

20

230

mit

15

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

llm long-context training-free large-language-models

Created 2024-03-03

41 commits to main branch, last one 2 months ago

InfiniteBench OpenBMB

15

206

mit

9

Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718

benchmark long-context large-language-models

Created 2023-11-22

68 commits to main branch, last one 16 days ago

LongAlign THUDM

9

149

apache-2.0

8

LongAlign: A Recipe for Long Context Alignment Encompassing Data, Training, and Evaluation

llm longtext alignment long-context

Created 2024-01-27

45 commits to main branch, last one 2 months ago

LongQLoRA yangjianxin1

11

148

unknown

2

LongQLoRA: Extent Context Length of LLMs Efficiently

llm lora qlora longlora long-context

Created 2023-10-22

25 commits to master branch, last one 7 months ago

TriForce Infini-AI-Lab

12

136

unknown

1

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

llm inference efficiency acceleration long-context llm-inference speculative-decoding

Created 2024-04-04

15 commits to main branch, last one 3 days ago

LooGLE bigai-nlco

5

122

mit

3

ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models

llm acl2024 long-context large-language-models

Created 2023-11-02

25 commits to main branch, last one 4 months ago

Retrieval_Head nightdessert

10

116

unknown

2

open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality

long-context large-language-models

Created 2024-04-18

19 commits to main branch, last one 10 days ago

lloco jeffreysijuntan

9

98

mit

3

The official repo for "LLoCo: Learning Long Contexts Offline"

llm pytorch finetune long-context context-compression

Created 2024-04-12

4 commits to main branch, last one 14 days ago

infini-transformer-pytorch lucidrains

0

94

mit

3

Implementation of Infini-Transformer in Pytorch

memory long-context transformers deep-learning attention-mechanism artificial-intelligence

Created 2024-05-01

42 commits to main branch, last one about a month ago

LM-Infinite Glaciohound

10

89

mit

4

Implementation of paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

long-context language-model model-diagnostics

Created 2023-05-31

16 commits to main branch, last one 2 months ago

perceiver-ar-pytorch lucidrains

4

86

mit

4

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

transformer long-context deep-learning attention-mechanism artficial-intelligence

Created 2022-06-18

20 commits to main branch, last one about a year ago

awesome-llm-plaza metame-ai

5

82

mit

6

awesome llm plaza: daily tracking all sorts of awesome topics of llm, e.g. llm for coding, robotics, reasoning, multimod etc.

llm awesome awesome-rlhf long-context llm-application awesome-code-llm awesome-llm-plaza awesome-llm-agents awesome-llm-prompt awesome-llm-security awesome-multimod-llm awesome-robotics-llm awesome-llm-reasoning

Created 2023-11-12

165 commits to main branch, last one 21 hours ago

Counting-Stars nick7nlp

2

61

unknown

3

Counting-Stars (★)

long-context evaluation-metrics large-language-model

Created 2024-03-13

179 commits to main branch, last one a day ago

flash-genomics-model lucidrains

5

52

mit

6

My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other hierarchical methods)

genomics long-context transformers deep-learning attention-mechanisms artificial-intelligence

Created 2023-05-18

10 commits to main branch, last one 12 months ago