Search Results - RepositoryStats

InternLM InternLM

463

6.6k

apache-2.0

59

Official release of InternLM2.5 base and chat models. 1M context support

gpt llm rlhf chatbot chinese long-context fine-tuning-llm flash-attention pretrained-models large-language-model

Created 2023-07-06

235 commits to main branch, last one about a month ago

LongLoRA dvlab-research

278

2.6k

apache-2.0

12

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

llm lora long-context fine-tuning-llm large-language-models

Created 2023-09-21

173 commits to main branch, last one 4 months ago

LongWriter THUDM

144

1.5k

apache-2.0

17

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

llm long-text fine-tuning long-context

Created 2024-08-12

31 commits to main branch, last one about a month ago

LongBench THUDM

59

704

mit

7

LongBench v2 and LongBench (ACL 2024)

llm longtext benchmark long-context

Created 2023-07-29

76 commits to main branch, last one a day ago

ringattention haoliuhl

53

657

apache-2.0

7

Large Context Attention

long-context transformers memory-efficient large-language-models

Created 2023-06-01

54 commits to main branch, last one 4 months ago

MEGABYTE-pytorch lucidrains

53

631

mit

11

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

long-context transformers deep-learning attention-mechanisms learned-tokenization artificial-intelligence

Created 2023-05-15

35 commits to main branch, last one 3 months ago

ring-attention-pytorch lucidrains

29

484

mit

11

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

long-context attention-mechanism efficient-attention distributed-attention

Created 2024-02-14

221 commits to main branch, last one about a month ago

LongCite THUDM

33

430

apache-2.0

11

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

llm benchmark fine-tuning long-context citation-generation

Created 2024-08-31

54 commits to main branch, last one 5 days ago

recurrent-memory-transformer-pytorch lucidrains

15

397

mit

13

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

memory recurrence long-context transformers deep-learning attention-mechanisms artificial-intelligence

Created 2023-04-24

69 commits to main branch, last one about a month ago

InfLLM thunlp

29

312

mit

16

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

llm long-context training-free large-language-models

Created 2024-03-03

41 commits to main branch, last one 8 months ago

InfiniteBench OpenBMB

25

297

mit

9

Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718

benchmark long-context large-language-models

Created 2023-11-22

76 commits to main branch, last one 2 months ago

infini-transformer dingo-actual

23

286

mit

6

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

pytorch long-context transformers deep-learning infini-attention mixture-of-depths attention-mechanism

Created 2024-04-15

81 commits to main branch, last one 7 months ago

kvpress NVIDIA

14

272

apache-2.0

6

LLM KV cache compression made easy

llm python pytorch kv-cache inference long-context transformers kv-cache-compression large-language-models

Created 2024-11-06

16 commits to main branch, last one 2 days ago

TriForce Infini-AI-Lab

13

236

unknown

1

[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

llm inference efficiency acceleration long-context llm-inference speculative-decoding

Created 2024-04-04

17 commits to main branch, last one 3 months ago

LongAlign THUDM

15

230

apache-2.0

8

[EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs

llm longtext alignment long-context

Created 2024-01-27

46 commits to main branch, last one 5 days ago

LooGLE bigai-nlco

7

170

mit

3

ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models

llm acl2024 long-context large-language-models

Created 2023-11-02

26 commits to main branch, last one 2 months ago

awesome-llm-plaza metame-ai

13

168

mit

13

awesome llm plaza: daily tracking all sorts of awesome topics of llm, e.g. llm for coding, robotics, reasoning, multimod etc.

llm awesome awesome-rlhf long-context llm-application awesome-code-llm awesome-llm-plaza awesome-llm-agents awesome-llm-prompt awesome-llm-security awesome-multimod-llm awesome-robotics-llm awesome-llm-reasoning

Created 2023-11-12

220 commits to main branch, last one 6 days ago

Retrieval_Head nightdessert

14

168

unknown

2

open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality

long-context large-language-models

Created 2024-04-18

23 commits to main branch, last one 4 months ago

LongQLoRA yangjianxin1

15

160

unknown

2

LongQLoRA: Extent Context Length of LLMs Efficiently

llm lora qlora longlora long-context

Created 2023-10-22

25 commits to master branch, last one about a year ago

ShadowKV bytedance

6

140

apache-2.0

3

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

low-rank research cpu-offload long-context llm-inference high-throughput sparse-attention

Created 2024-10-22

13 commits to main branch, last one about a month ago

LM-Infinite Glaciohound

13

132

mit

4

Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

long-context language-model model-diagnostics

Created 2023-05-31

19 commits to main branch, last one 2 months ago

lloco jeffreysijuntan

9

113

mit

3

The official repo for "LLoCo: Learning Long Contexts Offline"

llm pytorch finetune long-context context-compression

Created 2024-04-12

4 commits to main branch, last one 6 months ago

infini-transformer-pytorch lucidrains

2

106

mit

3

Implementation of Infini-Transformer in Pytorch

memory long-context transformers deep-learning attention-mechanism artificial-intelligence

Created 2024-05-01

45 commits to main branch, last one 2 months ago

MM-NIAH OpenGVLab

6

106

unknown

1

[NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.

benchmark long-context vision-language-model multimodal-large-language-models

Created 2024-06-05

53 commits to main branch, last one 26 days ago

perceiver-ar-pytorch lucidrains

4

86

mit

4

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

transformer long-context deep-learning attention-mechanism artficial-intelligence

Created 2022-06-18

20 commits to main branch, last one about a year ago

LongRAG QingFei1

11

86

unknown

6

[EMNLP 2024] LongRAG: A Dual-perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering

llm rag long-context

Created 2024-10-03

3 commits to main branch, last one about a month ago

Counting-Stars nick7nlp

2

77

mit

3

Counting-Stars (★)

long-context evaluation-metrics large-language-model

Created 2024-03-13

192 commits to main branch, last one 3 months ago

VideoLLaMB bigai-nlco

1

56

unknown

3

Official Repository of VideoLLaMB: Long Video Understanding with Recurrent Memory Bridges

long-context video-language-pretraining video-language-understanding

Created 2024-08-19

13 commits to master branch, last one 3 months ago

flash-genomics-model lucidrains

5

52

mit

6

My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other hierarchical methods)

genomics long-context transformers deep-learning attention-mechanisms artificial-intelligence

Created 2023-05-18

10 commits to main branch, last one about a year ago

Ada-LEval open-compass

2

51

unknown

3

The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"

llm gpt4 long-context

Created 2024-04-10

53 commits to main branch, last one 8 months ago