11 results found Sort:

HDT3213
godis HDT3213
575
3.6k
gpl-3.0
37
A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群
go godis redis golang cluster kv-cache redis-server redis-cluster
Created 2019-06-01
261 commits to master branch, last one 5 months ago
Zefan-Cai
KVCache-Factory Zefan-Cai
147
1.1k
mit
96
Unified KV Cache Compression Methods for Auto-Regressive Models
llm kv-cache kv-cache-compression
Created 2024-06-05
135 commits to main branch, last one 19 days ago
FMInference
H2O FMInference
51
413
unknown
5
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
gpt-3 kv-cache sparsity heavy-hitters high-throughput large-language-models
Created 2023-06-12
41 commits to main branch, last one 6 months ago
harleyszhang
llm_note harleyszhang
28
319
unknown
5
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes
llm vllm kv-cache llm-inference triton-kernels cuda-programming transformer-models
Created 2024-09-18
177 commits to main branch, last one 3 days ago
NVIDIA
kvpress NVIDIA
14
280
apache-2.0
6
LLM KV cache compression made easy
llm python pytorch kv-cache inference long-context transformers kv-cache-compression large-language-models
Created 2024-11-06
16 commits to main branch, last one 11 days ago
Zefan-Cai
Awesome-LLM-KV-Cache Zefan-Cai
7
159
gpl-3.0
4
Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
llm kv-cache kv-cache-compression kv-cache-quantization
Created 2024-07-24
17 commits to main branch, last one 23 days ago
itsnamgyu
block-transformer itsnamgyu
7
147
mit
6
Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
llm kv-cache llm-inference llm-architecture kv-cache-compression
Created 2024-05-29
33 commits to main branch, last one 12 days ago
kddubey
cappr kddubey
3
71
apache-2.0
2
Completion After Prompt Probability. Make your LLM make a choice
kv-cache llamacpp zero-shot huggingface probability llm-inference prompt-engineering text-classification
Created 2023-02-22
448 commits to main branch, last one about a month ago
DRSY
EasyKV DRSY
4
60
unknown
3
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
llm kv-cache cache-eviction cache-management
Created 2024-01-14
54 commits to main branch, last one 10 months ago
aju22
LLaMA2 aju22
9
55
unknown
3
This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture ...
gpt llm rope llama llama2 kv-cache rms-norm attention transformer natural-language-processing
Created 2023-10-01
5 commits to main branch, last one about a year ago
hkproj
pytorch-llama-notes hkproj
5
50
unknown
7
Notes about LLaMA 2 model
llama2 rmsprop kv-cache study-notes rotary-position-encoding attention-is-all-you-need
Created 2023-08-21
4 commits to main branch, last one about a year ago