11 results found Sort:

HDT3213
godis HDT3213
577
3.6k
gpl-3.0
35
A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群
go godis redis golang cluster kv-cache redis-server redis-cluster
Created 2019-06-01
264 commits to master branch, last one 17 days ago
Zefan-Cai
KVCache-Factory Zefan-Cai
151
854
mit
67
Unified KV Cache Compression Methods for Auto-Regressive Models
llm kv-cache kv-cache-compression
Created 2024-06-05
123 commits to main branch, last one 26 days ago
harleyszhang
llm_note harleyszhang
38
423
unknown
6
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
llm vllm kv-cache llm-inference triton-kernels cuda-programming transformer-models
Created 2024-09-18
212 commits to main branch, last one 5 days ago
FMInference
H2O FMInference
51
419
unknown
5
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
gpt-3 kv-cache sparsity heavy-hitters high-throughput large-language-models
Created 2023-06-12
41 commits to main branch, last one 7 months ago
NVIDIA
kvpress NVIDIA
24
359
apache-2.0
10
LLM KV cache compression made easy
llm python pytorch kv-cache inference long-context transformers kv-cache-compression large-language-models
Created 2024-11-06
23 commits to main branch, last one 9 days ago
Zefan-Cai
Awesome-LLM-KV-Cache Zefan-Cai
10
198
gpl-3.0
4
Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
llm kv-cache kv-cache-compression kv-cache-quantization
Created 2024-07-24
17 commits to main branch, last one about a month ago
itsnamgyu
block-transformer itsnamgyu
7
149
mit
6
Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
llm kv-cache llm-inference llm-architecture kv-cache-compression
Created 2024-05-29
33 commits to main branch, last one about a month ago
kddubey
cappr kddubey
3
73
apache-2.0
1
Completion After Prompt Probability. Make your LLM make a choice
kv-cache llamacpp zero-shot huggingface probability llm-inference prompt-engineering text-classification
Created 2023-02-22
448 commits to main branch, last one 2 months ago
DRSY
EasyKV DRSY
4
60
unknown
3
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
llm kv-cache cache-eviction cache-management
Created 2024-01-14
54 commits to main branch, last one 11 months ago
aju22
LLaMA2 aju22
9
59
unknown
3
This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture ...
gpt llm rope llama llama2 kv-cache rms-norm attention transformer natural-language-processing
Created 2023-10-01
5 commits to main branch, last one about a year ago
hkproj
pytorch-llama-notes hkproj
5
52
unknown
7
Notes about LLaMA 2 model
llama2 rmsprop kv-cache study-notes rotary-position-encoding attention-is-all-you-need
Created 2023-08-21
4 commits to main branch, last one about a year ago