6 results found Sort:

HDT3213
godis HDT3213
553
3.4k
gpl-3.0
34
A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群
go godis redis golang cluster kv-cache redis-server redis-cluster
Created 2019-06-01
258 commits to master branch, last one about a month ago
FMInference
H2O FMInference
27
299
unknown
5
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
gpt-3 kv-cache sparsity heavy-hitters high-throughput large-language-models
Created 2023-06-12
38 commits to main branch, last one 2 months ago
kddubey
cappr kddubey
1
64
apache-2.0
2
Completion After Prompt Probability. Make your LLM make a choice
kv-cache llamacpp zero-shot huggingface probability llm-inference prompt-engineering text-classification
Created 2023-02-22
420 commits to main branch, last one 14 days ago
DRSY
EasyKV DRSY
4
53
unknown
3
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
llm kv-cache cache-eviction cache-management
Created 2024-01-14
54 commits to main branch, last one 4 months ago
hkproj
pytorch-llama-notes hkproj
4
35
unknown
6
Notes about LLaMA 2 model
llama2 rmsprop kv-cache study-notes rotary-position-encoding attention-is-all-you-need
Created 2023-08-21
4 commits to main branch, last one 9 months ago
aju22
LLaMA2 aju22
4
32
unknown
3
This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture ...
gpt llm rope llama llama2 kv-cache rms-norm attention transformer natural-language-processing
Created 2023-10-01
5 commits to main branch, last one 8 months ago