Search Results - RepositoryStats

2 results found Sort:

unknown

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

llm kv-cache cache-eviction cache-management

Created 2024-01-14

54 commits to main branch, last one 9 months ago

bsd-2-clause

SIEVE cache - simpler than LRU

lru-cache sieve-cache cache-eviction

Created 2024-01-04

6 commits to main branch, last one 10 months ago