7 results found Sort:

microsoft
LoRA microsoft
694
11.0k
mit
70
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
lora gpt-2 gpt-3 deberta pytorch roberta low-rank adaptation deep-learning language-model
Created 2021-06-18
46 commits to main branch, last one 4 days ago
lezcano
geotorch lezcano
35
663
mit
9
Constrained optimization toolkit for PyTorch
pytorch low-rank orthogonality manifold-optimization positive-semi-definite constrained-optimization invertible-neural-networks positive-definite-matrices
Created 2020-02-19
224 commits to master branch, last one 2 years ago
VITA-Group
Q-GaLore VITA-Group
13
181
apache-2.0
11
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
low-rank quantization large-language-models memory-efficient-learning
Created 2024-06-06
9 commits to main branch, last one 5 months ago
bytedance
ShadowKV bytedance
6
140
apache-2.0
3
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
low-rank research cpu-offload long-context llm-inference high-throughput sparse-attention
Created 2024-10-22
13 commits to main branch, last one about a month ago
juliagusak
model-compression-and-acceleration-progress juliagusak
20
105
unknown
8
Repository to track the progress in model compression and acceleration
pruning low-rank compression acceleration neural-network sparsification architecture-search tensor-decomposition knowledge-distillation
Created 2019-06-05
45 commits to master branch, last one 4 years ago
musco-ai
musco-pytorch musco-ai
16
71
bsd-3-clause
9
MUSCO: MUlti-Stage COmpression of neural networks
vbmf tucker pytorch low-rank truncated-svd cp-decomposition model-compression model-acceleration network-compression deep-neural-networks network-acceleration tensor-decomposition
Created 2019-05-08
59 commits to master branch, last one 3 years ago
Reason-Wang
flan-alpaca-lora Reason-Wang
8
47
unknown
3
This repository contains the code to train flan t5 with alpaca instructions and low rank adaptation.
t5 lora peft alpaca flan-t5 pytorch low-rank huggingface instruction-tuning
Created 2023-03-29
16 commits to main branch, last one about a year ago