4 results found Sort:

Tebmer
Awesome-Knowledge-Distillation-of-LLMs Tebmer
22
366
unknown
5
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...
kd llm survey feedback alignment compression multi-modal self-training data-synthesis data-augmentation self-distillation large-language-model instruction-following supervised-finetuning knowledge-distillation
Created 2024-02-08
47 commits to main branch, last one 2 months ago
luanyunteng
pytorch-be-your-own-teacher luanyunteng
27
148
unknown
4
A pytorch implementation of paper 'Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation', https://arxiv.org/abs/1905.08094
self-distillation knowledge-distillation
Created 2020-02-26
9 commits to master branch, last one 2 years ago
sail-sg
sdft sail-sg
4
56
unknown
6
[ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
language-model self-distillation supervised-finetuning
Created 2024-02-20
17 commits to main branch, last one about a month ago
youngkyunJang
Deep-Hash-Distillation youngkyunJang
6
43
unknown
2
Deep Hash Distillation for Image Retrieval - ECCV 2022
hashing pytorch deep-learning image-retrieval self-distillation vision-transformer representation-learning
Created 2021-11-22
70 commits to main branch, last one about a month ago