4 results found Sort:

Tebmer
Awesome-Knowledge-Distillation-of-LLMs Tebmer
40
701
unknown
11
This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...
kd llm survey feedback alignment compression multi-modal self-training data-synthesis data-augmentation self-distillation large-language-model instruction-following supervised-finetuning knowledge-distillation
Created 2024-02-08
52 commits to main branch, last one 2 months ago
luanyunteng
pytorch-be-your-own-teacher luanyunteng
27
165
unknown
4
A pytorch implementation of paper 'Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation', https://arxiv.org/abs/1905.08094
self-distillation knowledge-distillation
Created 2020-02-26
9 commits to master branch, last one 2 years ago
sail-sg
sdft sail-sg
4
105
unknown
6
[ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
language-model self-distillation supervised-finetuning
Created 2024-02-20
25 commits to main branch, last one 3 months ago
youngkyunJang
Deep-Hash-Distillation youngkyunJang
6
45
unknown
2
Deep Hash Distillation for Image Retrieval - ECCV 2022
hashing pytorch deep-learning image-retrieval self-distillation vision-transformer representation-learning
Created 2021-11-22
72 commits to main branch, last one 5 months ago