Trending repositories for topic knowledge-distillation

A curated list for Efficient Large Language Models

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

Collection of AWESOME vision-language models for vision tasks

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

OpenMMLab Model Compression Toolbox and Benchmark.

Awesome Knowledge Distillation

A treasure chest for visual classification and recognition powered by PaddlePaddle

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

A curated list of awesome knowledge distillation papers and codes for object detection.

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

A curated list of awesome knowledge distillation papers and codes for object detection.

A curated list for Efficient Large Language Models

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

Collection of AWESOME vision-language models for vision tasks

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

OpenMMLab Model Compression Toolbox and Benchmark.

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Awesome Knowledge Distillation

A treasure chest for visual classification and recognition powered by PaddlePaddle

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Awesome Knowledge Distillation

OpenMMLab Model Compression Toolbox and Benchmark.

Efficient computing methods developed by Huawei Noah's Ark Lab

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/I...

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

A treasure chest for visual classification and recognition powered by PaddlePaddle

A curated list of awesome knowledge distillation papers and codes for object detection.

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

Pytorch implementation of various Knowledge Distillation (KD) methods.

Gather research papers, corresponding codes (if having), reading notes and any other related materials about Hot🔥🔥🔥 fields in Computer Vision based on Deep Learning.

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

大模型/LLM推理和部署理论与实践

A curated list of awesome knowledge distillation papers and codes for object detection.

Gather research papers, corresponding codes (if having), reading notes and any other related materials about Hot🔥🔥🔥 fields in Computer Vision based on Deep Learning.

A curated list for Efficient Large Language Models

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

(NeurIPS 2022) Official Implementation of "Preservation of the Global Knowledge by Not-True Distillation in Federated Learning"

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

SimXNS is a research project for information retrieval. This repo contains official implementations by MSRA NLC team.

Collection of AWESOME vision-language models for vision tasks

👗 DM-VTON: Distilled Mobile Real-time Virtual Try-On

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

[ICCV 2023] - Zero-shot Composed Image Retrieval with Textual Inversion

[CVIU2023] Codes for my paper "SSDA-YOLO: Semi-supervised Domain Adaptive YOLO for Cross-Domain Object Detection"

Code for "Lion: Adversarial Distillation of Proprietary Large Language Models (EMNLP 2023)"

Knowledge distillation in text classification with pytorch. 知识蒸馏，中文文本分类，教师模型BERT、XLNET，学生模型biLSTM。

模型压缩的小白入门教程

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

OpenMMLab Model Compression Toolbox and Benchmark.

The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/I...

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

Awesome Knowledge Distillation

A treasure chest for visual classification and recognition powered by PaddlePaddle

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

大模型/LLM推理和部署理论与实践

OpenMMLab Model Compression Toolbox and Benchmark.

Pytorch implementation of various Knowledge Distillation (KD) methods.

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

模型压缩的小白入门教程

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

Efficient computing methods developed by Huawei Noah's Ark Lab

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

大模型/LLM推理和部署理论与实践

[IEEE TIV] Official PyTorch Implementation of ''A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving.''

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

TensorFlow code for our ECCV'24 Workshop paper "LightAvatar: Efficient Head Avatar as Dynamic NeLF"

模型压缩的小白入门教程

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[MICCAI 2023] MDViT: Multi-domain Vision Transformer for Small Medical Image Segmentation Datasets (an official implementation)

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

A curated list for Efficient Large Language Models

Collection of AWESOME vision-language models for vision tasks

👗 DM-VTON: Distilled Mobile Real-time Virtual Try-On

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

(CVPR 2023) Revisiting Reverse Distillation for Anomaly Detection

[ICCV 2023] - Zero-shot Composed Image Retrieval with Textual Inversion

PPSNet: Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos (ECCV, 2024)

A curated list of awesome knowledge distillation papers and codes for object detection.

Deep Multimodal Guidance for Medical Image Classification: https://arxiv.org/pdf/2203.05683.pdf

Gather research papers, corresponding codes (if having), reading notes and any other related materials about Hot🔥🔥🔥 fields in Computer Vision based on Deep Learning.

[CVIU2023] Codes for my paper "SSDA-YOLO: Semi-supervised Domain Adaptive YOLO for Cross-Domain Object Detection"

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

模型压缩的小白入门教程

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

大模型/LLM推理和部署理论与实践

Mechanistically interpretable neurosymbolic AI (Nature Comput Sci 2024): losslessly compressing NNs to computer code and discovering new algorithms which generalize out-of-distribution and outperform ...

Code for CVPR'24 Paper: Segment Any Event Streams via Weighted Adaptation of Pivotal Tokens

Official code for TPAMI2024 paper: Pixel Distillation: Cost-flexible Distillation across Image Sizes and Heterogeneous Networks

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

[WWW'2024] "PromptMM: Multi-Modal Knowledge Distillation for Recommendation with Prompt-Tuning"

Attention-guided Feature Distillation for Semantic Segmentation

[CVPR'24] Official implementation of paper "FreeKD: Knowledge Distillation via Semantic Frequency Prompt".

PPSNet: Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos (ECCV, 2024)

PyTorch code and pretrained weights for the UNIC models.

[ECCV2024] Mitigating Background Shift in Class-Incremental Semantic Segmentation

TensorFlow code for our ECCV'24 Workshop paper "LightAvatar: Efficient Head Avatar as Dynamic NeLF"

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

A treasure chest for visual classification and recognition powered by PaddlePaddle

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

Awesome Knowledge Distillation

This is a collection of our NAS and Vision Transformer work.

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

OpenMMLab Model Compression Toolbox and Benchmark.

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

Pytorch implementation of various Knowledge Distillation (KD) methods.

模型压缩的小白入门教程

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

Efficient computing methods developed by Huawei Noah's Ark Lab

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

[IEEE TIV] Official PyTorch Implementation of ''A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving.''

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

Attention-guided Feature Distillation for Semantic Segmentation

👗 DM-VTON: Distilled Mobile Real-time Virtual Try-On

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

[ECCV2024] Mitigating Background Shift in Class-Incremental Semantic Segmentation

A curated list for Efficient Large Language Models

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

CVPR 2023, Class Attention Transfer Based Knowledge Distillation

Decoupled Kullback-Leibler Divergence Loss (DKL), NeurIPS 2024

Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)

[ICASSP 2023] Prototype Knowledge Distillation for Medical Segmentation with Missing Modality

(CVPR 2023) Revisiting Reverse Distillation for Anomaly Detection

A curated list of awesome knowledge distillation papers and codes for object detection.

Collection of AWESOME vision-language models for vision tasks

[ICCV 2023] - Zero-shot Composed Image Retrieval with Textual Inversion

[WWW'23] "SimRec: Graph-less Collaborative Filtering"