Trending repositories for topic knowledge-distillation

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

Awesome Knowledge Distillation

大模型/LLM推理和部署理论与实践

A treasure chest for visual classification and recognition powered by PaddlePaddle

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

Pytorch implementation of various Knowledge Distillation (KD) methods.

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

Efficient computing methods developed by Huawei Noah's Ark Lab

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

OpenMMLab Model Compression Toolbox and Benchmark.

This is a collection of our NAS and Vision Transformer work.

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

大模型/LLM推理和部署理论与实践

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

A curated list for Efficient Large Language Models

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

Collection of AWESOME vision-language models for vision tasks

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

Awesome Knowledge Distillation

Pytorch implementation of various Knowledge Distillation (KD) methods.

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

Efficient computing methods developed by Huawei Noah's Ark Lab

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

OpenMMLab Model Compression Toolbox and Benchmark.

This is a collection of our NAS and Vision Transformer work.

A treasure chest for visual classification and recognition powered by PaddlePaddle

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

A treasure chest for visual classification and recognition powered by PaddlePaddle

Awesome Knowledge Distillation

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

大模型/LLM推理和部署理论与实践

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

Pytorch implementation of various Knowledge Distillation (KD) methods.

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

模型压缩的小白入门教程

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

OpenMMLab Model Compression Toolbox and Benchmark.

This is a collection of our NAS and Vision Transformer work.

IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation (CVPR 2022)

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

[AAAI 2023] Official PyTorch Code for "Curriculum Temperature for Knowledge Distillation"

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

大模型/LLM推理和部署理论与实践

PPSNet: Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos (ECCV, 2024)

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

模型压缩的小白入门教程

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[AAAI 2023] Official PyTorch Code for "Curriculum Temperature for Knowledge Distillation"

A curated list for Efficient Large Language Models

IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation (CVPR 2022)

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

Collection of AWESOME vision-language models for vision tasks

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

[CVPR 2022] Learning Multiple Adverse Weather Removal via Two-stage Knowledge Learning and Multi-contrastive Regularization: Toward a Unified Model

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

Awesome Knowledge Distillation

Pytorch implementation of various Knowledge Distillation (KD) methods.

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

A treasure chest for visual classification and recognition powered by PaddlePaddle

Awesome Knowledge Distillation

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

模型压缩的小白入门教程

大模型/LLM推理和部署理论与实践

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

This is a collection of our NAS and Vision Transformer work.

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

Efficient computing methods developed by Huawei Noah's Ark Lab

大模型/LLM推理和部署理论与实践

模型压缩的小白入门教程

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

[IEEE TIV] Official PyTorch Implementation of ''A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving.''

Code for CVPR'24 Paper: Segment Any Event Streams via Weighted Adaptation of Pivotal Tokens

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)

Decoupled Kullback-Leibler Divergence Loss (DKL), NeurIPS 2024

A curated list for Efficient Large Language Models

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation (ICCV 2023)

Collection of AWESOME vision-language models for vision tasks

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

PPSNet: Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos (ECCV, 2024)

[ICCV 2023] Official implementation for "Label-Guided Knowledge Distillation for Continual Semantic Segmentation on 2D Images and 3D Point Clouds".

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

模型压缩的小白入门教程

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

大模型/LLM推理和部署理论与实践

Mechanistically interpretable neurosymbolic AI (Nature Comput Sci 2024): losslessly compressing NNs to computer code and discovering new algorithms which generalize out-of-distribution and outperform ...

Code for CVPR'24 Paper: Segment Any Event Streams via Weighted Adaptation of Pivotal Tokens

Official code for TPAMI2024 paper: Pixel Distillation: Cost-flexible Distillation across Image Sizes and Heterogeneous Networks

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

[WWW'2024] "PromptMM: Multi-Modal Knowledge Distillation for Recommendation with Prompt-Tuning"

[IEEE TIV] Official PyTorch Implementation of ''A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving.''

Attention-guided Feature Distillation for Semantic Segmentation

[CVPR'24] Official implementation of paper "FreeKD: Knowledge Distillation via Semantic Frequency Prompt".

PyTorch code and pretrained weights for the UNIC models.

[ECCV2024] Mitigating Background Shift in Class-Incremental Semantic Segmentation

PPSNet: Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos (ECCV, 2024)

Collection of AWESOME vision-language models for vision tasks

A curated list for Efficient Large Language Models

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

A treasure chest for visual classification and recognition powered by PaddlePaddle

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Traine...

This is a collection of our NAS and Vision Transformer work.

[CVPR 2024 Highlight] Logit Standardization in Knowledge Distillation

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

Awesome Knowledge Distillation

OpenMMLab Model Compression Toolbox and Benchmark.

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

Efficient computing methods developed by Huawei Noah's Ark Lab

Pytorch implementation of various Knowledge Distillation (KD) methods.

模型压缩的小白入门教程

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert...

[IEEE TIV] Official PyTorch Implementation of ''A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving.''

Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasoning elevation🍓 and hallucination alleviation🍄.

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet,...

Attention-guided Feature Distillation for Semantic Segmentation

👗 DM-VTON: Distilled Mobile Real-time Virtual Try-On

A curated list for Efficient Large Language Models

Decoupled Kullback-Leibler Divergence Loss (DKL), NeurIPS 2024

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

[ECCV2024] Mitigating Background Shift in Class-Incremental Semantic Segmentation

[ICASSP 2023] Prototype Knowledge Distillation for Medical Segmentation with Missing Modality

CVPR 2023, Class Attention Transfer Based Knowledge Distillation

Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)

(CVPR 2023) Revisiting Reverse Distillation for Anomaly Detection

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation (ICCV 2023)

A curated list of awesome knowledge distillation papers and codes for object detection.

[ICCV 2023] - Zero-shot Composed Image Retrieval with Textual Inversion

This repository aims at providing efficient CNNs for Audio Tagging. We provide AudioSet pre-trained models ready for downstream training and extraction of audio embeddings.

Collection of AWESOME vision-language models for vision tasks