33 results found Sort:

mlfoundations
open_clip mlfoundations
898
9.0k
other
77
An open source implementation of CLIP.
pytorch deep-learning language-model computer-vision contrastive-loss pretrained-models multi-modal-learning zero-shot-classification
Created 2021-07-28
534 commits to main branch, last one 3 days ago
OFA-Sys
Chinese-CLIP OFA-Sys
414
3.9k
mit
35
Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.
nlp clip chinese pytorch multi-modal transformers coreml-models deep-learning computer-vision vision-language contrastive-loss pretrained-models image-text-retrieval multi-modal-learning vision-and-language-pre-training
Created 2022-07-08
374 commits to master branch, last one 7 months ago
lyuchenyang
Macaw-LLM lyuchenyang
114
1.5k
apache-2.0
33
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
deep-learning language-model neural-networks machine-learning multi-modal-learning natural-language-processing
Created 2023-05-23
142 commits to main branch, last one 2 months ago
QIN2DIM
hcaptcha-challenger QIN2DIM
257
1.4k
gpl-3.0
21
🥂 Gracefully face hCaptcha challenge with MoE(ONNX) embedded solution.
clip onnx yolo solver yolov5 hcaptcha playwright multi-modal onnx-models onnxruntime opencv-python computer-vision hcaptcha-solver object-detection image-segmentation multi-modal-learning zero-shot-classification
Created 2022-02-15
885 commits to main branch, last one 6 months ago
NVlabs
prismer NVlabs
75
1.3k
other
15
The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".
vqa language-model image-captioning multi-task-learning vision-and-language multi-modal-learning vision-language-model
Created 2023-03-02
36 commits to main branch, last one 5 months ago
lucidrains
x-clip lucidrains
46
664
mit
25
A concise but complete implementation of CLIP with various experimental improvements from recent papers
deep-learning zero-shot-learning contrastive-learning multi-modal-learning artificial-intelligence
Created 2021-12-01
76 commits to main branch, last one 8 months ago
jokieleung
awesome-visual-question-answering jokieleung
95
647
unknown
26
A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.
vqa multi-modal awesome-list attention-networks multi-modal-learning
Created 2019-03-03
36 commits to master branch, last one about a year ago
OpenRobotLab
EmbodiedScan OpenRobotLab
24
358
apache-2.0
7
[CVPR 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
robotics 3d-vision computer-vision multi-modal-learning
Created 2023-12-11
101 commits to main branch, last one 12 days ago
DmitryRyumin
CVPR-2023-24-Papers DmitryRyumin
21
328
mit
8
CVPR 2023-2024 Papers: Dive into advanced research presented at the leading computer vision conference. Keep up to date with the latest developments in computer vision and deep learning. Code included...
cvpr cvpr2023 cvpr2024 datasets biometrics segmentation deep-learning scene-analysis shape-analysis computer-vision image-synthesis video-synthesis face-recognition action-recognition autonomous-driving gesture-recognition pattern-recognition multi-modal-learning medical-image-processing self-supervised-learning
Created 2023-08-09
1,138 commits to main branch, last one 8 hours ago
kyegomez
zeta kyegomez
26
310
apache-2.0
4
Build high-performance AI models with modular building blocks
gpt4 llama2 longnet pytorch multi-modal transformer transformers deep-learning multi-platform speech-recognition multi-agent-systems multi-modal-learning artificial-intelligence
Created 2023-07-09
791 commits to master branch, last one 2 days ago
zjukg
KG-MM-Survey zjukg
13
228
mit
6
Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey
awsome survey surveys paper-list awsome-list entity-linking knowledge-graph entity-alignment image-generation multi-modal-fusion image-classification multi-modal-learning cross-modal-retrieval large-language-models information-extraction visual-question-answering knowledge-graph-embeddings multi-modal-knowledge-graph
Created 2024-01-29
77 commits to main branch, last one about a month ago
Ysz2022
NeRCo Ysz2022
14
211
unknown
5
[ICCV 2023] Implicit Neural Representation for Cooperative Low-light Image Enhancement
iccv iccv2023 low-light-image multi-modal-learning neural-representation low-light-image-enhancement
Created 2023-03-20
38 commits to main branch, last one 3 months ago
zhengli97
PromptKD zhengli97
1
138
apache-2.0
3
[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"
clip cvpr2024 prompt-learning multi-modal-learning vision-language-model knowledge-distillation
Created 2024-02-27
34 commits to main branch, last one 13 days ago
huggingface
chug huggingface
9
137
apache-2.0
11
Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.
datasets webdataset dataloading pdf-document computer-vision distributed-training multi-modal-learning document-understanding
Created 2023-06-06
36 commits to main branch, last one 2 months ago
GuanRunwei
Achelous GuanRunwei
7
133
unknown
4
Achelous: A Fast Unified Water-surface Panoptic Perception Framework based on Fusion of Monocular Camera and 4D mmWave Radar
4d-mmwave-radar object-tracking object-detection multi-modal-fusion multi-task-learning panoptic-perception multi-modal-learning semantic-segmentation point-cloud-segmentation
Created 2023-03-17
149 commits to main branch, last one 2 months ago
qizekun
ReCon qizekun
12
113
mit
8
[ICML 2023] Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining
3d-point-clouds multi-modal-learning representation-learning self-supervised-learning
Created 2023-02-07
11 commits to main branch, last one 3 months ago
shikras
d-cube shikras
6
99
other
7
A detection/segmentation dataset with labels characterized by intricate and flexible expressions. "Described Object Detection: Liberating Object Detection with Flexible Expressions" (NeurIPS 2023).
dataset vision-language object-detection multi-modal-learning open-vocabulary-detection referring-expression-comprehension
Created 2023-07-19
13 commits to main branch, last one 3 months ago
wjun0830
CGDETR wjun0830
11
89
other
5
Official pytorch repository for CG-DETR "Correlation-guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding"
detr pytorch computer-vision video-grounding moment-retrieval temporal-grounding highlight-detection video-summarization video-understanding multi-modal-learning text-video-retrieval detection-transformer
Created 2023-11-10
16 commits to main branch, last one 2 months ago
WillDreamer
Aurora WillDreamer
6
79
unknown
5
[NeurIPS2023] Parameter-efficient Tuning of Large-scale Multimodal Foundation Model
multi-modal-learning parameter-efficient-tuning
Created 2023-05-14
20 commits to main branch, last one 7 months ago
likyoo
Multimodal-Remote-Sensing-Toolkit likyoo
12
75
gpl-3.0
3
A python tool to perform deep learning experiments on multimodal remote sensing data.
python pytorch remote-sensing multi-modal-learning
Created 2021-02-28
26 commits to main branch, last one 2 years ago
RL4M
MRM-pytorch RL4M
4
70
mit
1
An official implementation of Advancing Radiograph Representation Learning with Masked Record Modeling (ICLR'23)
chest-xray-images pre-trained-model multi-modal-learning representation-learning self-supervised-learning
Created 2023-01-18
14 commits to main branch, last one about a year ago
rinnakk
japanese-clip rinnakk
6
66
apache-2.0
8
Japanese CLIP by rinna Co., Ltd.
clip cloob vision japanese language-model pretrained-models multi-modal-learning
Created 2022-04-25
31 commits to master branch, last one about a year ago
rentainhe
TRAR-VQA rentainhe
18
61
mit
3
[ICCV 2021] Official implementation of the paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering"
clevr vqav2 pytorch iccv2021 official attention multi-modal transformer visualization multi-modality dynamic-network local-and-global vision-and-language multi-modal-learning multi-scale-features visual-question-answering
Created 2021-07-23
61 commits to main branch, last one 2 years ago
RAIVNLab
sugar-crepe RAIVNLab
7
58
mit
10
[NeurIPS 2023] A faithful benchmark for vision-language compositionality
pytorch benchmark deep-learning vision-and-language multi-modal-learning
Created 2023-06-04
54 commits to main branch, last one 4 months ago
924973292
EDITOR 924973292
5
56
mit
2
【CVPR2024】Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification
reid msvr310 cvpr2024 rgbnt100 rgbnt201 multi-modal person-reid token-selection frequency-analysis multi-modal-learning vehicle-reidentification
Created 2024-03-14
121 commits to master branch, last one 13 days ago
zhjohnchan
awesome-vision-and-language-pretraining zhjohnchan
6
54
unknown
3
A curated list of vision-and-language pre-training (VLP). :-)
pre-training multi-modal-learning vision-and-language-pre-training
Created 2021-10-30
19 commits to main branch, last one about a year ago
YuanGongND
uavm YuanGongND
3
54
bsd-2-clause
2
Code for the IEEE Signal Processing Letters 2022 paper "UAVM: Towards Unifying Audio and Visual Models".
computer-vision audio-classification multi-modal-learning
Created 2023-02-16
14 commits to main branch, last one about a year ago
ttgeng233
UnAV ttgeng233
3
51
mit
3
Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline (CVPR 2023)
audio-visual-events multi-modal-learning audio-visual-learning
Created 2023-04-05
13 commits to main branch, last one 4 months ago
yihedeng9
STIC yihedeng9
2
39
apache-2.0
3
Enhancing Large Vision Language Models with Self-Training on Image Comprehension.
llava mistral-7b multi-modal llm-finetuning multi-modal-learning vision-language-model
Created 2024-05-27
17 commits to main branch, last one 27 days ago
liyichen-cly
MMEA liyichen-cly
4
37
unknown
2
MMEA: Entity Alignment for Multi-Modal Knowledge Graphs, KSEM 2020
knowledge-graph entity-alignment multi-modal-learning
Created 2021-03-22
8 commits to master branch, last one 2 years ago