36 results found Sort:

mlfoundations
open_clip mlfoundations
984
10.3k
other
79
An open source implementation of CLIP.
pytorch deep-learning language-model computer-vision contrastive-loss pretrained-models multi-modal-learning zero-shot-classification
Created 2021-07-28
566 commits to main branch, last one 21 days ago
OFA-Sys
Chinese-CLIP OFA-Sys
471
4.6k
mit
35
Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.
nlp clip chinese pytorch multi-modal transformers coreml-models deep-learning computer-vision vision-language contrastive-loss pretrained-models image-text-retrieval multi-modal-learning vision-and-language-pre-training
Created 2022-07-08
382 commits to master branch, last one 3 months ago
lyuchenyang
Macaw-LLM lyuchenyang
127
1.6k
apache-2.0
33
Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
deep-learning language-model neural-networks machine-learning multi-modal-learning natural-language-processing
Created 2023-05-23
142 commits to main branch, last one 7 months ago
NVlabs
prismer NVlabs
75
1.3k
other
16
The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".
vqa language-model image-captioning multi-task-learning vision-and-language multi-modal-learning vision-language-model
Created 2023-03-02
36 commits to main branch, last one 10 months ago
lucidrains
x-clip lucidrains
46
693
mit
25
A concise but complete implementation of CLIP with various experimental improvements from recent papers
deep-learning zero-shot-learning contrastive-learning multi-modal-learning artificial-intelligence
Created 2021-12-01
76 commits to main branch, last one about a year ago
jokieleung
awesome-visual-question-answering jokieleung
95
659
unknown
26
A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.
vqa multi-modal awesome-list attention-networks multi-modal-learning
Created 2019-03-03
36 commits to master branch, last one about a year ago
OpenRobotLab
EmbodiedScan OpenRobotLab
37
490
apache-2.0
7
[CVPR 2024 & NeurIPS 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
robotics 3d-vision computer-vision multi-modal-learning
Created 2023-12-11
105 commits to main branch, last one 2 months ago
kyegomez
zeta kyegomez
42
423
apache-2.0
5
Build high-performance AI models with modular building blocks
gpt4 llama2 longnet pytorch multi-modal transformer transformers deep-learning multi-platform speech-recognition multi-agent-systems multi-modal-learning artificial-intelligence
Created 2023-07-09
882 commits to master branch, last one 19 days ago
DmitryRyumin
CVPR-2023-24-Papers DmitryRyumin
28
420
mit
10
CVPR 2023-2024 Papers: Dive into advanced research presented at the leading computer vision conference. Keep up to date with the latest developments in computer vision and deep learning. Code included...
cvpr cvpr2023 cvpr2024 datasets biometrics segmentation deep-learning scene-analysis shape-analysis computer-vision image-synthesis video-synthesis face-recognition action-recognition autonomous-driving gesture-recognition pattern-recognition multi-modal-learning medical-image-processing self-supervised-learning
Created 2023-08-09
1,146 commits to main branch, last one 4 months ago
zjukg
KG-MM-Survey zjukg
16
339
mit
7
Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey
awsome survey surveys paper-list awsome-list entity-linking knowledge-graph entity-alignment image-generation multi-modal-fusion image-classification multi-modal-learning cross-modal-retrieval large-language-models information-extraction visual-question-answering knowledge-graph-embeddings multi-modal-knowledge-graph
Created 2024-01-29
77 commits to main branch, last one 6 months ago
zhengli97
PromptKD zhengli97
3
237
apache-2.0
4
[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"
clip cvpr2024 prompt-learning multi-modal-learning vision-language-model knowledge-distillation
Created 2024-02-27
40 commits to main branch, last one about a month ago
Ysz2022
NeRCo Ysz2022
15
222
unknown
5
[ICCV 2023] Implicit Neural Representation for Cooperative Low-light Image Enhancement
iccv iccv2023 low-light-image multi-modal-learning neural-representation low-light-image-enhancement
Created 2023-03-20
38 commits to main branch, last one 8 months ago
huggingface
chug huggingface
10
151
apache-2.0
11
Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.
datasets webdataset dataloading pdf-document computer-vision distributed-training multi-modal-learning document-understanding
Created 2023-06-06
36 commits to main branch, last one 7 months ago
GuanRunwei
Achelous GuanRunwei
9
145
unknown
5
The official repository of Achelous and Achelous++
4d-mmwave-radar object-tracking object-detection multi-modal-fusion multi-task-learning panoptic-perception multi-modal-learning semantic-segmentation point-cloud-segmentation
Created 2023-03-17
150 commits to main branch, last one 4 months ago
qizekun
ReCon qizekun
13
129
mit
8
[ICML 2023] Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining
3d-point-clouds multi-modal-learning representation-learning self-supervised-learning
Created 2023-02-07
13 commits to main branch, last one 4 months ago
wjun0830
CGDETR wjun0830
11
117
other
5
Official pytorch repository for CG-DETR "Correlation-guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding"
detr pytorch computer-vision video-grounding moment-retrieval temporal-grounding highlight-detection video-summarization video-understanding multi-modal-learning text-video-retrieval detection-transformer
Created 2023-11-10
18 commits to main branch, last one 3 months ago
shikras
d-cube shikras
7
108
other
8
A detection/segmentation dataset with labels characterized by intricate and flexible expressions. "Described Object Detection: Liberating Object Detection with Flexible Expressions" (NeurIPS 2023).
dataset vision-language object-detection multi-modal-learning open-vocabulary-detection referring-expression-comprehension
Created 2023-07-19
13 commits to main branch, last one 8 months ago
924973292
EDITOR 924973292
5
86
mit
2
【CVPR2024】Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification
reid msvr310 cvpr2024 rgbnt100 rgbnt201 multi-modal person-reid token-selection frequency-analysis multi-modal-learning vehicle-reidentification
Created 2024-03-14
124 commits to master branch, last one 28 days ago
WillDreamer
Aurora WillDreamer
7
83
unknown
5
[NeurIPS2023] Parameter-efficient Tuning of Large-scale Multimodal Foundation Model
multi-modal-learning parameter-efficient-tuning
Created 2023-05-14
20 commits to main branch, last one 11 months ago
likyoo
Multimodal-Remote-Sensing-Toolkit likyoo
12
83
gpl-3.0
3
A python tool to perform deep learning experiments on multimodal remote sensing data.
python pytorch remote-sensing multi-modal-learning
Created 2021-02-28
26 commits to main branch, last one 2 years ago
RL4M
MRM-pytorch RL4M
5
74
mit
1
An official implementation of Advancing Radiograph Representation Learning with Masked Record Modeling (ICLR'23)
chest-xray-images pre-trained-model multi-modal-learning representation-learning self-supervised-learning
Created 2023-01-18
14 commits to main branch, last one about a year ago
RAIVNLab
sugar-crepe RAIVNLab
7
70
mit
10
[NeurIPS 2023] A faithful benchmark for vision-language compositionality
pytorch benchmark deep-learning vision-and-language multi-modal-learning
Created 2023-06-04
54 commits to main branch, last one 9 months ago
rinnakk
japanese-clip rinnakk
8
68
apache-2.0
8
Japanese CLIP by rinna Co., Ltd.
clip cloob vision japanese language-model pretrained-models multi-modal-learning
Created 2022-04-25
31 commits to master branch, last one 2 years ago
rentainhe
TRAR-VQA rentainhe
18
65
mit
3
[ICCV 2021] Official implementation of the paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering"
clevr vqav2 pytorch iccv2021 official attention multi-modal transformer visualization multi-modality dynamic-network local-and-global vision-and-language multi-modal-learning multi-scale-features visual-question-answering
Created 2021-07-23
61 commits to main branch, last one 3 years ago
ttgeng233
UnAV ttgeng233
4
61
mit
3
Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline (CVPR 2023)
audio-visual-events multi-modal-learning audio-visual-learning
Created 2023-04-05
13 commits to main branch, last one 9 months ago
yihedeng9
STIC yihedeng9
4
59
apache-2.0
3
Enhancing Large Vision Language Models with Self-Training on Image Comprehension.
llava mistral-7b multi-modal llm-finetuning multi-modal-learning vision-language-model
Created 2024-05-27
17 commits to main branch, last one 5 months ago
zhjohnchan
awesome-vision-and-language-pretraining zhjohnchan
7
56
unknown
3
A curated list of vision-and-language pre-training (VLP). :-)
pre-training multi-modal-learning vision-and-language-pre-training
Created 2021-10-30
19 commits to main branch, last one 2 years ago
YuanGongND
uavm YuanGongND
3
54
bsd-2-clause
2
Code for the IEEE Signal Processing Letters 2022 paper "UAVM: Towards Unifying Audio and Visual Models".
computer-vision audio-classification multi-modal-learning
Created 2023-02-16
14 commits to main branch, last one about a year ago
vishalned
MMEarth-data vishalned
3
50
mit
3
This repository contains code to download data for the preprint "MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning"
remote-sensing computer-vision earth-observation multi-modal-learning representation-learning
Created 2024-03-25
46 commits to main branch, last one 14 days ago
deep-symbolic-mathematics
Multimodal-Math-Pretraining deep-symbolic-mathematics
5
47
mit
3
[ICLR 2024 Spotlight] This is the official code for the paper "SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training"
ai4math ai4science multi-modal transformers deep-learning symbolic-math symbolic-regression multi-modal-learning representation-learning
Created 2024-01-07
33 commits to main branch, last one 29 days ago