97 results found Sort:

pliang279
awesome-multimodal-ml pliang279
856
6.2k
mit
179
Reading list for research topics in multimodal machine learning
robotics healthcare reading-list deep-learning computer-vision machine-learning speech-processing multimodal-learning reinforcement-learning representation-learning natural-language-processing
Created 2019-05-27
435 commits to master branch, last one 6 months ago
mlfoundations
open_flamingo mlfoundations
287
3.8k
mit
48
An open-source framework for training large multimodal models.
pytorch flamingo deep-learning language-model computer-vision in-context-learning multimodal-learning
Created 2022-10-20
502 commits to main branch, last one about a year ago
KaiyangZhou
CoOp KaiyangZhou
204
1.8k
mit
16
Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)
prompt-learning foundation-models multimodal-learning
Created 2021-09-01
66 commits to main branch, last one 2 years ago
Eurus-Holmes
Awesome-Multimodal-Research Eurus-Holmes
150
1.3k
mit
41
A curated list of Multimodal Related Research.
awesome multimodal multimodal-learning multimodal-research
Created 2019-07-31
206 commits to master branch, last one about a year ago
AILab-CVC
UniRepLKNet AILab-CVC
56
956
apache-2.0
13
[CVPR'24] UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition
architecture deep-learning multimodal-learning artificial-intelligence convolutional-neural-networks
Created 2023-11-24
62 commits to main branch, last one about a month ago
DmitryRyumin
ICCV-2023-Papers DmitryRyumin
41
946
mit
14
ICCV 2023 Papers: Discover cutting-edge research from ICCV 2023, the leading computer vision conference. Stay updated on the latest in computer vision and deep learning, with code included. ⭐ support ...
iccv datasets iccv2023 robotics biometrics 3d-graphics deep-learning explainable-ai photogrammetry computer-vision image-synthesis pose-estimation video-synthesis face-recognition image-processing 3d-reconstruction transfer-learning gesture-recognition multimodal-learning pattern-recognition
Created 2023-09-04
1,068 commits to main branch, last one 3 months ago
PreferredAI
cornac PreferredAI
148
897
apache-2.0
26
A Comparative Framework for Multimodal Recommender Systems
multimodality recommender-system multimodal-learning matrix-factorization recommendation-engine recommendation-system collaborative-filtering recommendation-algorithms
Created 2018-07-17
1,370 commits to master branch, last one 3 days ago
ArrowLuo
CLIP4Clip ArrowLuo
125
897
mit
13
An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"
clip msvd lsmdc didemo msrvtt search ranking retrieval multimodal activitynet multimodality retrieval-model multimodal-learning video-clip-retrieval video-text-retrieval
Created 2021-04-13
29 commits to master branch, last one 2 years ago
declare-lab
multimodal-deep-learning declare-lab
156
775
mit
7
This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.
multimodal-learning multimodal-interactions multimodal-deep-learning multimodal-sentiment-analysis
Created 2021-08-28
95 commits to main branch, last one 2 years ago
HuaizhengZhang
Awsome-Deep-Learning-for-Video-Analysis HuaizhengZhang
172
767
mit
34
Papers, code and datasets about deep learning and multi-modal learning for video analysis
paper deep-learning video-dataset video-analysis machine-learning multimodal-learning video-classification
Created 2017-06-14
91 commits to master branch, last one 3 years ago
henghuiding
ReLA henghuiding
19
656
mit
5
[CVPR2023 Highlight] GRES: Generalized Referring Expression Segmentation
cvpr2023 multimodal-learning vision-language-transformer referring-image-segmentation referring-expression-segmentation referring-expression-comprehension
Created 2023-03-11
14 commits to main branch, last one about a year ago
richard-peng-xia
awesome-multimodal-in-medical-imaging richard-peng-xia
61
604
mit
18
A collection of resources on applications of multi-modal learning in medical imaging.
medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models
Created 2022-07-13
151 commits to main branch, last one about a month ago
georgian-io
Multimodal-Toolkit georgian-io
85
594
apache-2.0
25
Multimodal model for text and tabular data with HuggingFace transformers as building block for text data
transformer tabular-data multimodal-learning huggingface-transformers natural-language-processing
Created 2020-08-20
169 commits to master branch, last one about a month ago
njustkmg
OMML njustkmg
98
565
apache-2.0
42
Multi-Modal learning toolkit based on PaddlePaddle and PyTorch, supporting multiple applications such as multi-modal classification, cross-modal retrieval and image caption.
python pytorch multimodal paddlepaddle classification imagecaptioning multimodal-learning crossmodal-retrieval
Created 2022-01-05
198 commits to Pytorch branch, last one about a year ago
sangminwoo
awesome-vision-and-language sangminwoo
40
511
unknown
12
A curated list of awesome vision and language resources (still under construction... stay tuned!)
awesome awesome-list multimodal-learning vision-and-language
Created 2019-10-25
48 commits to master branch, last one about a month ago
pliang279
MultiBench pliang279
71
498
mit
16
[NeurIPS 2021] Multiscale Benchmarks for Multimodal Representation Learning
robotics healthcare deep-learning computer-vision machine-learning speech-processing multimodal-learning representation-learning natural-language-processing
Created 2021-03-05
1,258 commits to main branch, last one 10 months ago
henghuiding
MeViS henghuiding
20
494
mit
8
[ICCV 2023] MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions
mose-dataset mevis-dataset multimodal-learning video-understanding referring-expression-segmentation referring-expression-comprehension referring-video-object-segmentation
Created 2023-08-01
11 commits to main branch, last one about a year ago
microsoft
XPretrain microsoft
36
479
other
14
Multi-modality pre-training
nlp multimedia pre-training computer-vision multimodal-learning
Created 2022-03-15
104 commits to main branch, last one 7 months ago
pykale
pykale pykale
64
445
mit
12
Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the 🔥PyTorch ecosystem. ⭐ Star to support our work!
python pytorch multimodal data-science deep-learning meta-learning graph-analysis computer-vision machine-learning domain-adaptation transfer-learning multimodal-learning medical-image-analysis knowledge-aware-learning
Created 2020-06-30
3,054 commits to main branch, last one 2 months ago
DmitryRyumin
ICASSP-2023-24-Papers DmitryRyumin
17
418
mit
29
ICASSP 2023-2024 Papers: A complete collection of influential and exciting research papers from the ICASSP 2023-24 conferences. Explore the latest advancements in acoustics, speech and signal processi...
asr vad icassp denoising icassp2023 icassp2024 face-recognition image-generation keyword-spotting music-generation domain-adaptation generative-models language-modeling signal-processing signal-restoration speech-recognition multimodal-learning semantic-segmentation self-supervised-learning spoken-language-understanding
Created 2023-08-01
975 commits to main branch, last one 16 hours ago
MMMU-Benchmark
MMMU MMMU-Benchmark
29
370
apache-2.0
4
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
llm llms stem evaluation multimodal deep-learning multimodality computer-vision machine-learning foundation-models question-answering multimodal-learning deep-neural-networks large-language-models large-multimodal-models multimodal-deep-learning visual-question-answering natural-language-processing
Created 2023-11-23
131 commits to main branch, last one 11 days ago
kyegomez
CM3Leon kyegomez
18
364
mit
22
An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images
dalle attention multimodal multimodality imagegeneration multimodal-learning attention-is-all-you-need
Created 2023-07-14
73 commits to main branch, last one about a year ago
HenryHZY
Awesome-Multimodal-LLM HenryHZY
16
356
mit
17
Research Trends in LLM-guided Multimodal Learning.
llm multimodal instruction-tuning in-context-learning multimodal-learning large-language-models parameter-efficient-tuning parameter-efficient-learning multimodal-large-language-models
Created 2023-05-29
16 commits to main branch, last one about a year ago
Pointcept
GPT4Point Pointcept
22
351
mit
24
[CVPR'24 Highlight] GPT4Point: A Unified Framework for Point-Language Understanding and Generation.
llm 3d-generation multimodal-learning
Created 2023-08-12
76 commits to main branch, last one 7 months ago
HUANGLIZI
LViT HUANGLIZI
26
313
mit
3
[IEEE Transactions on Medical Imaging/TMI] This repo is the official implementation of "LViT: Language meets Vision Transformer in Medical Image Segmentation"
pytorch segmentation vision-language multimodal-learning medical-image-analysis
Created 2022-03-10
64 commits to main branch, last one about a year ago
mmaaz60
mvits_for_class_agnostic_od mmaaz60
25
303
mit
8
[ECCV'22] Official repository of paper titled "Class-agnostic Object Detection with Multi-modal Transformer".
pytorch psuedo-labels object-detection multimodal-learning open-world-detection class-agnostic-detection
Created 2021-11-16
33 commits to main branch, last one about a year ago
UCSC-VLAA
CLIPA UCSC-VLAA
13
303
apache-2.0
14
[NeurIPS 2023] This repository includes the official implementation of our paper "An Inverse Scaling Law for CLIP Training"
pytorch neurips-2023 deep-learning foundation-models zero-shot-learning multimodal-learning contrastive-learning zero-shot-classification
Created 2023-04-26
79 commits to master branch, last one 6 months ago
ilaria-manco
multimodal-ml-music ilaria-manco
12
285
mit
14
List of academic resources on Multimodal ML for Music
music-ai resources awesome-list music-research multimodal-data multimodal-learning academic-publications multimodal-deep-learning music-information-retrieval
Created 2022-12-29
11 commits to main branch, last one about a year ago
ys-zong
awesome-self-supervised-multimodal-learning ys-zong
7
233
unknown
5
[T-PAMI] A curated list of self-supervised multimodal learning resources.
awesome-list machine-learning multimodal-learning self-supervised-learning
Created 2023-03-31
14 commits to main branch, last one 4 months ago
kyegomez
NaViT kyegomez
10
197
mit
7
My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"
vit clip gpt4 multimodal multimodality attention-mechanism multimodal-learning multimodal-deep-learning
Created 2023-09-28
18 commits to main branch, last one 10 months ago