13 results found Sort:

facebookresearch
mmf facebookresearch
935
5.5k
other
114
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
vqa dialog pytorch textvqa captioning multimodal deep-learning hateful-memes multi-tasking pretrained-models
Created 2018-06-27
1,097 commits to main branch, last one 4 months ago
roboflow
maestro roboflow
95
1.3k
apache-2.0
19
streamline the fine-tuning process for multimodal models: PaliGemma, Florence-2, and Qwen2-VL
vqa paligemma captioning florence-2 multimodal fine-tuning phi-3-vision transformers objectdetection vision-and-language
Created 2023-11-24
233 commits to develop branch, last one 2 days ago
DavidHuji
CapDec DavidHuji
19
181
mit
5
CapDec: SOTA Zero Shot Image Captioning Using CLIP and GPT2, EMNLP 2022 (findings)
clip gpt-2 clipcap captioning zero-shot-learning multimodal-deep-learning
Created 2022-05-22
291 commits to main branch, last one 8 months ago
Labbeti
aac-datasets Labbeti
6
100
mit
2
Audio Captioning datasets for PyTorch.
audio caption dataset pytorch datasets captioning deep-learning audio-captioning
Created 2022-05-19
13 commits to main branch, last one 6 months ago
mitvis
vistext mitvis
3
80
gpl-3.0
6
VisText is a benchmark dataset for semantically rich chart captioning.
t5 charts dataset captioning captioning-images
Created 2023-04-04
93 commits to main branch, last one 12 months ago
TheShadow29
VidSitu TheShadow29
8
57
mit
3
[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)
nlp srl video vision grounding captioning semantic-roles video-language event-relations captioning-videos vision-and-language
Created 2021-02-10
57 commits to main branch, last one 3 years ago
ParitoshParmar
MTL-AQA ParitoshParmar
15
55
unknown
4
What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment [CVPR 2019]
c3d lstm mtl-aqa pytorch captioning dilated-c3d video-captioning video-processing action-recognition multitask-learning dilated-convolution video-understanding representation-learning action-quality-assessment fine-grained-classification fine-grained-action-recognition
Created 2019-06-11
24 commits to master branch, last one about a year ago
Mauville
MedCLIP Mauville
13
55
unknown
4
Medical image captioning using OpenAI's CLIP
clip captioning deep-learning medical-imaging machine-learning what-a-challenge-this-was
Created 2021-12-02
16 commits to main branch, last one about a year ago
aimagelab
pacscore aimagelab
4
51
unknown
6
Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation. CVPR 2023
cvpr cvpr2023 captioning computer-vision captioning-images captioning-videos vision-and-language
Created 2023-01-29
13 commits to main branch, last one about a year ago
DavidMChan
caption-by-committee DavidMChan
4
40
other
3
Using LLMs and pre-trained caption models for super-human performance on image captioning.
ai image python chatgpt captioning deep-learning machine-learning
Created 2022-12-14
50 commits to main branch, last one 11 months ago
42lux
CaptainCaption 42lux
5
36
mit
3
A gradio based image captioning tool that uses the GPT-4-Vision API to generate detailed descriptions of images.
gradio tagging captioning openai-api gpt-4-vision
Created 2023-12-19
14 commits to main branch, last one 3 days ago
CurryYuan
X-Trans2Cap CurryYuan
3
33
apache-2.0
3
[CVPR 2022] X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning
cvpr2022 captioning
Created 2022-03-08
14 commits to main branch, last one 2 years ago
Labbeti
aac-metrics Labbeti
3
33
mit
3
Metrics for evaluating Automated Audio Captioning systems, designed for PyTorch.
text audio metrics captioning audio-captioning
Created 2022-09-20
18 commits to main branch, last one 6 months ago