14 results found Sort:

showlab
VLog showlab
21
497
mit
6
Transform Video as a Document with ChatGPT, CLIP, BLIP2, GRIT, Whisper, LangChain.
chatgpt whisper langchain video-language large-language-model
Created 2023-04-20
25 commits to main branch, last one 10 months ago
microsoft
UniVL microsoft
54
330
mit
10
An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"
coin joint video msrvtt caption pretrain alignment youcookii video-text pretraining caption-task localization segmentation multimodality retrieval-task video-language video-text-retrieval multimodal-sentiment-analysis
Created 2020-10-30
20 commits to main branch, last one about a year ago
wjn922
ReferFormer wjn922
26
311
apache-2.0
7
[CVPR2022] Official Implementation of ReferFormer
video-language referring-video-object-segmentation
Created 2022-01-02
56 commits to main branch, last one about a month ago
showlab
UniVTG showlab
21
290
mit
5
[ICCV2023] UniVTG: Towards Unified Video-Language Temporal Grounding
pretraining video-language video-grounding moment-retrieval highlight-detection video-summarization
Created 2023-05-27
134 commits to main branch, last one 23 days ago
showlab
all-in-one showlab
16
274
unknown
6
[CVPR2023] All in One: Exploring Unified Video-Language Pre-training
pytorch codebase pre-training video-language
Created 2022-03-14
29 commits to main branch, last one about a year ago
junchen14
Multi-Modal-Transformer junchen14
29
211
unknown
8
The repository collects many various multi-modal transformer architectures, including image transformer, video transformer, image-language transformer, video-language transformer and self-supervised l...
language mlp-mixer multi-modal video-language image-transformer video-transformer vision-transformer multi-modal-cvpr2021 efficiency-transformer transformer-readling-list
Created 2021-04-07
418 commits to main branch, last one about a year ago
showlab
EgoVLP showlab
19
210
unknown
3
[NeurIPS2022] Egocentric Video-Language Pretraining
pytorch pretraining video-language egocentric-vision
Created 2022-05-31
48 commits to main branch, last one 23 days ago
salesforce
ALPRO salesforce
17
184
bsd-3-clause
7
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
video-language prompt-learning vision-and-language video-text-retrieval representation-learning video-question-answering
Created 2021-12-11
14 commits to main branch, last one about a year ago
MikeWangWZHL
VidIL MikeWangWZHL
2
110
mit
5
Pytorch code for Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
blip clip msvd vlep gpt-3 vatex msrvtt youcook2 video-language vision-language
Created 2022-05-20
17 commits to main branch, last one about a year ago
bytedance
Shot2Story bytedance
2
61
unknown
6
A new multi-shot video understanding benchmark Shot2Story with comprehensive video summaries and detailed shot-level captions.
dataset benchmark video-story video-language vision-language video-captioning video-summarization large-language-models video-story-generation video-question-answering video-language-pretraining
Created 2023-12-16
30 commits to master branch, last one 8 days ago
TheShadow29
VidSitu TheShadow29
8
55
mit
3
[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)
nlp srl video vision grounding captioning semantic-roles video-language event-relations captioning-videos vision-and-language
Created 2021-02-10
57 commits to main branch, last one 2 years ago
liveseongho
Awesome-Video-Language-Understanding liveseongho
2
44
mit
1
A Survey on video and language understanding.
paper dataset deep-learning awesome-papers video-language machine-learning multimodal-deep-learning video-language-pretraining video-language-understanding
Created 2023-04-14
23 commits to main branch, last one about a year ago
showlab
Region_Learner showlab
2
42
unknown
5
The Pytorch implementation for "Video-Text Pre-training with Learned Regions"
video-language
Created 2021-11-21
18 commits to main branch, last one about a year ago
zinengtang
Perceiver_VL zinengtang
3
32
mit
3
PyTorch code for "Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention" (WACV 2023)
retrieval efficiency scalability video-language vision-and-language
Created 2022-03-14
123 commits to main branch, last one about a year ago