20 results found Sort:

salesforce
LAVIS salesforce
974
9.9k
bsd-3-clause
98
LAVIS - A One-stop Library for Language-Vision Intelligence
salesforce deep-learning image-captioning vision-framework multimodal-datasets vision-and-language deep-learning-library multimodal-deep-learning visual-question-anwsering vision-language-pretraining vision-language-transformer
Created 2022-08-24
492 commits to main branch, last one 2 days ago
DAMO-NLP-SG
Video-LLaMA DAMO-NLP-SG
261
2.8k
bsd-3-clause
32
[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
blip2 llama minigpt4 multi-modal-chatgpt large-language-models cross-modal-pretraining video-language-pretraining vision-language-pretraining
Created 2023-05-06
145 commits to main branch, last one 5 months ago
deepseek-ai
DeepSeek-VL deepseek-ai
195
2.1k
mit
19
DeepSeek-VL: Towards Real-World Vision-Language Understanding
foundation-models vision-language-model vision-language-pretraining
Created 2024-03-07
11 commits to main branch, last one 7 months ago
mbzuai-oryx
Video-ChatGPT mbzuai-oryx
108
1.2k
cc-by-4.0
15
[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for ...
clip gpt-4 llama llava vicuna chatbot mulit-modal video-chatboat vision-language video-conversation vision-language-pretraining
Created 2023-05-18
43 commits to main branch, last one 2 months ago
deepseek-ai
Janus deepseek-ai
51
1.1k
mit
22
Janus-Series: Unified Multimodal Understanding and Generation Models
llm any-to-any multimodal unified-model foundation-models vision-language-pretraining
Created 2024-10-18
16 commits to main branch, last one 8 days ago
Sense-GVT
DeCLIP Sense-GVT
31
637
unknown
19
Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm
clip big-model zero-shot image-text multi-model self-supervised vision-language-pretraining
Created 2021-10-09
34 commits to main branch, last one 2 years ago
TXH-mercury
VALOR TXH-mercury
16
262
mit
9
Codes and Models for VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
audio-language-pretraining vision-language-pretraining audiovisual-language-pretraining multimodal-representation-learning
Created 2023-01-23
47 commits to master branch, last one 5 months ago
mbzuai-oryx
VideoGPT-plus mbzuai-oryx
15
217
cc-by-4.0
5
Official Repository of paper VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding
clip gpt4 gpt4o llava llama3 vicuna chatbot multimodal phi-3-mini dual-encoder image-encoder video-chatbot video-encoder vision-language video-conversation vision-language-pretraining
Created 2024-06-13
6 commits to main branch, last one 4 months ago
sail-sg
ptp sail-sg
4
149
apache-2.0
7
[CVPR2023] The code for 《Position-guided Text Prompt for Vision-Language Pre-training》
vlp cross-modality vision-language-pretraining
Created 2022-12-16
16 commits to main branch, last one about a year ago
Surrey-UP-Lab
RegionSpot Surrey-UP-Lab
4
118
other
1
Recognize Any Regions
zero-shot open-world auto-labeling open-vocabulary object-detection instance-segmentation vision-language-model vision-foundation-model vision-language-pretraining vision-language-foundation-model multimodal-representation-learning
Created 2023-10-30
11 commits to main branch, last one about a month ago
jusiro
FLAIR jusiro
10
86
apache-2.0
2
[MedIA'24] FLAIR: A Foundation LAnguage-Image model of the Retina for fundus image understanding.
medical-imaging foundation-models fundus-image-analysis vision-language-pretraining
Created 2023-07-17
25 commits to main branch, last one 6 months ago
ArrowLuo
SegCLIP ArrowLuo
8
83
unknown
9
PyTorch implementation of ICML 2023 paper "SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation"
open-vocabulary transfer-learning contrastive-learning semantic-segmentation vision-language-pretraining zero-shot-semantic-segmentation open-vocabulary-semantic-segmentation
Created 2022-11-11
8 commits to main branch, last one about a year ago
vgthengane
Continual-CLIP vgthengane
2
81
apache-2.0
5
Official repository for "CLIP model is an Efficient Continual Learner".
clip baseline continual-learning foundational-models vision-language-pretraining
Created 2022-10-03
5 commits to master branch, last one about a year ago
marslanm
Multimodality-Representation-Learning marslanm
7
68
unknown
7
This repository provides a comprehensive collection of research papers focused on multimodal representation learning, all of which have been cited and discussed in the survey just accepted https://dl....
cross-modal multimodal-pretext transformer-models multimodal-datasets multimodal-applications multimodal-deep-learning vision-language-pretraining multimodal-pre-trained-model
Created 2022-03-13
66 commits to main branch, last one about a year ago
HieuPhan33
CVPR2024_MAVL HieuPhan33
1
64
unknown
1
Multi-Aspect Vision Language Pretraining - CVPR2024
vision-language-model zero-shot-segmentation zero-shot-classification vision-language-pretraining medical-vision-and-language-pretraining
Created 2024-03-06
72 commits to main branch, last one 3 months ago
megvii-research
protoclip megvii-research
0
48
other
8
📍 Official pytorch implementation of paper "ProtoCLIP: Prototypical Contrastive Language Image Pretraining" (IEEE TNNLS)
contrastive-learning self-supervised-learning vision-language-pretraining
Created 2022-06-20
4 commits to main branch, last one about a year ago
Zoky-2020
SGA Zoky-2020
3
47
mit
4
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models. [ICCV 2023 Oral]
adversarial-attack vision-language-pretraining
Created 2023-07-30
7 commits to main branch, last one about a year ago
TXH-mercury
COSA TXH-mercury
3
39
mit
2
Codes and Models for COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
video-qa video-retrieval video-captioning video-language-pretrainng vision-language-pretraining
Created 2023-05-24
8 commits to master branch, last one about a year ago
TencentARC
FLM TencentARC
1
31
unknown
6
Accelerating Vision-Language Pretraining with Free Language Modeling (CVPR 2023)
language-modeling vision-language-pretraining
Created 2023-02-28
6 commits to master branch, last one about a year ago
alinlab
b2t alinlab
0
25
unknown
3
Bias-to-Text: Debiasing Unknown Visual Biases through Language Interpretation
explainable-ai bias-and-fairness vision-language-pretraining
Created 2023-01-31
4 commits to master branch, last one about a year ago