42 results found Sort:

BradyFU
Awesome-Multimodal-Large-Language-Models BradyFU
691
10.3k
unknown
238
:sparkles::sparkles:Latest Advances on Multimodal Large Language Models
multi-modality chain-of-thought instruction-tuning in-context-learning instruction-following large-language-models visual-chain-of-thought visual-instruction-tuning visual-in-context-learning large-vision-language-model multimodal-chain-of-thought large-vision-language-models multimodal-instruction-tuning multimodal-in-context-learning multimodal-large-language-models
Created 2023-05-19
645 commits to main branch, last one a day ago
X-PLUG
MobileAgent X-PLUG
187
2.3k
mit
35
Mobile-Agent: The Powerful Mobile Device Operation Assistant Family
app gui ios mllm agent gpt4v mobile android copilot harmony automation multimodal mobile-agents multimodal-agent multimodal-large-language-models
Created 2024-01-26
98 commits to main branch, last one 2 days ago
modelscope
modelscope-agent modelscope
244
2.2k
apache-2.0
30
ModelScope-Agent: An agent framework connecting models in ModelScope with the world
llm rag gpts qwen agent chatbot chatglm-4 open-gpts assistantapi mobile-agent multi-agents mobile-agents android-application multimodal-large-language-models
Created 2023-08-03
368 commits to master branch, last one a day ago
YangLing0818
RPG-DiffusionMaster YangLing0818
90
1.6k
unknown
26
[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (PRG)
text-to-image image-editting large-language-models multimodal-large-language-models
Created 2024-01-22
50 commits to main branch, last one 22 days ago
X-PLUG
mPLUG-DocOwl X-PLUG
66
1.1k
apache-2.0
27
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding
mllm multimodal chart-understanding table-understanding document-understanding multimodal-large-language-models
Created 2023-07-04
117 commits to main branch, last one 25 days ago
cambrian-mllm
cambrian cambrian-mllm
51
942
apache-2.0
16
Cambrian-1 is a family of multimodal LLMs with a vision-centric design.
clip dino llms mllm chatbot computer-vision instruction-tuning large-language-models representation-learning multimodal-large-language-models
Created 2024-06-17
24 commits to main branch, last one 4 days ago
BAAI-DCAI
Bunny BAAI-DCAI
56
771
apache-2.0
21
A family of lightweight multimodal models.
vlm mllm gpt-4 chatgpt chinese english multimodal-large-language-models
Created 2024-01-31
102 commits to main branch, last one 4 days ago
LLaVA-VL
LLaVA-Plus-Codebase LLaVA-VL
50
652
apache-2.0
11
LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
agent tool-use large-language-models large-multimodal-models multimodal-large-language-models
Created 2023-11-07
404 commits to main branch, last one 6 months ago
BradyFU
Woodpecker BradyFU
29
571
unknown
15
✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.
llm mllm hallucination multimodality hallucinations large-language-models multimodal-large-language-models
Created 2023-09-26
106 commits to main branch, last one 11 days ago
rese1f
MovieChat rese1f
37
449
bsd-3-clause
10
[CVPR 2024] 🎬💭 chat with over 10K frames of video!
llama dataset computer-vision large-language-models long-video-understanding multimodal-large-language-models
Created 2023-06-26
93 commits to main branch, last one 13 days ago
X-LANCE
SLAM-LLM X-LANCE
29
385
mit
19
Speech, Language, Audio, Music Processing with Large Language Model
peft audio-processing music-processing speech-processing large-language-model multimodal-large-language-models
Created 2023-10-23
612 commits to main branch, last one 17 days ago
richard-peng-xia
awesome-multimodal-in-medical-imaging richard-peng-xia
38
375
mit
14
A collection of resources on applications of multi-modal learning in medical imaging.
medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models
Created 2022-07-13
130 commits to main branch, last one 6 days ago
HenryHZY
Awesome-Multimodal-LLM HenryHZY
16
336
mit
16
Research Trends in LLM-guided Multimodal Learning.
llm multimodal instruction-tuning in-context-learning multimodal-learning large-language-models parameter-efficient-tuning parameter-efficient-learning multimodal-large-language-models
Created 2023-05-29
16 commits to main branch, last one 8 months ago
tsujuifu
pytorch_mgie tsujuifu
28
336
unknown
12
A Gradio demo of MGIE
pytorch iclr2024 image-editing vision-and-language multimodal-large-language-models
Created 2023-09-28
1 commits to main branch, last one 4 months ago
Coobiw
MiniGPT4Qwen Coobiw
14
287
unknown
4
Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...
mllm qwen deepspeed fine-tuning pretraining model-parallel pipeline-parallelism video-language-model video-large-language-models multimodal-large-language-models
Created 2023-10-24
129 commits to master branch, last one 2 days ago
BradyFU
Video-MME BradyFU
8
278
unknown
5
✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
mme video video-mme large-language-models large-vision-language-models multimodal-large-language-models
Created 2024-06-02
46 commits to main branch, last one 11 days ago
X-PLUG
Youku-mPLUG X-PLUG
11
267
apache-2.0
5
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks
mllm video youku chinese dataset benchmark multimodal video-retrieval multimodal-pretraining video-question-answering multimodal-large-language-models
Created 2023-06-06
18 commits to main branch, last one 5 months ago
YingqingHe
Awesome-LLMs-meet-Multimodal-Generation YingqingHe
14
194
unknown
8
🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).
aigc text-to-3d multimodality text-to-audio text-to-image text-to-music text-to-sound text-to-video text-to-speech multimodal-models large-language-models multimodal-generation large-vision-language-models multimodal-large-language-models
Created 2023-11-17
325 commits to main branch, last one 23 hours ago
IrohXu
Awesome-Multimodal-LLM-Autonomous-Driving IrohXu
7
167
mit
6
[WACV 2024 Survey Paper] Multimodal Large Language Models for Autonomous Driving
multimodal self-driving autonomous-car foundation-models autonomous-driving vision-transformer autonomous-vehicles large-language-models vision-language-model multimodal-large-language-models
Created 2023-10-22
90 commits to main branch, last one 3 months ago
burglarhobbit
Awesome-Medical-Large-Language-Models burglarhobbit
12
151
cc0-1.0
7
Curated papers on Large Language Models in Healthcare and Medical domain
large-language-models large-vision-language-models multimodal-large-language-models
Created 2023-06-28
35 commits to main branch, last one 13 days ago
AviSoori1x
seemore AviSoori1x
8
122
mit
4
From scratch implementation of a vision language model in pure PyTorch
llm llava pytorch multimodal deep-learning neural-networks multimodal-learning large-language-models vision-language-model pytorch-implementation artificial-intelligence multimodal-large-language-models
Created 2024-04-17
47 commits to main branch, last one about a month ago
zjukg
KoPA zjukg
8
111
mit
3
[Paper][Preprint 2023] Making Large Language Models Perform Better in Knowledge Graph Completion
multi-modal knowledge-graph instruction-tuning large-language-models knowledge-graph-completion knowledge-graph-embeddings multimodal-large-language-models
Created 2023-10-10
32 commits to main branch, last one 4 months ago
baaivision
EVE baaivision
0
65
mit
9
The code of “Unveiling Encoder-Free Vision-Language Models“
llm vlm clip mllm chatbot computer-vision encoder-free-vlm instruction-following large-language-models vision-language-models multimodal-large-language-models
Created 2024-06-14
8 commits to main branch, last one 6 days ago
ChenDelong1999
polite-flamingo ChenDelong1999
3
63
unknown
5
🦩 Visual Instruction Tuning with Polite Flamingo - training multi-modal LLMs to be both clever and polite! (AAAI-24 Oral)
large-language-models visual-instruction-tuning multimodal-large-language-models
Created 2023-07-02
35 commits to main branch, last one 6 months ago
X-PLUG
mPLUG-HalOwl X-PLUG
1
62
mit
1
mPLUG-HalOwl: Multimodal Hallucination Evaluation and Mitigating
mllm benchmark hallucinations contrastive-learning multimodal-hallucination multimodal-large-language-models
Created 2024-01-18
14 commits to main branch, last one 5 months ago
vincentlux
Awesome-Multimodal-LLM vincentlux
7
58
mit
3
Reading list for Multimodal Large Language Models
paper-list awesome-list computer-vision machine-learning large-language-models vision-language-model multimodal-machine-learning natural-language-processing multimodal-large-language-models
Created 2023-06-06
39 commits to main branch, last one 10 months ago
invictus717
MiCo invictus717
3
58
apache-2.0
2
Explore the Limits of Omni-modal Pretraining at Scale
scale-up omnimodal multimodal pretraining deep-learning multimodal-large-language-models
Created 2024-06-11
4 commits to master branch, last one a day ago
mu-cai
matryoshka-mm mu-cai
2
54
apache-2.0
3
Matryoshka Multimodal Models
chatb llama llava llava-next matryoshka multimodal multimodal-large-language-models
Created 2024-05-27
466 commits to main branch, last one 25 days ago
OpenGVLab
MM-NIAH OpenGVLab
4
46
unknown
0
This is the official implementation of the paper "Needle In A Multimodal Haystack"
benchmark long-context vision-language-model multimodal-large-language-models
Created 2024-06-05
40 commits to main branch, last one 9 days ago
OpenKG-ORG
EasyDetect OpenKG-ORG
3
41
apache-2.0
4
An Easy-to-use Hallucination Detection Framework for LLMs.
aigc easydetect generation genrative-ai hallucinations knowledge-graph large-language-models hallucination-detection natural-language-processing multimodal-large-language-models
Created 2023-12-31
56 commits to main branch, last one 2 months ago