Search Results - RepositoryStats

Awesome-Multimodal-Large-Language-Models BradyFU

810

12.7k

unknown

273

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

multi-modality chain-of-thought instruction-tuning in-context-learning instruction-following large-language-models visual-instruction-tuning large-vision-language-model multimodal-chain-of-thought large-vision-language-models multimodal-instruction-tuning multimodal-in-context-learning multimodal-large-language-models

Created 2023-05-19

728 commits to main branch, last one a day ago

MobileAgent X-PLUG

279

3.0k

mit

49

Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

app gui ios mllm agent gpt4v mobile android copilot harmony automation multimodal mobile-agents multimodal-agent multimodal-large-language-models

Created 2024-01-26

139 commits to main branch, last one about a month ago

modelscope-agent modelscope

311

2.7k

apache-2.0

38

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

llm rag code gpts qwen agent chatbot chatglm-4 open-gpts codexgraph assistantapi data-science mobile-agent multi-agents mobile-agents android-application data-science-assistant multimodal-large-language-models

Created 2023-08-03

474 commits to master branch, last one 7 days ago

LLaMA-Omni ictnlp

174

2.6k

apache-2.0

28

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

speech-to-text speech-to-speech speech-interaction large-language-models speech-language-model multimodal-large-language-models

Created 2024-09-10

13 commits to main branch, last one 7 days ago

cambrian cambrian-mllm

115

1.8k

apache-2.0

21

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

clip dino llms mllm chatbot computer-vision instruction-tuning large-language-models representation-learning multimodal-large-language-models

Created 2024-06-17

59 commits to main branch, last one 21 days ago

RPG-DiffusionMaster YangLing0818

99

1.7k

mit

25

[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (RPG)

text-to-image image-editting large-language-models multimodal-large-language-models

Created 2024-01-22

58 commits to main branch, last one about a month ago

mPLUG-DocOwl X-PLUG

101

1.6k

apache-2.0

31

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

mllm multimodal chart-understanding table-understanding document-understanding multimodal-large-language-models

Created 2023-07-04

134 commits to main branch, last one about a month ago

VITA VITA-MLLM

59

968

other

40

✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM

large-multimodal-models multimodal-large-language-models

Created 2024-08-10

65 commits to main branch, last one 28 days ago

Bunny BAAI-DCAI

69

933

apache-2.0

18

A family of lightweight multimodal models.

vlm mllm gpt-4 chatgpt chinese english multimodal-large-language-models

Created 2024-01-31

114 commits to main branch, last one 2 days ago

LLaVA-Plus-Codebase LLaVA-VL

53

706

apache-2.0

11

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

agent tool-use large-language-models large-multimodal-models multimodal-large-language-models

Created 2023-11-07

404 commits to main branch, last one 11 months ago

Woodpecker BradyFU

29

611

unknown

15

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.

llm mllm hallucination multimodality hallucinations large-language-models multimodal-large-language-models

Created 2023-09-26

106 commits to main branch, last one 5 months ago

SLAM-LLM X-LANCE

52

587

mit

21

Speech, Language, Audio, Music Processing with Large Language Model

peft audio-processing music-processing speech-processing large-language-model multimodal-large-language-models

Created 2023-10-23

696 commits to main branch, last one 3 days ago

awesome-multimodal-in-medical-imaging richard-peng-xia

60

552

mit

17

A collection of resources on applications of multi-modal learning in medical imaging.

medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models

Created 2022-07-13

151 commits to main branch, last one 9 days ago

MovieChat rese1f

41

530

bsd-3-clause

10

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

llama dataset computer-vision large-language-models long-video-understanding multimodal-large-language-models

Created 2023-06-26

107 commits to main branch, last one 22 days ago

Ovis AIDC-AI

31

527

apache-2.0

7

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

qwen llama3 chatbot multimodal multimodality vision-language-model vision-language-learning multimodal-large-language-models

Created 2024-06-13

31 commits to main branch, last one 16 days ago

VideoChat Henry-23

57

427

mit

7

实时语音交互数字人，支持端到端语音方案（GLM-4-Voice - THG）和级联方案（ASR-LLM-TTS-THG）。可自定义形象与音色，无须训练，支持音色克隆，首包延迟低至3s。Real-time voice interactive digital human, supporting end-to-end voice solutions (GLM-4-Voice - THG) and cas...

asr tts lip-sync musetalk real-time streaming end-to-end talking-head digital-human dialogue-systems gradio-python-app multimodal-large-language-models

Created 2024-10-18

40 commits to master branch, last one 6 days ago

Video-MME BradyFU

12

406

unknown

5

✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

mme video video-mme large-language-models large-vision-language-models multimodal-large-language-models

Created 2024-06-02

46 commits to main branch, last one 5 months ago

Awesome_Matching_Pretraining_Transfering Paranioar

47

401

mit

11

The Paper List of Large Multi-Modality Model, Parameter-Efficient Finetuning, Vision-Language Pretraining, Conventional Image-Text Matching for Preliminary Insight.

tutorial awesome-list image-text-matching large-vision-models vision-and-language image-text-retrieval video-text-retrieval cross-modal-retrieval large-language-models multimodal-pretraining video-text-recognition memory-efficient-tuning visual-semantic-embedding large-vision-language-models parameter-efficient-fine-tuning multimodal-large-language-models

Created 2020-12-22

129 commits to main branch, last one 4 months ago

Vitron SkyworkAI

24

386

unknown

14

NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

mllm segmentation multimodal-large-language-models

Created 2024-03-18

78 commits to main branch, last one about a month ago

MPP-LLaVA Coobiw

20

381

unknown

4

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...

mllm qwen deepspeed fine-tuning pretraining model-parallel pipeline-parallelism video-language-model video-large-language-models multimodal-large-language-models

Created 2023-10-24

133 commits to master branch, last one about a month ago

unicom deepglint

19

379

unknown

8

MLCD & UNICOM : Large-Scale Visual Representation Model

laion400m vision-transformer large-language-models large-sacle-pretrained-model embodied-artificial-intelligence multimodal-large-language-models

Created 2023-02-15

101 commits to main branch, last one 2 days ago

Awesome-LLMs-meet-Multimodal-Generation YingqingHe

20

362

unknown

14

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

llm aigc lvlm mllm text-to-3d multimodality text-to-audio text-to-image text-to-music text-to-sound text-to-video text-to-speech multimodal-models large-language-models multimodal-generation large-vision-language-models multimodal-large-language-models

Created 2023-11-17

346 commits to main branch, last one 6 days ago

Awesome-Multimodal-LLM HenryHZY

16

355

mit

17

Research Trends in LLM-guided Multimodal Learning.

llm multimodal instruction-tuning in-context-learning multimodal-learning large-language-models parameter-efficient-tuning parameter-efficient-learning multimodal-large-language-models

Created 2023-05-29

16 commits to main branch, last one about a year ago

pytorch_mgie tsujuifu

29

344

unknown

12

A Gradio demo of MGIE

pytorch iclr2024 image-editing vision-and-language multimodal-large-language-models

Created 2023-09-28

1 commits to main branch, last one 9 months ago

EVF-SAM hustvl

14

313

apache-2.0

7

Official code of "EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model"

multimodal segmentation segment-anything referring-image-segmentation multimodal-large-language-models

Created 2024-06-12

47 commits to main branch, last one 4 days ago

Youku-mPLUG X-PLUG

11

284

apache-2.0

5

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

mllm video youku chinese dataset benchmark multimodal video-retrieval multimodal-pretraining video-question-answering multimodal-large-language-models

Created 2023-06-06

18 commits to main branch, last one 10 months ago

Awesome-Medical-Large-Language-Models burglarhobbit

31

250

cc0-1.0

9

Curated papers on Large Language Models in Healthcare and Medical domain

large-language-models large-vision-language-models multimodal-large-language-models

Created 2023-06-28

45 commits to main branch, last one 4 months ago

Awesome-Multimodal-LLM-Autonomous-Driving IrohXu

12

239

mit

8

[WACV 2024 Survey Paper] Multimodal Large Language Models for Autonomous Driving

multimodal self-driving autonomous-car foundation-models autonomous-driving vision-transformer autonomous-vehicles large-language-models vision-language-model multimodal-large-language-models

Created 2023-10-22

90 commits to main branch, last one 8 months ago

EVE baaivision

3

231

mit

8

[NeurIPS'24 Spotlight] EVE: Encoder-Free Vision-Language Models

llm vlm clip mllm encoder-free-vlm instruction-following large-language-models vision-language-models multimodal-large-language-models

Created 2024-06-14

19 commits to main branch, last one about a month ago

lmms-finetune zjysteven

22

181

apache-2.0

5

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, qwen-vl, qwen2-vl, phi3-v etc.

llava qwen-vl finetuning llava-next multimodal vision-language foundation-models instruction-tuning large-language-model large-multimodal-models visual-instruction-tuning multimodal-large-language-models

Created 2024-07-20

95 commits to main branch, last one 29 days ago