Search Results - RepositoryStats

Awesome-Multimodal-Large-Language-Models BradyFU

839

13.2k

unknown

256

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

multi-modality chain-of-thought instruction-tuning in-context-learning instruction-following large-language-models visual-instruction-tuning large-vision-language-model multimodal-chain-of-thought large-vision-language-models multimodal-instruction-tuning multimodal-in-context-learning multimodal-large-language-models

Created 2023-05-19

782 commits to main branch, last one 8 hours ago

MobileAgent X-PLUG

300

3.2k

mit

53

Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

app gui ios mllm agent gpt4v mobile android copilot harmony automation multimodal mobile-agents multimodal-agent multimodal-large-language-models

Created 2024-01-26

139 commits to main branch, last one 2 months ago

modelscope-agent modelscope

321

2.8k

apache-2.0

39

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

llm rag code gpts qwen agent chatbot chatglm-4 open-gpts codexgraph assistantapi data-science mobile-agent multi-agents mobile-agents android-application data-science-assistant multimodal-large-language-models

Created 2023-08-03

474 commits to master branch, last one about a month ago

LLaMA-Omni ictnlp

185

2.7k

apache-2.0

29

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

speech-to-text speech-to-speech speech-interaction large-language-models speech-language-model multimodal-large-language-models

Created 2024-09-10

13 commits to main branch, last one about a month ago

mPLUG-DocOwl X-PLUG

115

2.0k

apache-2.0

33

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

mllm multimodal chart-understanding table-understanding document-understanding multimodal-large-language-models

Created 2023-07-04

134 commits to main branch, last one 2 months ago

cambrian cambrian-mllm

118

1.8k

apache-2.0

23

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

clip dino llms mllm chatbot computer-vision instruction-tuning large-language-models representation-learning multimodal-large-language-models

Created 2024-06-17

59 commits to main branch, last one about a month ago

RPG-DiffusionMaster YangLing0818

101

1.7k

mit

26

[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (RPG)

text-to-image image-editting large-language-models multimodal-large-language-models

Created 2024-01-22

58 commits to main branch, last one 2 months ago

VITA VITA-MLLM

63

1.0k

other

39

✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM

large-multimodal-models multimodal-large-language-models

Created 2024-08-10

99 commits to main branch, last one 8 hours ago

Bunny BAAI-DCAI

71

962

apache-2.0

20

A family of lightweight multimodal models.

vlm mllm gpt-4 chatgpt chinese english multimodal-large-language-models

Created 2024-01-31

114 commits to main branch, last one about a month ago

LLaVA-Plus-Codebase LLaVA-VL

53

717

apache-2.0

12

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

agent tool-use large-language-models large-multimodal-models multimodal-large-language-models

Created 2023-11-07

404 commits to main branch, last one about a year ago

Woodpecker BradyFU

29

619

unknown

15

✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models. The first work to correct hallucinations in MLLMs.

llm mllm hallucination multimodality hallucinations large-language-models multimodal-large-language-models

Created 2023-09-26

106 commits to main branch, last one 6 months ago

SLAM-LLM X-LANCE

56

617

mit

21

Speech, Language, Audio, Music Processing with Large Language Model

peft audio-processing music-processing speech-processing large-language-model multimodal-large-language-models

Created 2023-10-23

706 commits to main branch, last one 21 days ago

awesome-multimodal-in-medical-imaging richard-peng-xia

61

604

mit

18

A collection of resources on applications of multi-modal learning in medical imaging.

medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models

Created 2022-07-13

151 commits to main branch, last one about a month ago

Ovis AIDC-AI

33

584

apache-2.0

7

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

qwen llama3 chatbot multimodal multimodality vision-language-model vision-language-learning multimodal-large-language-models

Created 2024-06-13

32 commits to main branch, last one 25 days ago

MovieChat rese1f

42

553

bsd-3-clause

11

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

llama dataset computer-vision large-language-models long-video-understanding multimodal-large-language-models

Created 2023-06-26

120 commits to main branch, last one 3 days ago

VideoChat Henry-23

72

551

mit

9

实时语音交互数字人，支持端到端语音方案（GLM-4-Voice - THG）和级联方案（ASR-LLM-TTS-THG）。可自定义形象与音色，无须训练，支持音色克隆，首包延迟低至3s。Real-time voice interactive digital human, supporting end-to-end voice solutions (GLM-4-Voice - THG) and cas...

asr tts lip-sync musetalk real-time streaming end-to-end talking-head digital-human dialogue-systems gradio-python-app multimodal-large-language-models

Created 2024-10-18

40 commits to master branch, last one about a month ago

unicom deepglint

20

451

unknown

8

MLCD & UNICOM : Large-Scale Visual Representation Model

laion400m vision-transformer large-language-models large-sacle-pretrained-model embodied-artificial-intelligence multimodal-large-language-models

Created 2023-02-15

101 commits to main branch, last one about a month ago

Video-MME BradyFU

18

424

unknown

5

✨✨Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

mme video video-mme large-language-models large-vision-language-models multimodal-large-language-models

Created 2024-06-02

50 commits to main branch, last one 7 days ago

Vitron SkyworkAI

24

414

unknown

14

NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

mllm segmentation multimodal-large-language-models

Created 2024-03-18

78 commits to main branch, last one 2 months ago

Awesome_Matching_Pretraining_Transfering Paranioar

47

405

mit

11

The Paper List of Large Multi-Modality Model (Perception, Generation, Unification), Parameter-Efficient Finetuning, Vision-Language Pretraining, Conventional Image-Text Matching for Preliminary Insigh...

tutorial awesome-list image-text-matching large-vision-models vision-and-language image-text-retrieval large-language-model video-text-retrieval cross-modal-retrieval large-language-models multimodal-pretraining video-text-recognition memory-efficient-tuning text-to-image-synthesis text-to-image-generation text-to-video-generation visual-semantic-embedding large-vision-language-models parameter-efficient-fine-tuning multimodal-large-language-models

Created 2020-12-22

130 commits to main branch, last one 6 days ago

MPP-LLaVA Coobiw

20

390

unknown

4

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...

mllm qwen deepspeed fine-tuning pretraining model-parallel pipeline-parallelism video-language-model video-large-language-models multimodal-large-language-models

Created 2023-10-24

134 commits to master branch, last one 11 days ago

Awesome-LLMs-meet-Multimodal-Generation YingqingHe

22

389

unknown

14

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

llm aigc lvlm mllm text-to-3d multimodality text-to-audio text-to-image text-to-music text-to-sound text-to-video text-to-speech multimodal-models large-language-models multimodal-generation large-vision-language-models multimodal-large-language-models

Created 2023-11-17

346 commits to main branch, last one about a month ago

Awesome-Multimodal-LLM HenryHZY

16

356

mit

17

Research Trends in LLM-guided Multimodal Learning.

llm multimodal instruction-tuning in-context-learning multimodal-learning large-language-models parameter-efficient-tuning parameter-efficient-learning multimodal-large-language-models

Created 2023-05-29

16 commits to main branch, last one about a year ago

pytorch_mgie tsujuifu

30

345

unknown

12

A Gradio demo of MGIE

pytorch iclr2024 image-editing vision-and-language multimodal-large-language-models

Created 2023-09-28

1 commits to main branch, last one 10 months ago

EVF-SAM hustvl

15

327

apache-2.0

7

Official code of "EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model"

multimodal segmentation segment-anything referring-image-segmentation multimodal-large-language-models

Created 2024-06-12

47 commits to main branch, last one about a month ago

Youku-mPLUG X-PLUG

11

286

apache-2.0

5

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

mllm video youku chinese dataset benchmark multimodal video-retrieval multimodal-pretraining video-question-answering multimodal-large-language-models

Created 2023-06-06

18 commits to main branch, last one 11 months ago