Search Results - RepositoryStats

44

1.3k

unknown

32

[NeurIPS 2024] An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

gpt sora gpt-4v chatgpt text-to-video large-language-models large-multimodal-models large-video-language-models large-vision-language-models

Created 2024-06-06

44 commits to master branch, last one about a month ago

OpenAdapt OpenAdaptAI

138

992

mit

8

Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models

Created 2023-04-12

923 commits to main branch, last one 5 days ago

VITA VITA-MLLM

59

968

other

40

✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM

large-multimodal-models multimodal-large-language-models

Created 2024-08-10

65 commits to main branch, last one 28 days ago

LLaVA-Plus-Codebase LLaVA-VL

53

706

apache-2.0

11

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

agent tool-use large-language-models large-multimodal-models multimodal-large-language-models

Created 2023-11-07

404 commits to main branch, last one 11 months ago

TinyLLaVA_Factory TinyLLaVA

68

657

apache-2.0

12

A Framework of Small-scale Large Multimodal Models

nlp llama llava tinyllama transformers vision-language large-multimodal-models

Created 2024-02-21

212 commits to main branch, last one 2 days ago

awesome-multimodal-in-medical-imaging richard-peng-xia

60

556

mit

17

A collection of resources on applications of multi-modal learning in medical imaging.

medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models

Created 2022-07-13

151 commits to main branch, last one 9 days ago

Open-LLaVA-NeXT xiaoachen98

20

395

unknown

13

An open-source implementation for training LLaVA-NeXT.

gpt-4 gpt4o llama llava llama3 chatbot chatgpt llava-next multimodal multi-modality vision-language-model large-multimodal-models visual-language-learning

Created 2024-05-11

36 commits to master branch, last one 29 days ago

MMMU MMMU-Benchmark

25

359

apache-2.0

4

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

llm llms stem evaluation multimodal deep-learning multimodality computer-vision machine-learning foundation-models question-answering multimodal-learning deep-neural-networks large-language-models large-multimodal-models multimodal-deep-learning visual-question-answering natural-language-processing

Created 2023-11-23

127 commits to main branch, last one 19 hours ago

OPERA shikiw

26

288

mit

2

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

gpt-4 llama chatbot chatgpt multimodal vision-language-model large-multimodal-models vision-language-learning

Created 2023-11-30

50 commits to main branch, last one 2 months ago

LEGENT thunlp

15

270

apache-2.0

11

Open Platform for Embodied Agents

embodied-ai physics-engine robot-simulator language-grounding large-multimodal-models

Created 2024-03-13

126 commits to main branch, last one about a month ago

MixEval Psycoy

35

224

unknown

1

The official evaluation suite and dynamic data release for MixEval.

mixeval benchmark evaluation llm-inference llm-evaluation benchmark-mixture foundation-models benchmarking-suite evaluation-framework large-language-model large-language-models benchmarking-framework large-multimodal-models llm-evaluation-framework

Created 2024-06-01

120 commits to main branch, last one 11 days ago

lmms-finetune zjysteven

22

181

apache-2.0

5

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, qwen-vl, qwen2-vl, phi3-v etc.

llava qwen-vl finetuning llava-next multimodal vision-language foundation-models instruction-tuning large-language-model large-multimodal-models visual-instruction-tuning multimodal-large-language-models

Created 2024-07-20

95 commits to main branch, last one 29 days ago

multi_token sshh12

12

176

apache-2.0

3

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

llm llava multimodal large-context multi-modality large-language-models vision-language-model large-multimodal-models

Created 2023-10-11

84 commits to main branch, last one 7 months ago

ShareGPT4V ShareGPT4Omni

4

157

unknown

3

[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions

gpt gpt4v gpt-4v chatgpt eccv2024 language-model instruction-tuning large-language-models vision-language-model large-multimodal-models large-vision-language-models

Created 2024-06-06

3 commits to master branch, last one 4 months ago

MMStar MMStar-Benchmark

5

149

unknown

1

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

llm llms lvlm lvlms evaluation multimodal multimodality multimodal-learning large-language-models large-multimodal-models visual-question-answering large-vision-language-model large-vision-language-models

Created 2024-03-29

19 commits to main branch, last one about a month ago

Awesome-Multimodal-Papers friedrichor

8

97

mit

4

A curated list of awesome Multimodal studies.

multimodal deep-learning multimodal-data multimodal-dialogue multimodal-learning large-multimodal-models multimodal-deep-learning multimodal-large-language-models

Created 2024-04-05

55 commits to main branch, last one 18 days ago

Modality-Integration-Rate shikiw

1

82

mit

2

The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".

llama llava gpt-4o chatbot multimodal vision-language-model large-multimodal-models vision-language-learning

Created 2024-10-09

15 commits to main branch, last one 24 days ago

BenchLMM AIFEG

6

82

apache-2.0

0

[ECCV 2024] BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models

cv dataset benchmark large-language-models large-multimodal-models

Created 2023-11-20

93 commits to main branch, last one 3 months ago

apiprompting yu-rp

3

48

mit

2

[ECCV 2024] API: Attention Prompting on Image for Large Vision-Language Models

prompting visual-prompting vision-language-model vision-language-models large-multimodal-models large-vision-language-model large-vision-language-models

Created 2024-09-04

14 commits to master branch, last one about a month ago

VisualWebBench VisualWebBench

1

47

unknown

3

Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"

llm llms mllm evaluation multimodal deep-learning computer-vision machine-learning foundation-models question-answering large-language-models large-multimodal-models multimodal-deep-learning visual-question-answering natural-language-processing multimodal-large-language-models

Created 2024-04-02

26 commits to main branch, last one about a month ago

TextCoT bzluan

3

34

unknown

3

The official repo for “TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding”.

chain-of-thought large-multimodal-models

Created 2024-04-15

20 commits to main branch, last one about a month ago

LLaVA-UHD-Better ParadoxZW

3

31

apache-2.0

4

A bug-free and improved implementation of LLaVA-UHD, based on the code from the official repo

llava multimodal large-language-models large-multimodal-models

Created 2024-06-12

11 commits to main branch, last one 4 months ago

MMRole YanqiDai

2

31

mit

1

A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents

role-playing large-multimodal-models

Created 2024-07-26

15 commits to main branch, last one 3 days ago

Inner-Adaptor-Architecture 360CVGroup

4

30

apache-2.0

0

LMM which strictly superset LLM embedded

lmm large-multimodal-models

Created 2024-08-23

5 commits to main branch, last one 16 days ago

IVM 2toinf

2

29

apache-2.0

2

[NeurIPS-2024] The offical Implementation of "Instruction-Guided Visual Masking"

robotics multimodal deep-learning computer-vision instruction-tuning instruction-following large-language-models pytorch-implementation large-multimodal-models

Created 2024-04-17

71 commits to master branch, last one 6 days ago

MileBench MileBench

1

26

apache-2.0

4

This repo contains evaluation code for the paper "MileBench: Benchmarking MLLMs in Long Context"

llm llms benchmark evaluation multimodal deep-learning multimodality computer-vision machine-learning foundation-models deep-neural-networks large-language-models long-context-modeling large-multimodal-models long-context-transformers visual-question-answering natural-language-processing multimodal-large-language-models

Created 2024-04-12

17 commits to main branch, last one 4 months ago