Search Results - RepositoryStats

44

1.3k

unknown

32

[NeurIPS 2024] An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

gpt sora gpt-4v chatgpt text-to-video large-language-models large-multimodal-models large-video-language-models large-vision-language-models

Created 2024-06-06

44 commits to master branch, last one 2 months ago

VITA VITA-MLLM

63

1.0k

other

39

✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM

large-multimodal-models multimodal-large-language-models

Created 2024-08-10

99 commits to main branch, last one 17 hours ago

OpenAdapt OpenAdaptAI

148

1.0k

mit

9

Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models

Created 2023-04-12

925 commits to main branch, last one 12 days ago

LLaVA-Plus-Codebase LLaVA-VL

53

717

apache-2.0

12

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

agent tool-use large-language-models large-multimodal-models multimodal-large-language-models

Created 2023-11-07

404 commits to main branch, last one about a year ago

TinyLLaVA_Factory TinyLLaVA

73

690

apache-2.0

11

A Framework of Small-scale Large Multimodal Models

nlp llama llava tinyllama transformers vision-language large-multimodal-models

Created 2024-02-21

222 commits to main branch, last one 3 days ago

awesome-multimodal-in-medical-imaging richard-peng-xia

61

604

mit

18

A collection of resources on applications of multi-modal learning in medical imaging.

medical-imaging multimodal-learning large-language-models large-multimodal-models multimodal-deep-learning medical-report-generation visual-question-answering multimodal-large-language-models

Created 2022-07-13

151 commits to main branch, last one about a month ago

Open-LLaVA-NeXT xiaoachen98

20

401

unknown

13

An open-source implementation for training LLaVA-NeXT.

gpt-4 gpt4o llama llava llama3 chatbot chatgpt llava-next multimodal multi-modality vision-language-model large-multimodal-models visual-language-learning

Created 2024-05-11

36 commits to master branch, last one about a month ago

MMMU MMMU-Benchmark

29

370

apache-2.0

4

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

llm llms stem evaluation multimodal deep-learning multimodality computer-vision machine-learning foundation-models question-answering multimodal-learning deep-neural-networks large-language-models large-multimodal-models multimodal-deep-learning visual-question-answering natural-language-processing

Created 2023-11-23

131 commits to main branch, last one 11 days ago

OPERA shikiw

28

296

mit

3

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

gpt-4 llama chatbot chatgpt multimodal vision-language-model large-multimodal-models vision-language-learning

Created 2023-11-30

50 commits to main branch, last one 3 months ago

LEGENT thunlp

16

278

apache-2.0

11

Open Platform for Embodied Agents

embodied-ai physics-engine robot-simulator language-grounding large-multimodal-models

Created 2024-03-13

126 commits to main branch, last one 2 months ago

MixEval Psycoy

37

229

unknown

1

The official evaluation suite and dynamic data release for MixEval.

mixeval benchmark evaluation llm-inference llm-evaluation benchmark-mixture foundation-models benchmarking-suite evaluation-framework large-language-model large-language-models benchmarking-framework large-multimodal-models llm-evaluation-framework

Created 2024-06-01

120 commits to main branch, last one about a month ago

lmms-finetune zjysteven

24

208

apache-2.0

5

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

llava qwen-vl finetuning llava-next multimodal vision-language foundation-models instruction-tuning large-language-model large-multimodal-models visual-instruction-tuning multimodal-large-language-models

Created 2024-07-20

104 commits to main branch, last one 5 days ago

multi_token sshh12

12

176

apache-2.0

3

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

llm llava multimodal large-context multi-modality large-language-models vision-language-model large-multimodal-models

Created 2023-10-11

84 commits to main branch, last one 8 months ago

ShareGPT4V ShareGPT4Omni

4

175

unknown

3

[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions

gpt gpt4v gpt-4v chatgpt eccv2024 language-model instruction-tuning large-language-models vision-language-model large-multimodal-models large-vision-language-models

Created 2024-06-06

3 commits to master branch, last one 5 months ago

MMStar MMStar-Benchmark

5

156

unknown

1

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

llm llms lvlm lvlms evaluation multimodal multimodality multimodal-learning large-language-models large-multimodal-models visual-question-answering large-vision-language-model large-vision-language-models

Created 2024-03-29

19 commits to main branch, last one 2 months ago

Awesome-Multimodal-Papers friedrichor

9

111

mit

4

A curated list of awesome Multimodal studies.

multimodal deep-learning multimodal-data multimodal-dialogue multimodal-learning large-multimodal-models multimodal-deep-learning multimodal-large-language-models

Created 2024-04-05

60 commits to main branch, last one 3 days ago

Modality-Integration-Rate shikiw

3

89

mit

3

The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".

llama llava gpt-4o chatbot multimodal vision-language-model large-multimodal-models vision-language-learning

Created 2024-10-09

17 commits to main branch, last one 24 days ago

BenchLMM AIFEG

6

84

apache-2.0

0

[ECCV 2024] BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models

cv dataset benchmark large-language-models large-multimodal-models

Created 2023-11-20

93 commits to main branch, last one 4 months ago

apiprompting yu-rp

3

54

mit

2

[ECCV 2024] API: Attention Prompting on Image for Large Vision-Language Models

prompting visual-prompting vision-language-model vision-language-models large-multimodal-models large-vision-language-model large-vision-language-models

Created 2024-09-04

14 commits to master branch, last one 2 months ago

VisualWebBench VisualWebBench

1

47

unknown

3

Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"

llm llms mllm evaluation multimodal deep-learning computer-vision machine-learning foundation-models question-answering large-language-models large-multimodal-models multimodal-deep-learning visual-question-answering natural-language-processing multimodal-large-language-models

Created 2024-04-02

26 commits to main branch, last one 2 months ago

TextCoT bzluan

4

37

unknown

3

The official repo for “TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding”.

chain-of-thought large-multimodal-models

Created 2024-04-15

20 commits to main branch, last one 2 months ago

Inner-Adaptor-Architecture 360CVGroup

4

34

apache-2.0

0

LMM which strictly superset LLM embedded

lmm large-multimodal-models

Created 2024-08-23

5 commits to main branch, last one about a month ago

MMRole YanqiDai

2

33

mit

1

A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agents

role-playing large-multimodal-models

Created 2024-07-26

15 commits to main branch, last one about a month ago

LLaVA-UHD-Better ParadoxZW

3

32

apache-2.0

4

A bug-free and improved implementation of LLaVA-UHD, based on the code from the official repo

llava multimodal large-language-models large-multimodal-models

Created 2024-06-12

11 commits to main branch, last one 5 months ago

Camel-Bench mbzuai-oryx

1

30

mit

3

CAMEL-Bench is an Arabic benchmark for evaluating multimodal models across eight domains with 29,000 questions.

vqa arabic mbzuai benchmark multimodal-learning large-multimodal-models visual-question-answering

Created 2024-10-23

21 commits to main branch, last one 14 days ago

IVM 2toinf

2

30

apache-2.0

2

[NeurIPS-2024] The offical Implementation of "Instruction-Guided Visual Masking"

robotics multimodal deep-learning computer-vision instruction-tuning instruction-following large-language-models pytorch-implementation large-multimodal-models

Created 2024-04-17

71 commits to master branch, last one about a month ago

MileBench MileBench

1

28

apache-2.0

4

This repo contains evaluation code for the paper "MileBench: Benchmarking MLLMs in Long Context"

llm llms benchmark evaluation multimodal deep-learning multimodality computer-vision machine-learning foundation-models deep-neural-networks large-language-models long-context-modeling large-multimodal-models long-context-transformers visual-question-answering natural-language-processing multimodal-large-language-models

Created 2024-04-12

17 commits to main branch, last one 5 months ago

Multi-Modal-Large-Language-Learning zchoi

0

25

unknown

3

Awesome multi-modal large language paper/project, collections of popular training strategies, e.g., PEFT, LoRA.

awesome benchmark multimodal pre-training foundation-models large-language-models large-multimodal-models

Created 2023-06-13

29 commits to main branch, last one 4 months ago