34 results found Sort:

hiyouga
LLaMA-Factory hiyouga
4.5k
36.5k
apache-2.0
218
Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)
ai gpt llm moe lora peft qwen rlhf agent llama qlora llama3 chatglm mistral fine-tuning quantization transformers language-model instruction-tuning large-language-models
Created 2023-05-28
2,493 commits to main branch, last one 12 hours ago
sgl-project
sglang sgl-project
586
6.6k
apache-2.0
61
SGLang is a fast serving framework for large language models and vision language models.
llm moe vlm cuda llama llava llama2 llama3 pytorch llama3-1 inference llm-serving transformer
Created 2024-01-08
1,526 commits to main branch, last one 2 hours ago
czy0729
Bangumi czy0729
135
3.8k
mit
25
:electron: An unofficial https://bgm.tv ui first app client for Android and iOS, built with React Native. 一个无广告、以爱好为驱动、不以盈利为目的、专门做 ACG 的类似豆瓣的追番记录，bgm.tv 第三方客户端。为移动端重新设计，内置大量加强的网页端难以实现的功能，且提供了相当的自定义选项。...
ios moe expo mobx react design android bangumi ios-app android-app react-native
Created 2019-05-08
2,711 commits to master branch, last one 3 days ago
PKU-YuanGroup
MoE-LLaVA PKU-YuanGroup
127
2.0k
apache-2.0
24
Mixture-of-Experts for Large Vision-Language Models
moe multi-modal mixture-of-experts large-vision-language-model
Created 2023-12-14
228 commits to main branch, last one 18 days ago
davidmrau
mixture-of-experts davidmrau
105
1.0k
gpl-3.0
6
PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
moe pytorch re-implementation mixture-of-experts sparsely-gated-mixture-of-experts
Created 2019-07-19
30 commits to master branch, last one 8 months ago
pjlab-sys4nlp
llama-moe pjlab-sys4nlp
47
896
apache-2.0
8
⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)
llm moe llama expert-partition mixture-of-experts continual-pre-training
Created 2023-07-24
212 commits to main branch, last one 5 months ago
sail-sg
Adan sail-sg
64
771
apache-2.0
7
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
mae moe vit adan gpt2 llms timm resnet fairseq pytorch convnext diffusion optimizer bert-model dreamfusion llm-training deep-learning transformer-xl cuda-programming artificial-intelligence
Created 2022-09-01
29 commits to main branch, last one 5 months ago
open-compass
MixtralKit open-compass
81
765
apache-2.0
8
A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI
llm moe mistral
Created 2023-12-09
54 commits to main branch, last one about a year ago
microsoft
Tutel microsoft
94
740
mit
15
Tutel MoE: An Optimized Mixture-of-Experts Implementation
moe nlp pytorch transformer mixture-of-experts
Created 2021-08-06
182 commits to main branch, last one 29 days ago
ymcui
Chinese-Mixtral ymcui
43
589
apache-2.0
15
中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）
32k 64k llm moe nlp mixtral mixture-of-experts large-language-models
Created 2024-01-11
31 commits to main branch, last one 7 months ago
mindspore-courses
step_into_llm mindspore-courses
100
435
apache-2.0
9
MindSpore online courses: Step into LLM
gpt llm moe nlp bert gpt2 peft rlhf llama llama2 chatglm chatgpt chatglm2 codegeex mindspore prompt-tuning instruction-tuning parallel-computing large-language-models natural-language-processing
Created 2023-03-21
126 commits to master branch, last one about a month ago
kokororin
pixiv.moe kokororin
44
362
mit
12
😘 A pinterest-style layout site, shows illusts on pixiv.net order by popularity.
moe comic pixiv react redux comics illust webapp illusts website lovelive typescript
Created 2016-09-05
405 commits to master branch, last one 2 years ago
LISTEN-moe
android-app LISTEN-moe
25
253
mit
9
Official LISTEN.moe Android app
moe jpop kpop anime japan music kotlin android android-auto music-player
Created 2017-03-26
1,747 commits to main branch, last one 6 days ago
inferflow
inferflow inferflow
24
236
mit
8
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
moe qwen bloom gemma phi-2 falcon llama2 m2m100 minicpm mistral mixtral deepseek internlm llamacpp baichuan2 llm-inference mixture-of-experts model-quantization multi-gpu-inference
Created 2023-12-26
116 commits to main branch, last one 9 months ago
libgdx
gdx-pay libgdx
84
224
apache-2.0
38
A libGDX cross-platform API for InApp purchasing.
iap ios moe java libgdx robovm android gdx-pay in-app-purchase multi-os-engine
Created 2014-10-06
538 commits to master branch, last one 4 months ago
IBM
ModuleFormer IBM
12
216
apache-2.0
11
ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language M...
lm moe
Created 2023-08-24
19 commits to main branch, last one 11 months ago
SkyworkAI
MoH SkyworkAI
7
191
apache-2.0
3
MoH: Multi-Head Attention as Mixture-of-Head Attention
dit moe vit llms attention transformer mixture-of-experts
Created 2024-10-08
19 commits to main branch, last one about a month ago
SkyworkAI
MoE-plus-plus SkyworkAI
4
166
apache-2.0
2
MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
moe llms mixture-of-experts large-language-models
Created 2024-10-08
11 commits to main branch, last one 2 months ago
LISTEN-moe
desktop-app LISTEN-moe
9
95
unknown
4
Official LISTEN.moe Desktop Client
app moe jpop anime linux macos music client listen desktop windows
Created 2018-11-28
125 commits to master branch, last one 4 years ago
kyegomez
MoE-Mamba kyegomez
5
89
mit
5
Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Zeta
ai ml moe swarms multi-modality multi-modal-fusion
Created 2024-01-21
14 commits to main branch, last one 11 months ago
simplifine-llm
Simplifine simplifine-llm
3
85
gpl-3.0
2
🚀 Easy, open-source LLM finetuning with one-line commands, seamless cloud integration, and popular optimization frameworks. ✨
ai gpt llm moe phi lora peft qwen cloud llama llama3 mistral fine-tuning open-source llm-training fine-tuning-llm finetuning-llms instruction-tuning large-language-models
Created 2024-07-05
189 commits to main branch, last one 4 months ago
xrsrke
pipegoose xrsrke
18
81
mit
4
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
moe zero-1 megatron megatron-lm transformers 3d-parallelism data-parallelism model-parallelism mixture-of-experts tensor-parallelism pipeline-parallelism sequence-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling
Created 2023-06-14
554 commits to main branch, last one about a year ago
phanirithvij
twist.moe phanirithvij
15
73
unlicense
7
Batch download high quality videos from https://twist.moe
moe anime twist-moe anime-downloader twist-moe-downloader
This repository has been archived (exclude archived)
Created 2018-01-06
150 commits to master branch, last one about a year ago
marisukukise
japReader marisukukise
5
72
gpl-3.0
4
japReader is an app for breaking down Japanese sentences and tracking vocabulary progress
moe anki ichi manga jmdict ichimoe electron furigana japanese flashcards javascript visual-novel japanese-study japanese-language language-learning progress-tracking japanese-dictionary language-learning-tool japanese-language-learners
Created 2022-07-25
260 commits to master branch, last one 7 months ago
OpenSparseLLMs
LLaMA-MoE-v2 OpenSparseLLMs
8
63
apache-2.0
2
🚀LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
moe sft llama llama3 sparsity attention fine-tuning instruction-tuning mixture-of-experts
Created 2024-11-26
272 commits to main branch, last one 18 days ago
kyegomez
SwitchTransformers kyegomez
10
61
mit
3
Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"
ai ml moe gpt4 llama multi-modal mixture-model mixture-of-models mixture-of-experts
Created 2024-01-22
11 commits to main branch, last one 11 months ago
LINs-lab
DynMoE LINs-lab
9
55
apache-2.0
11
[Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
moe language-model mixture-of-experts adaptive-computation multimodal-large-language-models
Created 2024-05-17
12 commits to main branch, last one 4 months ago
mifjpn
Moebuntu-SetupHelperScript2 mifjpn
0
45
mit
1
This repository has no description...
moe cute kawaii ubuntu moebuntu animation
Created 2022-12-13
39 commits to main branch, last one about a year ago
VITA-Group
Random-MoE-as-Dropout VITA-Group
2
45
apache-2.0
2
[ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal, Shiwei Liu, Zhangyang Wang
moe dropout transformer self-slimmable
Created 2023-02-18
2 commits to main branch, last one about a year ago
facebookresearch
AdaTT facebookresearch
5
42
mit
1
pytorch open-source library for the paper "AdaTT Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations"
moe mtl multitask-learning
Created 2023-06-03
3 commits to main branch, last one 4 months ago