27 results found Sort:

InternLM
lmdeploy InternLM
428
4.7k
apache-2.0
38
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
llm llama llama2 llama3 internlm codellama deepspeed turbomind cuda-kernels llm-inference fastertransformer
Created 2023-06-15
1,012 commits to main branch, last one 2 days ago
OpenRLHF
OpenRLHF OpenRLHF
250
2.7k
apache-2.0
24
An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention)
vllm raylib deepspeed transformers large-language-models reinforcement-learning reinforcement-learning-from-human-feedback
Created 2023-07-30
952 commits to main branch, last one 4 days ago
PKU-Alignment
safe-rlhf PKU-Alignment
120
1.4k
apache-2.0
18
Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback
gpt llm llms rlhf llama alpaca beaver safety vicuna datasets ai-safety deepspeed safe-rlhf transformer transformers large-language-models reinforcement-learning safe-reinforcement-learning reinforcement-learning-from-human-feedback safe-reinforcement-learning-from-human-feedback
Created 2023-05-15
111 commits to main branch, last one 5 months ago
zjunlp
KnowLM zjunlp
125
1.2k
mit
11
An Open-sourced Knowledgable Large Language Model Framework.
lora gpt-3 llama knowlm models chinese english bilingual deepspeed reasoning instructie instructions pre-training deep-learning language-model pre-trained-model instruction-tuning instruction-following large-language-models pre-trained-language-models
Created 2023-04-01
537 commits to main branch, last one 4 months ago
Xirider
finetune-gpt2xl Xirider
73
433
mit
5
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and finetune GPT-NEO (2.7 B) on a single GPU with Huggingface Transformers using DeepSpeed
gpt2 gpt3 gpt-neo deepspeed finetuning huggingface gpt-neo-fine-tuning huggingface-transformers
Created 2021-03-26
118 commits to main branch, last one 2 years ago
shm007g
LLaMA-Cult-and-More shm007g
24
427
mit
34
Large Language Models for All, 🦙 Cult and More, Stay in touch !
gpt llm ggml gpt4 gptq llama alpaca vicuna chatgpt loralib pytorch deepspeed tensorflow transformers
Created 2023-03-30
27 commits to main branch, last one about a year ago
OpenMOSS
CoLLiE OpenMOSS
58
411
apache-2.0
10
Collaborative Training of Large Language Models in an Efficient Way
nlp pytorch deepspeed deep-learning
Created 2023-04-02
980 commits to main branch, last one 2 months ago
Coobiw
MPP-LLaVA Coobiw
20
381
unknown
4
Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...
mllm qwen deepspeed fine-tuning pretraining model-parallel pipeline-parallelism video-language-model video-large-language-models multimodal-large-language-models
Created 2023-10-24
133 commits to master branch, last one about a month ago
intelligent-machine-learning
glake intelligent-machine-learning
33
379
apache-2.0
7
GLake: optimizing GPU memory management and IO transmission.
gpu llm onnx memory pytorch deepspeed
Created 2023-06-06
53 commits to master branch, last one 3 months ago
sunzeyeah
RLHF sunzeyeah
36
287
unknown
8
Implementation of Chinese ChatGPT
glm nlp pangu chatgpt pytorch deepspeed deep-learning
Created 2023-02-17
371 commits to master branch, last one about a year ago
LambdaLabsML
distributed-training-guide LambdaLabsML
19
287
mit
7
Best practices & guides on how to write distributed pytorch training code
gpu mpi cuda fsdp nccl slurm cluster pytorch sharding deepspeed kuberentes lambdalabs gpu-cluster distributed-training
Created 2024-07-31
230 commits to main branch, last one 18 hours ago
stanleylsx
llms_tool stanleylsx
19
203
apache-2.0
4
一个基于HuggingFace开发的大语言模型训练、测试工具。支持各模型的webui、终端预测，低参数量及全参数模型训练(预训练、SFT、RM、PPO、DPO)和融合、量化。
moss qwen bloom llama aquila falcon llama2 xverse aquila2 chatglm mistral pytorch baichuan chatglm2 chatglm3 internlm baichuan2 deepspeed
Created 2023-07-20
268 commits to main branch, last one 11 months ago
git-cloner
llama2-lora-fine-tuning git-cloner
14
167
mit
3
llama2 finetuning with deepspeed and lora
lora llama2 deepspeed finetuning
Created 2023-07-21
8 commits to main branch, last one about a year ago
jackaduma
ChatGLM-LoRA-RLHF-PyTorch jackaduma
10
126
mit
6
A full pipeline to finetune ChatGLM LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the ChatGLM architecture. Basically ChatG...
gpt llm ppo lora peft rlhf llama chatglm chatgpt pytorch finetune deepspeed chatglm-6b reward-models
Created 2023-04-18
17 commits to main branch, last one about a year ago
HomebrewNLP
revlib HomebrewNLP
6
124
bsd-2-clause
4
Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload
tpu xla revnet pytorch deepspeed momentumnet deep-learning
Created 2021-08-17
97 commits to master branch, last one 2 years ago
openpsi-project
ReaLHF openpsi-project
5
120
apache-2.0
3
Super-Efficient RLHF Training of LLMs with Parameter Reallocation
llm deepspeed megatron-lm llm-training transformers llm-framework distributed-systems distributed-computing large-language-models reinforcement-learning large-scale-machine-learning reinforcement-learning-from-human-feedback
Created 2024-06-18
1,071 commits to main branch, last one a day ago
bobo0810
LearnDeepSpeed bobo0810
1
117
mit
1
DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）
examples deepspeed large-language-models
Created 2023-08-11
16 commits to main branch, last one about a year ago
CoinCheung
gdGPT CoinCheung
8
90
apache-2.0
1
Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
llm nlp bloom llama2 pytorch pipeline deepspeed chatglm3-6b baichuan2-7b mixtral-8x7b full-finetune flash-attention model-parallization
Created 2023-06-24
27 commits to master branch, last one 9 months ago
OpenCSGs
llm-inference OpenCSGs
16
70
apache-2.0
13
llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource...
ray vllm deepspeed llama-cpp transformer llm-inference
Created 2024-02-28
116 commits to main branch, last one 6 months ago
xyjigsaw
LLM-Pretrain-SFT xyjigsaw
14
70
apache-2.0
5
Scripts of LLM pre-training and fine-tuning (w/wo LoRA, DeepSpeed)
lora llama mistral baichuan2 deepspeed large-language-models
Created 2023-11-14
21 commits to master branch, last one 9 months ago
saforem2
l2hmc-qcd saforem2
8
66
apache-2.0
5
Application of the L2HMC algorithm to simulations in lattice QCD.
hmc mcmc hydra horovod lattice pytorch deepspeed tensorflow lattice-qcd monte-carlo gauge-theory deep-learning machine-learning hamiltonian-monte-carlo
Created 2019-03-21
6,366 commits to main branch, last one 11 months ago
billvsme
train_law_llm billvsme
8
56
unknown
2
✏️0成本LLM微调上手项目，⚡️一步一步使用colab训练法律LLM，基于microsoft/phi-1_5、chatglm3，包含lora微调，全参微调
ai law llm lora llama2 python deepspeed
Created 2023-11-07
31 commits to master branch, last one 11 months ago
jackaduma
Alpaca-LoRA-RLHF-PyTorch jackaduma
6
56
mit
5
A full pipeline to finetune Alpaca LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Alpaca architecture. Basically ChatGPT...
gpt llm ppo lora peft rlhf llama alpaca chatgpt pytorch finetune deepspeed reward-models
Created 2023-04-18
25 commits to main branch, last one about a year ago
glb400
Toy-RecLM glb400
4
52
mit
1
A toy large model for recommender system based on LLaMA2/SASRec/Meta's generative recommenders. Besides, note and experiments of official implementation for Meta's generative recommenders.
llama2 sasrec deepspeed recommender-system large-language-models actions-speak-louder-than-words
Created 2024-03-29
36 commits to main branch, last one 6 months ago
l294265421
my-llm l294265421
6
49
mit
6
All about large language models
chatgpt deepspeed distributed-training large-language-models
Created 2023-05-09
106 commits to main branch, last one 5 months ago
pszemraj
ai-msgbot pszemraj
10
46
unknown
3
Training & Implementation of chatbots leveraging GPT-like architecture with the aitextgen package to enable dynamic conversations.
ai nlp gpt-2 gpt-j gradio chatbot gpt-j-6b telegram aitextgen deepspeed deployment huggingface nlp-parsing telegram-bot transformers deep-learning text-generation chat-application huggingface-transformers natural-language-processing
This repository has been archived (exclude archived)
Created 2021-10-15
369 commits to main branch, last one 2 years ago
5663015
LLMs_train 5663015
3
37
unknown
1
一套代码指令微调大模型
llms lora bloom llama pythia baichuan deepspeed chatglm-6b llm-training language-model
Created 2023-06-09
28 commits to main branch, last one about a year ago