Search Results - RepositoryStats

18

220

mit

20

Annotations of the interesting ML papers I read

gpt nlp bert gpt-2 xlnet megatron-lm transformers deep-learning research-paper annotated-paper machine-learning papers-annotations

Created 2021-04-18

127 commits to main branch, last one 2 days ago

9

159

apache-2.0

4

Super-Efficient RLHF Training of LLMs with Parameter Reallocation

llm deepspeed megatron-lm llm-training transformers llm-framework distributed-systems distributed-computing large-language-models reinforcement-learning large-scale-machine-learning reinforcement-learning-from-human-feedback

Created 2024-06-18

1,077 commits to main branch, last one 11 hours ago

18

81

mit

4

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

moe zero-1 megatron megatron-lm transformers 3d-parallelism data-parallelism model-parallelism mixture-of-experts tensor-parallelism pipeline-parallelism sequence-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling

Created 2023-06-14

554 commits to main branch, last one about a year ago

3

63

unknown

2

Odysseus: Playground of LLM Sequence Parallelism

llm pytorch megatron-lm

Created 2024-06-04

50 commits to main branch, last one 6 months ago

2

28

apache-2.0

1

A LLaMA1/LLaMA12 Megatron implement.

llm llama llama2 pytorch megatron megatron-lm llm-training

Created 2023-06-26

10 commits to main branch, last one about a year ago