Search Results - RepositoryStats

18

214

mit

20

Annotations of the interesting ML papers I read

gpt nlp bert gpt-2 xlnet megatron-lm transformers deep-learning research-paper annotated-paper machine-learning papers-annotations

Created 2021-04-18

123 commits to main branch, last one 10 days ago

5

120

apache-2.0

3

Super-Efficient RLHF Training of LLMs with Parameter Reallocation

llm deepspeed megatron-lm llm-training transformers llm-framework distributed-systems distributed-computing large-language-models reinforcement-learning large-scale-machine-learning reinforcement-learning-from-human-feedback

Created 2024-06-18

1,071 commits to main branch, last one a day ago

18

80

mit

4

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

moe zero-1 megatron megatron-lm transformers 3d-parallelism data-parallelism model-parallelism mixture-of-experts tensor-parallelism pipeline-parallelism sequence-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling

Created 2023-06-14

554 commits to main branch, last one 11 months ago

1

57

unknown

2

Odysseus: Playground of LLM Sequence Parallelism

llm pytorch megatron-lm

Created 2024-06-04

50 commits to main branch, last one 5 months ago

2

27

apache-2.0

1

A LLaMA1/LLaMA12 Megatron implement.

llm llama llama2 pytorch megatron megatron-lm llm-training

Created 2023-06-26

10 commits to main branch, last one 11 months ago