4 results found Sort:

shreyansh26
Annotated-ML-Papers shreyansh26
16
191
mit
18
Annotations of the interesting ML papers I read
gpt nlp bert gpt-2 xlnet megatron-lm transformers deep-learning research-paper annotated-paper machine-learning papers-annotations
Created 2021-04-18
96 commits to main branch, last one 6 days ago
xrsrke
pipegoose xrsrke
17
75
mit
4
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
moe zero-1 megatron megatron-lm transformers 3d-parallelism data-parallelism model-parallelism mixture-of-experts tensor-parallelism pipeline-parallelism sequence-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling
Created 2023-06-14
554 commits to main branch, last one 6 months ago
feifeibear
Odysseus-Transformer feifeibear
0
31
unknown
2
Odysseus: Playground of LLM Sequence Parallelism
llm pytorch megatron-lm
Created 2024-06-04
50 commits to main branch, last one 12 days ago
MoFHeka
LLaMA-Megatron MoFHeka
2
26
apache-2.0
1
A LLaMA1/LLaMA12 Megatron implement.
llm llama llama2 pytorch megatron megatron-lm llm-training
Created 2023-06-26
10 commits to main branch, last one 6 months ago