3 results found Sort:

bigscience-workshop
petals bigscience-workshop
489
8.9k
mit
91
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
gpt nlp bloom llama falcon llama2 chatbot guanaco pytorch transformer deep-learning language-models neural-networks machine-learning pretrained-models tensor-parallelism distributed-systems volunteer-computing pipeline-parallelism large-language-models
Created 2022-06-12
509 commits to main branch, last one 8 days ago
kaiyuyue
torchshard kaiyuyue
15
293
apache-2.0
11
Slicing a PyTorch Tensor Into Parallel Shards
pytorch model-parallelism tensor-parallelism
Created 2021-04-27
9 commits to main branch, last one 2 years ago
xrsrke
pipegoose xrsrke
17
75
mit
4
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
moe zero-1 megatron megatron-lm transformers 3d-parallelism data-parallelism model-parallelism mixture-of-experts tensor-parallelism pipeline-parallelism sequence-parallelism distributed-optimizers huggingface-transformers large-scale-language-modeling
Created 2023-06-14
554 commits to main branch, last one 6 months ago