7 results found Sort:

cupy
cupy cupy
776
7.9k
mit
127
NumPy & SciPy for GPU
gpu cuda cupy nccl nvtx rocm cudnn numpy nvrtc scipy cublas curand python tensor cusolver cusparse cutensor cusparselt
Created 2016-11-01
28,514 commits to main branch, last one a day ago
huggingface
large_language_model_training_playbook huggingface
18
442
apache-2.0
69
An open collection of implementation tips, tricks and resources for training large language models
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-06
26 commits to main branch, last one about a year ago
huggingface
llm_training_handbook huggingface
27
413
cc-by-sa-4.0
50
An open collection of methodologies to help with successful training of large language models.
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-08
18 commits to main branch, last one 10 months ago
Bluefog-Lib
bluefog Bluefog-Lib
71
290
apache-2.0
28
Distributed and decentralized training framework for PyTorch over graph
mpi nccl pytorch one-sided asynchronous deeplearning decentralized machine-learning distributed-computing
Created 2019-12-03
1,094 commits to master branch, last one about a year ago
microsoft
msrflute microsoft
22
180
mit
9
Federated Learning Utilities and Tools for Experimentation
gloo nccl pytorch simulation privacy-tools personalization machine-learning federated-learning transformers-models distributed-learning
Created 2021-11-17
54 commits to main branch, last one 9 months ago
FZJ-JSC
tutorial-multi-gpu FZJ-JSC
47
151
mit
9
Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial
gpu hpc mpi cuda nccl sc21 sc22 sc23 isc22 isc23 isc24 nvshmem multi-gpu supercomputing exascale-computing
Created 2021-09-23
240 commits to main branch, last one 3 days ago
google
nccl-fastsocket google
13
103
other
5
NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
nccl training machine-learning
Created 2021-07-19
20 commits to master branch, last one 6 months ago