9 results found Sort:

cupy
cupy cupy
855
9.5k
mit
128
NumPy & SciPy for GPU
gpu cuda cupy nccl nvtx rocm cudnn numpy nvrtc scipy cublas curand python tensor cusolver cusparse cutensor cusparselt
Created 2016-11-01
29,313 commits to main branch, last one 2 days ago
coreylowman
cudarc coreylowman
78
635
apache-2.0
13
Safe rust wrapper around CUDA toolkit
gpu cuda nccl rust cudnn nvrtc cublas curand cuda-kernels cuda-toolkit cuda-programming gpu-acceleration
Created 2022-09-16
267 commits to main branch, last one 2 months ago
huggingface
llm_training_handbook huggingface
35
462
cc-by-sa-4.0
53
An open collection of methodologies to help with successful training of large language models.
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-08
18 commits to main branch, last one about a year ago
huggingface
large_language_model_training_playbook huggingface
23
460
apache-2.0
68
An open collection of implementation tips, tricks and resources for training large language models
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-06
26 commits to main branch, last one about a year ago
Bluefog-Lib
bluefog Bluefog-Lib
71
292
apache-2.0
28
Distributed and decentralized training framework for PyTorch over graph
mpi nccl pytorch one-sided asynchronous deeplearning decentralized machine-learning distributed-computing
Created 2019-12-03
1,094 commits to master branch, last one about a year ago
LambdaLabsML
distributed-training-guide LambdaLabsML
19
291
mit
7
Best practices & guides on how to write distributed pytorch training code
gpu mpi cuda fsdp nccl slurm cluster pytorch sharding deepspeed kuberentes lambdalabs gpu-cluster distributed-training
Created 2024-07-31
237 commits to main branch, last one a day ago
FZJ-JSC
tutorial-multi-gpu FZJ-JSC
54
190
mit
11
Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial
gpu hpc mpi cuda nccl sc21 sc22 sc23 isc22 isc23 isc24 nvshmem multi-gpu supercomputing exascale-computing
Created 2021-09-23
283 commits to main branch, last one 5 days ago
microsoft
msrflute microsoft
22
185
mit
10
Federated Learning Utilities and Tools for Experimentation
gloo nccl pytorch simulation privacy-tools personalization machine-learning federated-learning transformers-models distributed-learning
Created 2021-11-17
54 commits to main branch, last one about a year ago
google
nccl-fastsocket google
13
112
other
5
NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
nccl training machine-learning
Created 2021-07-19
20 commits to master branch, last one about a year ago