9 results found Sort:

cupy
cupy cupy
861
9.6k
mit
127
NumPy & SciPy for GPU
gpu cuda cupy nccl nvtx rocm cudnn numpy nvrtc scipy cublas curand python tensor cusolver cusparse cutensor cusparselt
Created 2016-11-01
29,404 commits to main branch, last one 21 hours ago
coreylowman
cudarc coreylowman
83
665
apache-2.0
13
Safe rust wrapper around CUDA toolkit
gpu cuda nccl rust cudnn nvrtc cublas curand cuda-kernels cuda-toolkit cuda-programming gpu-acceleration
Created 2022-09-16
272 commits to main branch, last one 7 days ago
huggingface
llm_training_handbook huggingface
35
465
cc-by-sa-4.0
53
An open collection of methodologies to help with successful training of large language models.
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-08
18 commits to main branch, last one about a year ago
huggingface
large_language_model_training_playbook huggingface
23
464
apache-2.0
68
An open collection of implementation tips, tricks and resources for training large language models
llm nlp cuda nccl python pytorch performance scalability troubleshooting large-language-models
Created 2023-03-06
26 commits to main branch, last one about a year ago
LambdaLabsML
distributed-training-guide LambdaLabsML
21
320
mit
7
Best practices & guides on how to write distributed pytorch training code
gpu mpi cuda fsdp nccl slurm cluster pytorch sharding deepspeed kuberentes lambdalabs gpu-cluster distributed-training
Created 2024-07-31
238 commits to main branch, last one 8 days ago
Bluefog-Lib
bluefog Bluefog-Lib
71
295
apache-2.0
28
Distributed and decentralized training framework for PyTorch over graph
mpi nccl pytorch one-sided asynchronous deeplearning decentralized machine-learning distributed-computing
Created 2019-12-03
1,094 commits to master branch, last one about a year ago
FZJ-JSC
tutorial-multi-gpu FZJ-JSC
54
199
mit
11
Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial
gpu hpc mpi cuda nccl sc21 sc22 sc23 isc22 isc23 isc24 nvshmem multi-gpu supercomputing exascale-computing
Created 2021-09-23
283 commits to main branch, last one about a month ago
microsoft
msrflute microsoft
23
185
mit
10
Federated Learning Utilities and Tools for Experimentation
gloo nccl pytorch simulation privacy-tools personalization machine-learning federated-learning transformers-models distributed-learning
Created 2021-11-17
54 commits to main branch, last one about a year ago
google
nccl-fastsocket google
12
113
other
5
NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
nccl training machine-learning
Created 2021-07-19
20 commits to master branch, last one about a year ago