3 results found Sort:

rentruewang
koila rentruewang
64
1.8k
mit
12
Prevent PyTorch's `CUDA error: out of memory` in just 1 line of code.
python pytorch deep-learning out-of-memory neural-network lazy-evaluation machine-learning memory-management gradient-accumulation
Created 2021-11-17
152 commits to main branch, last one 3 months ago
hkproj
pytorch-transformer-distributed hkproj
28
64
unknown
2
Distributed training (multi-node) of a Transformer model
pytorch tutorial deep-learning data-parallelism machine-learning model-parallelism distributed-training gradient-accumulation collective-communication distributed-data-parallel
Created 2023-12-08
87 commits to main branch, last one about a year ago
andreped
GradientAccumulator andreped
11
53
mit
3
:dart: Accumulated Gradients for TensorFlow 2
gpu tf2 tpu keras float16 multi-gpu batch-size tensorflow huggingface tensorflow2 deep-learning hacktoberfest mixed-precision memory-constraints distributed-training accumulated-gradients gradient-accumulation adaptive-gradient-clipping accumulated-batch-normalization
Created 2022-05-31
698 commits to main branch, last one about a year ago