Search Results - RepositoryStats

62

1.8k

apache-2.0

11

Prevent PyTorch's `CUDA error: out of memory` in just 1 line of code.

python pytorch deep-learning out-of-memory neural-network lazy-evaluation machine-learning memory-management gradient-accumulation

This repository has been archived (exclude archived)

Created 2021-11-17

145 commits to main branch, last one about a month ago

11

50

mit

4

:dart: Accumulated Gradients for TensorFlow 2

gpu tf2 tpu keras float16 multi-gpu batch-size tensorflow huggingface tensorflow2 deep-learning hacktoberfest mixed-precision memory-constraints distributed-training accumulated-gradients gradient-accumulation adaptive-gradient-clipping accumulated-batch-normalization

Created 2022-05-31

698 commits to main branch, last one 5 months ago

11

28

unknown

3

Distributed training (multi-node) of a Transformer model

pytorch tutorial deep-learning data-parallelism machine-learning model-parallelism distributed-training gradient-accumulation collective-communication distributed-data-parallel

Created 2023-12-08

87 commits to main branch, last one 2 months ago