4 results found Sort:

LiyuanLucasLiu
RAdam LiyuanLucasLiu
334
2.5k
apache-2.0
58
On the Variance of the Adaptive Learning Rate and Beyond
adam warmup optimizer adam-optimizer
Created 2019-08-01
73 commits to master branch, last one 3 years ago
Tony-Y
pytorch_warmup Tony-Y
24
395
mit
5
Learning Rate Warmup in PyTorch
adam warmup pytorch deep-learning learning-rate-scheduling
Created 2019-10-31
107 commits to master branch, last one 10 days ago
sooftware
pytorch-lr-scheduler sooftware
17
87
mit
2
PyTorch implementation of some learning rate schedulers for deep learning researcher.
lr reduce warmup plateau scheduler tri-stage transformer learning-rate
Created 2021-05-20
11 commits to main branch, last one 2 years ago
Lance0218
Pytorch-DistributedDataParallel-Training-Tricks Lance0218
11
61
mit
2
A guide that integrates Pytorch DistributedDataParallel, Apex, warmup, learning rate scheduler, also mentions the set-up of early-stopping and random seed.
apex warmup pytorch distributed random-seeds early-stopping learning-rate-scheduling pytorch-distributeddataparallel
Created 2020-08-25
15 commits to master branch, last one 2 years ago