2 results found Sort:

sail-sg
Adan sail-sg
64
761
apache-2.0
7
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
mae moe vit adan gpt2 llms timm resnet fairseq pytorch convnext diffusion optimizer bert-model dreamfusion llm-training deep-learning transformer-xl cuda-programming artificial-intelligence
Created 2022-09-01
29 commits to main branch, last one 4 months ago
kozistr
pytorch_optimizer kozistr
19
237
apache-2.0
6
optimizer & lr scheduler & loss function collections in PyTorch
sam adai adan nero adamd adamp radam ranger madgrad pytorch adabound diffgrad adabelief chebyshev lookahead optimizer deep-learning loss-functions gradient-centralization learning-rate-scheduling
Created 2021-09-21
3,298 commits to main branch, last one 12 days ago