2 results found Sort:

sail-sg
Adan sail-sg
64
771
apache-2.0
7
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
mae moe vit adan gpt2 llms timm resnet fairseq pytorch convnext diffusion optimizer bert-model dreamfusion llm-training deep-learning transformer-xl cuda-programming artificial-intelligence
Created 2022-09-01
29 commits to main branch, last one 5 months ago
kozistr
pytorch_optimizer kozistr
22
250
apache-2.0
7
optimizer & lr scheduler & loss function collections in PyTorch
sam adai adan nero adamd adamp radam ranger madgrad pytorch adabound ademamix diffgrad adabelief lookahead optimizer deep-learning loss-functions gradient-centralization learning-rate-scheduling
Created 2021-09-21
3,449 commits to main branch, last one 6 days ago