prodigy
The Prodigy optimizer and its variants for training neural networks.
Likely Available
About
Research Scientist
Evidence-backed skills
Repositories
opt_methods
Benchmarking optimization methods on convex problems.
random_reshuffling
RR, SO, IG and SGD with logistic regression loss.
asynchronous_sgd
Code used for paper on Asynchronous SGD published at NeurIPS 2022
rr_prox_fed
Proximal and Federated Random Reshuffling
pytorch_optimizers
Stochastic optimizers in pytorch
examples
A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
modded-nanogpt
NanoGPT (124M) in 90 seconds