Explanations for the More Obscure Optimizers? #325
Unanswered
setothegreat
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
As a recent Kohya migrant, I am interested in getting the most out of the options available that I haven't seen in Kohya.
In particular, there are a ton more optimizers available that I haven't seen any documentation on, such as CAME, SGD, LARS, LAMB, RMSPROP and others.
How do these optimizers differ from the more well known ones like Prodigy, ADAFactor and Adam, and are there any benefits to using them?
Beta Was this translation helpful? Give feedback.
All reactions