EmoSens ｖ3.7 and later now complete. #482
Description
Paper
https://huggingface.co/muooon/EmoNAVI/raw/main/emo-v38-paper(ENG).txt
Paper "DOI" Acquired Version
https://huggingface.co/muooon/EmoTion-Optimizer
Code
https://github.com/muooon/EmoSens
kozistr さん、たびたびの更新でお手数を掛けます。EmoSens 完成しました。
これまでの知見をすべて注ぎ込んだ自信作です。 完全自動学習率を達成しています。恐らくですが、最近知ったばかりなのですが、Grokking というんですかね、それを超えていける、Grokking せずに超えていける、かも？ という実装です(もちろん optimizer 単体で) それとこれをもっと簡単にできないか？というところからも新手法を提案しています、詳しくは論文をご覧ください。 お時間ある時に EmoSens (とバリエーションの４種)を pytorch-optimizer に加えてください。どうぞ宜しくお願いします。
Kozistr, thank you for your patience with the frequent updates. EmoSens is finally complete.
This is my most confident work to date, consolidating everything I've learned into a single system. It achieves a fully automatic learning rate.
Interestingly, I recently came across the concept of "Grokking," and I believe this implementation might actually bypass the need for it entirely—achieving flat minima directly (all within the optimizer itself). We’ve also proposed a new, more streamlined methodology born from the question: "Can't we make this simpler?" Please refer to the paper for the technical details.
When you have a moment, I would be honored if you could add EmoSens (along with its 4 variations, Airy, Cats, and Tion) to pytorch-optimizer. Thank you as always for your support!