Added AdamW optimization function to deep_learning/optimizers.py

[showmyth]

Description

The AdamW optimizer is a variant of the Adam optimizer that improves deep learning model generalization by decoupling weight decay from the gradient update. While Adam couples L2 regularization with its gradient updates, which is problematic for adaptive methods like Adam, AdamW applies weight decay directly to the parameters after the gradient-based update.

Necessity

Leads to more stable, regularized and effective training, especially for large models like transformers. It is also used a lot more as compared to the traditional adam optimizer.

Features

• AdamW optimizer class with configurable hyperparameters (learning_rate, b1, b2, weight_decay)
• Decoupled weight decay application
• Bias correction with time step tracking

References

• https://docs.pytorch.org/docs/stable/generated/torch.optim.AdamW.html
• https://arxiv.org/abs/1711.05101

Designed to be compatible with mlfromscratch optimization API