Search for a command to run...
🚀 Implementation of ADAM In earlier posts, we implemented a momentum-based optimizer, which offered several advantages: Momentum helps avoid getting stuck in local minima, much like in physics 🏃♂️. It smooths updates by acting as a moving averag...
Implementing Momentum-based Optimizer 🚀 Context Last time, we implemented a simple learning rate scheduler and identified a major flaw: over time, it becomes unreliable for optimizing the training process. So, we decided to upgrade it. I had always ...
🚀 Learning Rate Scheduling: Fixing the Convergence Issue Context 📚 In my first post, I implemented simple Gradient Descent, and while it worked, we faced a common issue—the convergence of parameters was inconsistent. The intercept (b) converged muc...
Exploring Derivatives in Practice Context 📚 After completing a Linear Algebra course, I transitioned to studying Calculus. Unexpectedly, at work, I found myself contemplating derivatives and integrals. This sparked an idea—why not apply derivatives ...
