From the course: Neural Networks and Convolutional Neural Networks Essential Training
Join today to access over 25,300 courses taught by industry experts.
Optimizers
From the course: Neural Networks and Convolutional Neural Networks Essential Training
Optimizers
Remember our GPS metaphor, PyTorch computes gradients, telling us which direction leads downhill through that million-dimensional landscape. But here's the question, who actually turns the wheel? Who decides how sharply to turn or how fast to go, or when to accelerate, or when to brake? That's the optimizer's job, your neural network's driving instructor. You've got your network architecture, which is your car, your loss function, which is your destination, and then PyTorch's automatic differentiation, or your GPS, but the optimizer is the driving instructor sitting beside you making the moment-by-moment decisions about how to navigate. So the GPS says turn left towards lower loss, but the optimizer decides turn 15 degrees left at moderate speed versus hard left turn now. So let me walk you through one complete training iteration. And this happens thousands or millions of times during training. So the first step is the forward pass. You take a batch of training data, let's say 64…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.