From the course: Neural Networks and Convolutional Neural Networks Essential Training
Join today to access over 25,300 courses taught by industry experts.
Visualizing the loss
From the course: Neural Networks and Convolutional Neural Networks Essential Training
Visualizing the loss
Let me show you a picture that represents a problem in modern AI. This vertical axis represents our loss function. So that's how wrong our network predictions are. The higher up we are, the worse our neural network performs. Now, our goal is simple. Get to the bottom where our neural network makes the best predictions. The other two axes represents two weights in our network. This picture, navigating down seems reasonable. you calculate which direction is steepest downward, take a step, recalculate, and repeat. Now, this is gradient descent. If you study calculus, you've learned exactly the tools needed to solve this. But here's the problem. This visualization shows only two weights, just two dimensions of decision-making. The neural network we'll be creating to determine which item of clothing we're looking at in the Fashion MNIST dataset has over 100,000 weights, as we've learned in our earlier challenge. The GPT-3 model from OpenAI had 175 billion parameters or weights, and many of…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.