How do neural networks learn?

- [Instructor] artificial neural networks solve learning problems by modeling the relationship between a set of input signals and an output signal similar to the way biological neurons work. Just like any other supervised machine learning model, in order to train a neural network, we provide it with a training data set that includes a set of independent variables and a independent variable as shown here. As a neural network reads each row of independent variables, it learns to predict the value of the corresponding independent variable correctly by strengthening or weakening connections between neurons to reflect the patterns it observes in the training data. It accomplishes this by iteratively adjusting the connection weights using a technique known as backpropagation. Let's walk through a simple example to illustrate how backpropagation works. In the previous video, we learned that neural networks learn through numbers. This means that the data we pass to them must be represented in numeric form. The backpropagation process involves an iterative approach to adjusting connection weights through cycles known as epochs. Each epoch consists of a forward phase and a backward phase. The forward phase involves the activation of neurons in sequence from input to output. During this phase, as inputs or signals initially flow through the network, a set of connection, weights and and biases are set at random. The signals are weighted, summed, and passed on based on the activation function we chose for the model. The value passed on by the activation function is a predicted output for the given input. So in this example, given the values nine, seven, and eight as input, the model predicts 0.622. This concludes the forward phase of the backpropagation process. The backward phase begins with using a loss or error function to compare the model's output of 0.622 with the expected value of one. The objective in this phase is to adjust the weights and biases so that the difference between the predicted and expected value is minimized in subsequent iterations. To accomplish this, the network uses an optimization algorithm such as stochastic grain descent, to figure out how much to adjust the bias and the weights in the network. After weights and biases are adjusted, the backward phase concludes, so does the first epoch. In the next epoch, the network makes a new prediction based on the adjusted weights and bias. This is the forward phase again. During the backward phase of this new epoch, the network further adjust the weights and bias in order to minimize error. Note that the values of the weights and bias could be adjusted up or down, depending on what will best bring the predictive value closest to the expected value. The process repeats in the third epoch with an updated prediction during the forward phase, followed by gradual adjustments to the weights and bias in the backward phase. This back and forth continues in epoch after epoch onto the network, which is a satisfactory level of accuracy or converges on a stable set of weights and biases. The final set of weights and biases represent the trained neural network.