From the course: Deep Learning: Getting Started

Hidden layers

From the course: Deep Learning: Getting Started

Hidden layers

“

- Hidden layers in a neural network form the brain where knowledge is acquired and used. An ANN can have one or more hidden layers. The more the number of layers, the deeper the network is. Each hidden layer can have one or more nodes. Typically node count is configured in the range of 2 power n Example counts may be 8, 16, 32, 64, 128, et cetera. A neural networks architecture is defined by the number of layers and nodes in that layer. How are the inputs and outputs connected? The output of each node in the previous layer will become the input for every node in the current layer. Similarly, the output of each node in the current layer is passed to every node in the next layer. In the example shown, there are four nodes in the first hidden layer, producing four outputs from their activation functions. There are five nodes in the second layer. The four outputs from the first layer are passed to each of the five nodes in the second layer. How do we determine the right number of layers and nodes in a network? Each node in the neural network learns something about the relationship between the feature variables and the target variables. This knowledge is persisted in its weights and biases. When there are more nodes and layers, it usually results in better accuracy. Do note that this is always not true. More layers would also mean more compute resources and time for both training and inference. The right architecture for a given problem is determined by experimentation. As a general practice, start with small numbers and keep adding until acceptable accuracy levels are obtained.

Contents