From the course: Deep Learning: Getting Started

The input layer

From the course: Deep Learning: Getting Started

The input layer

“

- [Instructor] In this chapter, let's deep dive into the architecture of an ANN and explore various layers, parameters, and functions used in the architecture. We will start with the input layer. Let's first start with the concept of vectors. A vector is an ordered list of numeric values. The input to deep learning is usually a vector of numeric values. A vector can be set to be a tuple of one or more values. Vectors are usually defined using a NumPy array. It represents the feature variables or independent variables that are used for predictions as well as training. Input data sets that are available from the real world for machine learning contain samples and features. A sample is one instance of a real world example. It is equivalent to records in a database. Features are individual attributes in the sample. In the sample, we have a data set of employees. Each employee record represents a sample. Individual attributes like age, salary, and service are considered features. For text data, each document is a sample and it's numeric representation becomes its features. For images, each image is a sample and its pixel representation becomes its features. Similar to regular machine learning, input data needs to be pre-processed and transformed to appropriate numeric representations before they are fed into a neural network. Here are some popular pre-processing techniques used. For input data, we usually center and scale them to normalize the values. For categorical variables, we encode them using either integer encoding or one hot encoding techniques. Text data needs to be converted to equal numeric representations. TF-IDF, or text frequency-inverse document frequency, is a classic example to represent documents. Embeddings are becoming more popular in the deep learning world. Images are represented by pixels. We will create a vector of pixel values for this. Speech may be represented as a time series of numbers. This list is no way exhaustive. There are a number of advanced pre-processing techniques that are applied to data to prepare them for deep learning. This is an example of how employee data can be pre-processed. Here, we represent each attribute like age, salary, and service as individual features x1, x2, and x3. We will center and scale the values to normalize them into standard ranges. Optionally, we will also transpose them to represent each sample as a column. Once the input data is ready, it can be passed to the deep learning network for learning.

Contents