GPT and Natural Language Processing (NLP)

- [Instructor] The central function of GPT is the ability to ingest large amounts of data and use it as a basis to create original output in the form of text, video, and audio. The quality of text output, for example, is so high that it's often difficult to distinguish it from something written by a person. To enable and support this high quality output, a number of AI techniques are employed to make sense of input provided. GPT utilizes an approach that is at the intersection of computer and data science and human language called natural language processing or NLP. Let's begin with an example of NLP in action. Many of us now issue voice commands to digital services such as the popular virtual assistants Amazon Alexa and Google Home. These devices listen to our words, process this input, and then execute some actions such as playing a favorite album or turning on a light. This same technology that makes sense of voice commands can also be applied to text input on a computer. This enables solutions such as chatbots, search engines, language translation, and spell check. In all these instances AI is tasked with processing language, either audio or text. At a high level, NLP generally works like this. The software needs to pre-process the text in sentences in order to provide some form of structure that can be used as the basis for interpretation. First, the sentence is broken into each word. This is called tokenization. The individual words are known as tokens. Unnecessary punctuation is removed. Next words could be identified and tagged as nouns, verbs, adjectives, pronouns, et cetera. This is followed by the process of stemming. This is where words are standardized and put in context by reducing them to their root form. For example, the words banks, banker, and banking are all associated with the root word bank. This is a stem. That root word will be used to assign the context for the input to, for example, a financial institution and not the act of turning an aircraft. Once the text has been pre-processed, a machine learning or ML algorithm is used to interpret the input text. These algorithms use statistical models based on vast volumes of data called training data to suggest what action to perform. These statistical models are also referred to as large language models or LLMs. When the pre-processed text is analyzed by the ML algorithm, it is looking for words, phrases, and patterns of text that are familiar from the training data. If there is a high probability that the words and context are understood, the software now knows what to do next. In my examples, NLP is being used to take an action such as retrieving a search result or changing the temperature on a thermostat. However, input could also be used to generate new media outputs, such as text, audio, and video. This is where we get the term generative AI and it's where we'll go in the next video.