From the course: Building a Project with the ChatGPT API

Understand generative AI models - ChatGPT Tutorial

From the course: Building a Project with the ChatGPT API

Understand generative AI models

“

- Developers are creators. We love to take an idea or concept from a set of requirements on paper to a tangible product that improves the world. We build new things. Generative AI models are much like developers, they use learning patterns from data they're trained on to create new things like music, images, text, and other content like poetry or blog posts. Large language models, or LLMs, like OpenAI's GPT family of models, fall under the generative AI umbrella because they are used to generate something new, like the next word in a sentence based on the previous words. When considering LLMs, there are generally two types, the base or foundation model, and the fine-tuned models retrained for specific task. This further illustrates the difference between GPT and ChatGPT I introduced in the previous video. ChatGPT is a fine-tuned model from a foundation model in the GPT-3.5 family. ChatGPT is fine tuned to hold a dialogue by following instructions in a prompt or question and providing a detailed response called a completion. So, how are foundation models trained? They use large amounts of unlabeled data, like a massive collection of text data, webpages, books, and other sources to perform a wide variety of natural language processing, or NLP, tasks. Foundation models learn a representation that generalizes well to many generative tasks. LLMs are trained to process natural language data using the transformer architecture, a type of neural network that learns to transform input sequences into output sequences. This architecture is often used to learn contextual relationships between words in a sentence. And by the way, the name GPT comes from generative pre-trained transformer. There are several models in the GPT family, GPT-3.0, a set of models that can understand and generate natural language, text-davinci-003, text-curie-001, text-babbage-001, and text-ada-001. GPT-3.5, a set of models that improve on GPT-3 and can understand and generate natural language or code. Developers who use the GPT-3.5 Turbo model will always get the recommended stable version of that model. GPT-4.0, a set of models that improve on GPT-3.5 and can understand and generate natural language or code. ChatGPT is fine tuned from a model in the GPT-3.5 series using reinforcement learning from human feedback, or RLHF. The method includes an initial model that's trained using supervised fine tuning. This is where human AI trainers provide data in the form of conversations where they play both sides, the user and an AI assistant. Then, a reward model is created using reinforcement learning. The training data for the reward model consists of responses ranked by quality. I'm sure OpenAI will continue to iterate by introducing evolved models. The pace of innovation in AI is astounding. Now that you understand generative AI models available from OpenAI, let's explore the API you'll use to access these models.

Contents