From the course: OpenAI API for Python Developers

Introducing the Moderation API

From the course: OpenAI API for Python Developers

Introducing the Moderation API

“

- [Instructor] Now in a more serious tone, I'm going to introduce you to the moderation API. In today's world, when content creation and sharing are as easy as one click, the need for vigilant content moderation is critical to keep the internet a safe and a respectful environment. The moderations endpoint is a tool developers can use to check whether content complies with the OpenAI's policies to filter out harmful content that does not comply with the OpenAI's usage policies. So why you would need to add a moderation layer to your application. In a sense, a chatbot application is not just about answering queries or performing tasks. It's about creating a safe community and offering a pleasant experience for all users. In that sense, adding moderations to one application is a way to enforce community guidelines, ensure the quality of the content, and also provide a safe environment for all users. In this module, the goal is to learn to add a moderation layer and learn to filter out content which is not in compliance with the OpenAI's usage policy. And we're going to start with one introductions and an overview of the moderations API. So the moderations endpoint is a tool that we use to monitor the inputs, the text input, which is typed by the user. For example, right here, you're going to pass your text input, and we're going to be able to then make a classification of this text input and see if it falls under the list of categories. For example, hateful comments, targeting groups of people that can be easily identified and flagged, and other harmful content that expresses hate, threats, or violence can fall under the same treatment and be filtered out of the discussion in order to keep the chats and discussions free from inappropriate and harmful contents. Up next, we're going to see how to integrate a moderation layer as a safeguard. This will also help guarantee to keep your applications and your business a competitive edge by offering a respectful online environment.

Contents