From the course: OpenAI API and MCP Development

Challenge: Building a multimodal Q&A assistant

From the course: OpenAI API and MCP Development

Start my 1-month free trial

Challenge: Building a multimodal Q&A assistant

“

I'm sure that you have already dreamt about having your own assistants to write your emails, answer emails, write code, do math, and even more to assist you in your daily task, and help you out whenever you feel stuck. In the next project and challenge, you will combine the three powerful skills that you have learned, and apply text generation, image generation, and even code generation. The goal for the next challenge is to design one AI-powered assistant that can respond intelligently to different requests. What you want to do is to build an AI-powered assistant that can understand natural language and also generate responses as expected. So what you'll need to do first is to set up API requests using the OpenAI documentation, define the prompts and the instructions in order to guide the language model's responses and tell the models how to act as a helpful assistant, for example. Next, of course, you want to generate the outputs and also display the outputs, using, for example, the Streamlit Python library to create a nice and clean user interface, where we can read the response generated by the language model. So your mission is to bring it all together in one clean UI. And so you can use this documentation from Streamlit, the Streamlit library. You have access to different kind of components. You also have different examples in this gallery where you have examples showcased and see how you can build, put together your own AI assistance, just like this example. And what I've put together is one starter project. We're gonna check this one out very quickly. Here you have a starter project with some handler functions, helper functions, in order for you to get started. So what you want to do is to apply the text generation endpoints. We can use this one, and also if you'd like, this is one option, you can also allow to generate images. What you want to do for the next challenge is to put together a multi-model Q&A assistance that can do and process multiple AI-powered tasks. And as usual, you're gonna find also the instructions in the readme file. So I recommend that you also check this one out. Of course, to help you out, you can also check out the open AI documentation with examples in order for you to start building. You have multiple examples as to how to put one next generation app in order to read and generate text. Another example here to generate images. And also important is to check out the list of models available to see which one is the most suitable and effective in order for you to process the task that you want to accomplish. So here you can explore the different available models and compare the capabilities as well. So good luck, this is up to you now, so that's gonna be your mission, to put together this nice Q&A assistance, and of course you're gonna have access to one solution that you can check out afterwards.

Contents