3.1 Introducing prompt engineering - Microsoft Security Copilot Tutorial

- So, as we talked about in the previous section, prompting or engineering a prompt is one of the ways to communicate with the LLM or to modify the way that it gives you the outputs. And so this has been, Joey and I talk about this all the time, that this is a burgeoning area of technology, prompt engineering that's being now taught in high schools and colleges. We'll probably see it even in the earlier stages of schooling in the future. But what is the prompt engineering? What does this actually mean? And it's the process of writing, refining, and optimizing the inputs that go into that AI systems. And really what it's doing is helping the AI model better organize the questions that you're asking, and the responses that it gives back. I think of it as really natural language programming. You know, programming in code is probably not something people will be doing much or as much in the future. It's really around talking to the system, engineering how your conversation goes, and to be able to get the best answer out. So deep knowledge of data sets, you don't need to know that. You don't need to necessarily know how to write that code for the system, it's all built-in with this natural language processing. So who are these prompt engineers and what do they do? I think we're all going to end up being prompt engineers, especially if we work in the cyber world where we typically would've been scripting something to get an answer back for what our data was or problem was. Now we're going to be more of prompt engineers and we're going to play a pivotal role in crafting these queries, whether it's something that's for automation we're repeating, or if it's something that we are just doing on the fly to be able to help the AI model understand what our intent is, what the nuance is related to it. And it's important because now it gets better responses from AI models by being a prompt engineer. Next, let's talk about what the prompt interface is to these AI models. Many people have probably seen this if you've have been anywhere above ground over the past two years, you've heard about ChatGPT, and it was the fastest product ever to get over a million users in our history. At the time that we're recording this, it's over 180 million monthly active users. It's massive, it's incredible. And if you think about why it's so incredible and why it picked up so fast, it was because I could just talk to this LLM, I didn't have to go to an API build some sort of code or system to be able to talk to the LLM and ask it a question, instead I could just type in my question and ask it here. Now, obviously, the better my question is, and the prompt engineering I do, people have been getting better and better at building that muscle every day. The bottom one, I just want to point out between the ChatGPT interface and the Copilot for Security interface, you'll notice there's a lot of similarities. You know, there's questions, which came first, the cart or the horse, the chicken or the egg? This is another question that you could ask, but very similar. And Copilot for Security is the first conversational AI system that was built for cybersecurity purposes and being able to utilize this prompt engineering and this prompt interface. The next thing I like to talk about is the art of the prompt. And so those that love MadLibs and fill in the blank types of books and games are going to be great at prompt engineering because the better you are, the better results that you're going to get. And you can see sort of from the depiction on how this goal expectation, source, intent builds out. Similar to building a sentence structure as we learned, if anyone ever went through "Rex Barks" and understanding how a sentence and a paragraph, things are structured, you can use all of that skill in building prompts, building that engineering of how you interact with this. You can see now the elements of an effective prompt. So we have the goal, the intent, the expectations, and the source. So these are the pieces that we can build into our prompt to get the best answer. And we want to be clear, concise, and very specific with the model. If we just ask it open-ended questions, it's not going to give the best answers. And we'll see some examples of that as we go through. But this goal, what is the information that I'm asking it? I want to know information about this specific incident. Then the system can go and ask for that. The intent, why do I need it and how am I going to use it? Well, I'm going to use it for an investigation report that I can submit to my manager so it can tailor that output to what my intent is. And then give me the expectations, compile the information into a bullet list and have a short summary at the end. But I could ask it to format it as a table or other things. These are the expectations that I'm getting in giving to the system, to the model. And lastly is around source. Is there a source that I want to ask it for, a specific connector, a plugin or a data source that I can tell that orchestration engine, that reasoning engine to look at that particular source? In this case, look in defender incidents to be able to understand that. Another thing that we can do in prompt engineering is to give custom instructions. And so this is something that's going to direct the model or the system to understand that, and as you can see, I give it a custom information here around today's date is May 9th, 2024. And do not make things up. So we want to make sure that the model understands not to make things up. And my initial prompt though is tell me about a CVE, and, as you can see, this CVE number, 2025-29913, I don't know when you're listening to this training, but it's not 2025 yet. So I've told the model now that today's date is May 9th, 2024 and don't make things up. So now as it goes through the LLM, it's going to understand that you're trying to trick me, it's not even 2025 yet. So I can't tell you about a CVE with a 2025 number until that date. So there's a lot of benefits to prompt engineering. It improves the speed and efficiency of these generative AI tasks. So being able to summarize data, generate content, and write complex queries that we may be asking for, we can direct the model to be able to understand and do that. We're going to enhance a lot of the skills and confidence level of people that are newer to generative AI. So having feedback, being able to give information back from the model, help them kind of learn and come up to speed. Obviously, we're leveraging the power of foundation models, so we're getting optimal outputs without having to go back and retrain and revise that entire model. We're also using it to mitigate confusion or errors or even bias in there by fine tuning with those more effective prompts. And this is our way of bridging a gap between what that raw query is and what that meaningful output that we're looking to get. So we don't have to do as much post-generation editing if it's a report for our boss or to manually review it for accuracy if we're using it for a decision on our cyber strategy or our cyber operations. Now let's look at how the prompt flow works through Copilot for Security. We always start with a human in the loop. So this is why it's a copilot, not an autopilot or a pilot. The human is actually going to prompt the system, submit a prompt, and then expect a prompt back at the end. But how it goes through and moves through the system, we'll talk through a little bit here. So once the prompt is submitted, and we've talked about the engineering of that prompt and what it should look like, then it goes to the orchestrator. And this is that planner, that reasoning engine, that's then going to determine the initial context of what you are asking of it and then build a plan with the skill sets that it knows about. And those are those skill sets that you choose, those sources that you choose. And we'll show how to build some of those and what those look like later on in the series. And then it's going to build context. And this is where it executes that plan, gets the required data context that's going to be able to answer the prompt you're asking for. Next, it's going to go to the plugins, it's going to analyze all the data and patterns with the insights that it knows about, using that orchestration, using the plugins together, and then it's going to combine all of that data together and work out a response that the model or the system can give back. And lastly, it's going to format that data. So it may be in the form of a table or a bulleted list or a summary that you've asked it, and it's going to use the LLM to then make that formatting, that summarization, that build, and return the response back to that interface or that chat bar that was built out. So we started with the chat bar, we receive the response back. When we think about prompting in Copilot for Security, I like to think of it as people that were not very good at search engines or bad Googlers, they'll probably be bad prompters as well. So a couple of examples of poor prompts. "Just write something." That doesn't give the system or the model any understanding of what it needs to do. And so it's very vague. It doesn't give context, doesn't give instructions, any of those steps that we've went through. Another one is just summarize this and we know that that's not giving enough information on what you want it to summarize, how you want it to be built out. Give me some parameters or some guardrails that I need to work within. So if we improve a prompt on the write something, now we want to ask it to write a report about a Microsoft Sentinel incident with a number of that incident. And I want it reviewed by my SOC team manager whose name is Lee Majors. So now it knows it's specific, gives a clear context, guides it towards who you want to tailor this to, and what task you're asking it to do so it can respond back. The improved prompt on the summarize this, provide me a one paragraph summary of the key points from the Microsoft Sentinel incident with this number. And the reason that that's better is now you're telling it, this is the length I want, this is the focus that I want you to do and build me that response based on it. So the system is really good at being able to understand that, but you have to give it those parameters, you have to be a good searcher, a good Googler, or a good prompt engineer in this case. Now to look a little bit deeper into evoking those key words, starting with the prompt, ending with a response, and having those matches to skills, the skills that it's finding and the keywords that we're directing it in the middle. So the less effective side of prompting would be, just show me what this alert looks like, and it's weak, vague, unclear. Is it a movie show or is it that you want to show something? It's not very effective for an LLM, for a generative AI model to understand. And you're probably not going to get a great output or a skill match based on that information, "Just show me what alert looks like." The more effective way would be to say something like, "Visualize the attack path of Sentinel incident with 2313 in this case as a graph." So now we have key words that it can understand and narrow in on. We've mentioned the data source. We've been more precise related to the skillset. We know that we want to visualize, we want an intact path. We have Sentinel and we want it to be as a graph. So now it's going to visualize an investigation and use that skillset that it is built for and engineered to do. And lastly, we're going to get a graph of the attack path as expected and as requested from the system. So we went from a less effective prompt by giving it a little bit more information, giving it a little bit more detail, and how we would communicate just as if we were talking to a human and we wanted that human to visualize to do something better than "just show me something." So this is where that prompt engineering all comes together to send in a good prompt, get a great response back.