From the course: Complete Guide to Microsoft Copilot for Security: Empower and Protect the Security Operations Center (SOC) by Microsoft Press

2.5 Compound AI in Copilot for Security - Microsoft Security Copilot Tutorial

From the course: Complete Guide to Microsoft Copilot for Security: Empower and Protect the Security Operations Center (SOC) by Microsoft Press

Start my 1-month free trial Buy for my team

2.5 Compound AI in Copilot for Security

“

- So we've talked about these compound AI architectures and models. So let's talk about how it was utilized for Microsoft copilot for security. But first, let's just dive into how we can improve on LLMs. So we talked about different ways to take a monolithic and connected up to make a compound architecture. There's also ways to actually improve on the LLMs and so there's three different areas we'll talk about on how we can improve the LLMs or the foundational models. The first is around prompt engineering, and this is in context learning, being able to engineer the prompt to be able to guide the LLM to get us the best answer. The next is fine tuning. So this is something that they can permanently correct areas of the LLM by teaching it new skills, teaching it new information, and fine tuning it based on the information we talked about. Those numbers that the LLMs talk, those can be fine tuned in that. And then the last is RAG or retrieval-augmentation. So this is where we can temporarily teach it new facts in the aspect of our conversation. So in line or in our context, we can actually teach it some new facts temporarily. It doesn't go back and retrain the entire model. Doesn't fine tune the model, but it's using RAG to then have the model give you a best or better answer. Next, let's talk about the use of compound AI systems in cybersecurity. So we know that that LLM has been a game-changer as far as the foundation for conversational AI for that natural language processing to be able to discuss things and understand the scripting language of cyber. But it's also a number of other pieces that we can plug in here. So things like threat intelligence, in fact the copilot for securities kind of built on that threat intelligence as a grounding mechanism. We'll talk a little bit more about grounding the phishing and email detection. One of the things that most organizations struggle with is around being able to combat these phishing attacks that are constantly going on. But we can also look at the attack surface. One of the things that these compound AI systems for cyber are great at doing is looking at code or malware and understanding what that is, maybe even de-obfuscating it. We can look at vulnerabilities and looking across a broad set of data and thinking about what exposures or vulnerabilities an organization may have. And then it's also great at looking at protocol analysis. If you just drop a packet capture into a compound AI system, understanding what that analysis of the protocols that are going on and it can see and understand those. And lastly, we've got policy designs and validation. So looking at the policies that we've got, comparing it with what we may want to change to, and having that LLM do a lot of that mundane work that could bog down an analyst or even a seasoned professional in what they're trying to do and get their mission done. So in this picture or image, we're going to talk about where copilot for security is built on a compound architecture. As you can see from the image here, it started on the left with the most advanced general AI models. So things like from OpenAI, the GPT 3.5, GPT 4.0, being able to build that into this service. And again, this is a software as a service. It's a SaaS offering that was built with this compound architecture. The next thing was using Azure AI services because it's got all of those plumbing and components built-in to be able to utilize and build this massive scale system. And then we took the experience that Microsoft Security has and coupled that with a security-specific orchestrator. Now again, this all runs on the Microsoft hyperscale infrastructure. So being able to have those GPU resources to be able to infer this information, drive it and scale this up, again, this is something that's designed in, not bolted on. The next thing though is to take the answers that get returned from this orchestrator and run them through a, what we call evergreen threat intelligence or something that's constantly being updated. So that threat intel now grounds it. Make sure that yes, this is in reality, this is real threats that are going on in the industry as of the moment, as of today. And that's constantly being refreshed by the Microsoft research team and the threat intelligence center. And lastly, we've got those cyber skills and prompt books that we can bring together and we'll talk more and more in depth about what a prompt book is, where you use it. But just know that this compound architecture has that cyber skillset, that cyber-specific information, and that threat intelligence all designed in built-in. And when we look at the current event skills that copilot for security has, it's around having that orchestrator being able to call those specialties into play, those different plugins. So things that may be outside of even just the Microsoft ecosystem and going out and looking at additional information, that threat intelligence and those cyber skills and prompt books that we can then ground this information to get that best answer, to get that most accuracy out of using copilot for security.

Contents