Evaluate solution constraints (cost, compliance, scalability) - Azure Tutorial

- [Tim] Well, of course, we're concerned how much is this going to cost? Well, how much are these Azure AI services? What you'll find in terms of trend are you can save money by doing serverless or pay-as-you-go option. The problem with that is what's called cold start. The underlying computes going to need time to wake up. So it's a business thing. You have to cost on one hand, expected performance on the other. The cognitive services, now they're called Azure AI services, have a very generous free tier, where you can hit the API a lot for free. I'll make sure to show you that as we go along. But you want to think about, eventually it's going to cost you money. And depending on your region, there are regional pricing differences. How can we do cost optimization? We'll use the services in batches. Don't just unleash them to all of your internal dev base. Make sure that you're consuming the APIs intelligently. Especially with Azure AI services, make sure that you don't have multiples of a service that you're paying for. Within each service, taking advantage of caching. If you're using Azure storage, for example, in the blob service, you can use the archive tier. And then as a developer, you're thinking about resource scaling, you're thinking about back off and retry logic. And if you're a engineer, well, enough about that. Part of my goal as your trainer is to make sure that you're spec'd up, that you've got all the best first-party resources, go.techtrainertim.com/calc, C-A-L-C, will bring you to the Azure pricing calculator. The benefit here is not that Microsoft gives you a binding price quote, because it will not. Please understand that. But it does use the live Azure pricing APIs. So in terms of modeling how much all this AI stuff is going to cost, the pricing calculator can get you pretty close because you specify your regions and you specify all of the dials and switches of those AI services in your quote that you build, all right? More on compliance. Every IT professional's favorite subject, right? Yeah, right. With Azure AI, you've already been there, done that, bought the T-shirt. If you've supported Azure, it's basically the same thing. You choose your region for the Azure AI services. So if you've got industry-specific regs, data residency, sovereignty issues, audit requirements, make sure that you're in the right region. If your security team barks at you going over the public internet to interact with these cognitive services, you've got private endpoints. For at rest data encryption, customer managed keys, and role-based access control, both for identity governed by the Microsoft graph API, and as your resource manager, governed by the ARM API. Scalability and performance is one of the Azure well-architected framework pillars, isn't it? So these APIs, although they're regional, they're globally available, and so they're massively scalable so they can handle load. These are classic trade-offs of platform as a service services. Failover, you have limited choices with a lot of the Azure AI because you're delegating most of the availability and performance to Microsoft. We can control performance cost benefit, that kind of algebra we can solve for, but we don't get to failover the service from one region to another. That's on Microsoft. So what does this mean to us? We'll take advantage of any modularity that you get. We have a lot of docker ability, a lot of containerization available in the cloud. We've also got service bus, Azure storage queue service, and just the fact that there's some services that are global that are multi-region by default. And then other times, there are things like availability zones, where you can get higher layers of availability within a single region. Software integration patterns for us to keep in mind. Service communication. A lot of programming and applications I deal with nowadays are asynchronous and event-driven, as far as that goes, which brings a need for some of these other related services like message queue, service bus, and so forth. Within the code base, your error handling, this is standard design pattern 101, retry policy, circuit breaker. Are you going to be tested on these specific patterns on AI-102? No, not as such using this specific language, but it will absolutely be required knowledge for you to understand what the questions are asking you on the exam. So to that point, let me pass along an exam tip. Focus on understanding the best practice service integration patterns. And by best practice, let's face it, it's in the Microsoft Docs. This is a vendor-specific exam, right? So I want you to focus on these Microsoft provided proven practice patterns and how they making tweaks in different spots leads to different solution architectures. Another fictional case study. VanArsdel needs to design an AI solution for inventory management. The requirements, real-time stock monitoring. Now, where does that, does that have you think in terms of events and things like event hub? I hope so. Uh-oh, multi-language support. That's translation and language. Cost-effective scaling. You see these buzzwords? This is how I want you to approach the AZ-102. You're going to see these buzzwords. Every word in those questions means something. I kid you not. So let me leave you with that before we get into the demo. And also, let me give you this exam alert. I, over nearly 30 years of experience, have built this three legged stool as my metaphor. As we go through prepping for the exam, we've got the theory, the what, how does it work? And our foundation there is in the MS Learn conceptual docs and training docs. AI-102, Azure engineer, AI engineer is definitely a practitioner role, so there's hands on. This means you should create an Azure free account and get into your demo environment and make sure you know how to do this dev work, okay? Just those things. Theory and practice are not enough. You need to review using practice tests, legitimate practice tests that simulate the live exam environment. And then the newest member of my model, using generative AI, whichever one you choose that's best for you to help you synthesize all of these other three legs of the stool. That make sense? All right, it's really important that I put a face to a name. That is, we take all the theory that we've learned. I've mentioned a blistering array of these different products, and we can see 'em here, and that's the point. We're in the Microsoft Azure portal, portal.azure.com. And I just did a Azure resource graph search for Azure AI. And that'll bring you to the AI services blade, which is a roll up, as you can see, of all of those services that I've mentioned. And then very notably, the entire Azure OpenAI just meekly sits in the middle of all of these specific services. And it's one of those specific services we're going to double-click into and learn more about now, and that is Content safety. I did mention the exam's not going to get into pricing too, too much, but when you create these instances, it would be like per team. Because when you create an instance of an AI service, you're going to have two interchangeable API keys that you want to protect with your life. In fact, it's best to store 'em an Azure Key Vault and the other is the endpoint, okay? We're not going to worry about private access and all of that stuff. We're staying dev. But fundamentally, that's going to be your unit of authorization. Now, truly, you don't want to do that. You want to use Entra ID auth to make sure that you've got full traceability. But if a business has need, different groups are managing their own spend, they would need their own content safety service. And these are regional. As you can see, I've deployed this service to the East U.S. I'm paying for it as part of my standard Azure subscription. The endpoint is critical. If we're going to use an an API key, at least in our development work, that's fine, but you'll have to deliver them safely, hopefully through Key Vault to your developers. So I want you to see, let's pause, that in the Azure control plane, this is where we can control the backend of things. Things that your compliance officers care about, like the at rest encryption, whether you're using vendor keys or customer-managed keys. Of course, the default pricing tier. You'll find that for the AI services, this again, we're looking at content safety. This is the content moderation service. There's a generous free tier, and then there'll be a standard. Sometimes it goes on beyond that. If you do go for the Azure free account, once you sign into the portal, I want you to go to the Free services blade 'cause this lists all of the services that have a monthly free amount for 12 calendar months from the day and time that you start up your free trial. Why is that important? Look, Face API, Custom Vision. You can use at a standard tier, at a production level tier, you can use the service up to that number of requests. So those would be the number of HTTP requests or transactions. Isn't that interesting? Well, let me step back a couple steps back to Content safety 'cause I wanted to say we're more, we're devs here. I keep reminding ourselves. And you might be fighting back, looking at your monitor yelling, saying, "Tim, I'm not a dev." I have nothing against devs, but so it gets mixed up. The AI engineer is a little bit Ops, a little bit DevOps, a little bit architect, a little bit developer. You really do need to know a lot about a lot. Now look here, unfortunately, with the content safety service, it appears to be hidden a little bit. But on the Overview blade, there's a link to Content Safety Studio, which I've bookmarked already, contentsafety.cognitive.azure.com. And as is the case with Azure Data Services, there's a web portal experience that is customizable and usable by you. There'll be plenty of advertisements here. If you're watching this video around the time that I'm recording it for the, whatever it's being called now. It was called Azure AI Foundry. And I guess, now it's being called Azure AI Studio. Never can keep track with that. But the notion of having both a web portal as well as the software development kit is important here. I work in Visual Studio Code and we're looking at a Python file called analyze_text.py. I share all of these resources in the course repo. But here in this Python, I want you to see, I've got some handling logic just to check to make sure you've got the SDKs installed. We're bringing those dependencies in per language. You'll find that on the exam, that's actually a good question. Do you need to know Python? Do you need to know Node? I would say yes, you need to be conversationally proficient or familiar with both languages, both Node and Python for the exam. Really, you won't be writing code, but you'll be interpreting a lot of it. So we can see, here's where we're bringing in those libraries. And as I think, I've mentioned, and I'm sure I'll continue mentioning, when you're accessing these various services, you're just building an HTTP request. Look on lines 25 and 26. I've got my endpoint and I'm safely pulling one of the API keys from my environment variable and it's doing a check for that. Let's actually run the app so I can show you what it does. It's making a live call, authenticating the request, and it's going to send in some input text here. That's pretty... Ooh, that's pretty bothersome text. And notice that the service came back with numeric results. I've pretty printed these and I wanted it to be educational here. This is a very common 80% Pareto principle here with Azure AI. You're going to get an evaluation result from the model, and it's up to you to interpret what that number means and you can sculpt automation on it and so forth. So with all of that as precursor, let's take a closer look at some of the capabilities of the Content Safety Studio and content safety feature. Some of the 80% stuff are like moderating text content. And the portal here is super useful because not only does it give you links to code examples in the docs, but we can run this against our own resource. Now notice here, in order to try out the API, we have to acknowledge a little bit of legalese. And you have to choose your content safety resource 'cause you're paying for these requests. It's not synthetic. It's doing real tests. And note that you can do simple or bulk tests here. And you can just choose whichever category you want. And with these APIs, part of your work on the exam and your career is going to be to study those APIs and how you can twiddle the properties. Here, the content safety text moderation here allows us to choose thresholds for categories. So it's reading the docs. Let's take a look at the test text. I'm not going to read it aloud, of course. Okay, well that's pretty graphic. Let's take a look. Let's click run test. It's going to send the request directly against the API and then we get this frontend that shows us some of the example results we can get. So in this case, the content has been deemed as being blocked. I want you to think about, now, how can I leverage this service to look at new novel input coming in? Like, let's say, it's a ask me anything content form. Let's say it's comments. Let's say it's, you know, whatever it might be. You run it through your API filter, and then you can trigger additional automations, like suppressing the post, not publishing it, this kind of stuff. That's the main big thing I want you to understand. Let me go back. Let's take a look at a couple other of the big use cases. Protected material detection. Oh, those are pretty, in fact, it's amazing how many of these are now skewed toward generative AI. Look here. Protected material detection says look for third party. It's basically looking for copyright infringements in large language model output. And then there's Prompt Shields, which is an API that addresses people trying to circumvent the LLM system message. Wow, that's cool. And then we have the groundedness detector that looks at LLMs for, are you hallucinating? Wow, those are so cool. But let's stay with the 80%. I'm going to use the Pareto principle a lot. You'll probably be annoyed of hearing it by the time you finish. But the most common scenarios, let's take a look at the example for image content here. So we've got some categories. It's interesting how Microsoft is able to simulate these as it looks like. It looks like we could also upload a file if we want to. Yep, we have to acknowledge that we are using our real content safety resource here. We can configure what our threshold is for allowing. But the theme, I think you'll agree, in content safety is you're evaluating some piece of media against filters that are categorized according to acceptable behaviors and so on and so forth. And you want your results to come back with that thumbs up, that thumbs down, and exactly why. That's something else you'll see. Now what you don't see is the underlying JSON, or more sophisticated examples. So notice, by clicking code, we get this implementation modal, which can be helpful if you're a .NET shop because there's a C# option, but I would rather go with Python or Java here. But ultimately, you're just sending requests in. And actually, most of the time, let me see, you can see the results, the resulting JSON as well. I'm fumbling a little bit here. It's showing us the pretty print. But ultimately, it's just JSON. Yeah, it really is going to depend upon the service, whether you can see all of the underlying JSON or not. But it's just a question, as I said, one more time of you reading the docs and getting familiar with each of these APIs.