From the course: How to Measure Anything in AI: Quantitative Techniques for Decision-Making

Unlock this course with a free trial

Join today to access over 25,200 courses taught by industry experts.

Quantifying and decomposing risk in AI implementation

Quantifying and decomposing risk in AI implementation

- Given the advancements in AI, it is important to discuss risk and risk measurements associated with AI. There's a lot to cover about the quantification of risk, but if we define it like it is used in decision theory and actuarial science, it is the possibility of a loss. Quantifying the risk of AI or anything else comes down to quantifying the probability of a loss and the size of a loss. To quantify the probability of a loss, we can consider how frequently some undesired event occurs. For example, an error generated by an AI can result in a loss, and we can gather data on how frequent those errors are. In one case, Microsoft conducted experiments on Meta prompts, and they've shared some of the data with us. Meta prompts are prompts about how they generate prompts. Suppose you're using AI to generate graphics and you tell it not to do something. For example, you want a picture of an operating room and you tell it not to add penguins. Well, guess what it's going to do? It's going to…

Contents