Luciana Ledesma’s Post

Allowed does not mean expected. In a recent AI-related supply chain incident, an automated triage bot executed commands based on a prompt injection in a GitHub issue title. What is interesting is that nothing the system did was obviously “forbidden”. Installing packages? Allowed. Running commands in CI? Allowed. Publishing packages? Allowed. But the behaviour was clearly unexpected. A GitHub issue should not trigger dependency installation from an unknown repository. Security systems often ask: Is this action allowed? Governance systems should ask: Is this action expected in this context? https://lnkd.in/dECd-BJs Allowed actions can still produce unexpected behaviour. And unexpected behaviour is where risk usually begins.

To view or add a comment, sign in

Explore content categories