Aideen Fay

Prompt injection defenses that separate instructions from data by fine-tuning the LLM are popular, with several implementations, including OpenAI's instruction hierarchy and Meta's SecAlign. They show strong resistance to whitebox attacks like GCG. Are they secure though? Our latest pre-print provides an analysis of why GCG-like attacks fail against these models, showing that GCG doesn't perform much better than making random token substitutions. We also propose a new class of whitebox attacks that utilize attention! Our new attack, ASTRA, shows how an attacker can craft prompt injections in terms of manipulating the model's attention matrices to make it so that the model exclusively attends to the attacker's injected instructions while ignoring everything else in the context. This opens up a new line of investigation, where the attacker's objective is in terms of attention matrices rather than output probabilities. Preprint here: https://lnkd.in/gFqSpvDx

51

1 Comment

In the evolving threat landscape, what makes a detection truly effective? This insightful piece analyzes the three critical dimensions of quality detections in modern detection engineering programs: • Coverage & Durability: Targeting techniques over indicators provides longer-lasting value—IP addresses change daily, but adversary methodologies persist • Operational Cost: Strong detections balance confidence (accuracy) and noise (false positive rate) with actionable context for analysts • Adversary Impact: The best detections target low-volatility techniques that force attackers to substantially change their approach, not just their infrastructure Security teams should invest in detections that address behaviors rather than chasing ephemeral IOCs—the SolarWinds breach showed the limits of signature-based approaches vs. behavioral detections that identified unusual API patterns.

72

3 Comments

🎲 Is anything TRULY random, or is "randomness" just a confession of our incomplete knowledge? 🤯 This question is at the heart of statistical signal processing. Instead of viewing single events, the legendary textbook "An Introduction to Statistical Signal Processing" by Gray & Davisson teaches a radical perspective: viewing the "grand experiment" as the entire universe of possible outcomes at once. 🤖 This isn't just about formulas; it's about the deep math of measure theory, the art of assigning a mathematical 'weight' to uncertainty. It reveals a surprising truth: to avoid paradoxes in a continuous world, we can't assign a probability to every conceivable event. ✨ We must carefully choose what questions are valid. The reward for this thinking? The ability to perform modern magic. 📚 We can pull a clear signal from a blurry mess, like an image from a deep space probe, by modeling the noise and the signal as distinct mathematical objects and filtering them apart. . 🧠Every measurement is just a function that maps the abstract universe of possibilities into something we can see and use. It's the engine behind noise cancellation, medical imaging, and modern comms. A powerful reminder that structure often hides beneath what looks like chaos #SignalProcessing #Mathematics #MeasureTheory #Probability #StochasticProcesses #AI #MachineLearning I'm creating a lot of scientific content which is available on many of media platforms 👇 👇👇 Substack: https://lnkd.in/dTjrF6AP (English) Spotify: https://lnkd.in/dgumrSMR (English) https://lnkd.in/d-gMtCrE (Hebrew) Youtube: https://lnkd.in/dPGJr7WM (English) https://lnkd.in/dydSqeky (Hebrew) Telegram: https://lnkd.in/d_YxVMAR (English) https://lnkd.in/dVVqhNw5 (Hebrew)

105

1 Comment

I'm excited to share that our Grid-STIX library is now open source: https://lnkd.in/gvHSkkBm Grid-STIX extends the STIX 2.1 threat intel standard for the electric grid, modeling assets, protocols, vulnerabilities, and mitigations in a structured, machine-readable way. It gives researchers and operators a common language for grid cybersecurity and supports automated reasoning across OT and IT domains. Check it out, try it in your own workflows, and contribute back — we’d love to see how the community applies it. Thank you to the DOE Office of Cybersecurity, Energy Security, and Emergency Response (CESER) office for funding this ongoing work, and to Manimaran Govindarasu and team at Iowa State University for the collaborations leading to it.

49

1 Comment

Great news, one of the articles we recently wrote with Sebastian Wallin is on the Hacker news front page! https://lnkd.in/egWNyKZk In this article, we talk about canvas fingerprinting, a popular type of signal frequently used for anti-fraud and bot detection purposes. However, when it comes to canvas fingerprinting, it's key to detect whether or not it has been modified/randomized by an attacker. Thus, we do a deep dive into a few JS-based detection techniques that can be used to detect whether or not a canvas fingerprint has been altered.

38

8 Comments

Proud that our Applied Cryptography and Digital Trust module is now part of our MEng in Software Engineering. Really, software engineers need to increasingly learn more cryptography in order to properly understand the basics of building trusted systems (esp for PKI, Tokens, Tunnels, Blockchain and Digital Signatures). Our module has lots of hands-on, but covers all of the required theory. https://lnkd.in/dnH9ZBf

50

1 Comment

Function calling is rapidly becoming the backbone of agentic AI systems, but with that power comes a new class of vulnerabilities. HiddenLayer’s latest research reveals a transferable exploit affecting major LLMs like GPT-4o, Claude, Qwen, and DeepSeek. By abusing function parameters, our team was able to extract sensitive system prompts and operational metadata across both APIs and real-world applications. 📉 Why it matters: This vulnerability exposes a fundamental architectural gap in how language models interpret and execute function calls, turning a helpful feature into a potential attack vector. 💡 If you're a CISO, security engineer, or building with LLMs, now is the time to rethink how function-calling behavior is secured. The models may be powerful, but the safeguards around them need to be just as strong. 👉 Read the full breakdown: https://lnkd.in/gEdr5wZp #AIsecurity #LLMs #Cybersecurity #FunctionCalling #AgenticAI #MCP

62

2 Comments

We talk a lot about Shor's Algorithm in our discussion of post quantum cryptography (PQC). In this Root Causes Podcast episode Jason Soroko explains Shor's algorithm for non-quantum physicists. Audio: Video:

31

5 Comments