Ken Thompson, upon receiving the Turing award, wrote a terrifying paper. “Reflections on Trusting Trust” illustrates a scenario of original sin. Because the C compiler is written in C itself, a compromised compiler can self-replicate with no trace in source code. — If you can’t trust your compiler, you can’t trust any compiler you build with it either. Sin in the family tree, no matter how distant, can propagate to your clean code even decades later. — This creates a recursive paradox. What if the tools used to validate a compiler, are themselves products of said compiler? Source review never guarantees security. You have to verify the entire ancestry. — In critical applications, a special technique known as “Diverse double-compiling” reduces risk. Two independent compiler lineages are unlikely to contain identical backdoors. Therefore, comparing the resulting binaries offers *some* level of trust.
Trustworthy vs untrustworthy software
Explore top LinkedIn content from expert professionals.
Summary
Trustworthy software is code that can be relied upon to function safely and as intended, while untrustworthy software poses risks due to hidden vulnerabilities, malicious components, or unchecked dependencies. Recent discussions highlight how blindly trusting tools—including AI-generated code and open source libraries—can expose entire systems to unexpected threats.
- Verify dependencies: Always confirm that any package or library referenced in your code, especially those suggested by AI tools or open source repositories, actually exists and is reputable before integrating it.
- Review supply chain: Adopt a mindset of “zero trust” by scrutinizing the origin and lineage of all code, ensuring thorough checks before using new or unfamiliar software components.
- Implement manual controls: Require manual approval and regular review of new software dependencies to catch potential vulnerabilities or malicious additions early.
-
-
Global Head of Technology, Cyber, AI Audit and Associate General Auditor // ISACA NY Metropolitan Chapter Immediate Past President
5,482 followers⚠️ 𝐒𝐥𝐨𝐩𝐬𝐪𝐮𝐚𝐭𝐭𝐢𝐧𝐠: 𝐀𝐈 𝐇𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐢𝐨𝐧𝐬 𝐅𝐮𝐞𝐥 𝐚 𝐍𝐞𝐰 𝐒𝐮𝐩𝐩𝐥𝐲 𝐂𝐡𝐚𝐢𝐧 𝐓𝐡𝐫𝐞𝐚𝐭 ⚠️ 𝐒𝐥𝐨𝐩𝐬𝐪𝐮𝐚𝐭𝐭𝐢𝐧𝐠 is a new software supply chain attack that exploits AI hallucinations. AI coding tools like 𝐂𝐡𝐚𝐭𝐆𝐏𝐓 𝐨𝐫 𝐆𝐢𝐭𝐇𝐮𝐛 𝐂𝐨𝐩𝐢𝐥𝐨𝐭 sometimes fabricate package or library names that don’t actually exist. Attackers are now squatting on these AI-invented names by 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫𝐢𝐧𝐠 𝐭𝐡𝐞𝐦 𝐚𝐬 𝐫𝐞𝐚𝐥 𝐩𝐚𝐜𝐤𝐚𝐠𝐞𝐬 – 𝐛𝐮𝐭 𝐩𝐚𝐜𝐤𝐢𝐧𝐠 𝐭𝐡𝐞𝐦 𝐰𝐢𝐭𝐡 𝐦𝐚𝐥𝐰𝐚𝐫𝐞. It's like typosquatting – 𝘵𝘩𝘦 "𝘵𝘺𝘱𝘰" 𝘪𝘴 𝘣𝘺 𝘈𝘐, 𝘯𝘰𝘵 𝘢 𝘩𝘶𝘮𝘢𝘯. When a developer blindly uses code from an AI that references one of these fictitious libraries, they could inadvertently download the attacker’s malicious package, leading to a compromise. 💡 𝐏𝐞𝐫𝐯𝐚𝐬𝐢𝐯𝐞𝐧𝐞𝐬𝐬 & 𝐒𝐜𝐚𝐥𝐞: Researchers found roughly 20% of recommended dependencies from popular AI code assistants were non-existent hallucinations, yielding over 𝟐𝟎𝟓,𝟎𝟎𝟎 𝐟𝐚𝐤𝐞 𝐩𝐚𝐜𝐤𝐚𝐠𝐞 𝐧𝐚𝐦𝐞𝐬 in their study. Worryingly, these hallucinations were often persistent (reappearing across multiple runs) and believable (names often resembled real packages). 🔺 𝐖𝐡𝐲 𝐢𝐭 𝐌𝐚𝐭𝐭𝐞𝐫𝐬: It can poison your software supply chain – a malicious dependency can compromise your entire application or infect downstream systems. This risk is amplified by developers’ over-trust in AI-generated code. One researcher proved it: they published a fake package name that an AI invented, and it was downloaded over 32,000 times. If that package had been malicious, imagine the fallout. 🛡️ 𝐌𝐢𝐭𝐢𝐠𝐚𝐭𝐢𝐨𝐧 – 𝐒𝐭𝐚𝐲𝐢𝐧𝐠 𝐒𝐚𝐟𝐞: 🛡️ 𝐃𝐨𝐧’𝐭 𝐭𝐫𝐮𝐬𝐭, 𝐯𝐞𝐫𝐢𝐟𝐲: Treat AI suggestions as helpful hints, not gospel. Double-check any package or link AI suggests – verify on official repositories that it actually exists and is trustworthy. 🛡️ 𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭 𝐜𝐨𝐝𝐞 𝐫𝐞𝐯𝐢𝐞𝐰 & 𝐜𝐨𝐧𝐭𝐫𝐨𝐥𝐬: Require manual review/approval for any new dependency introduced via AI-generated code. Don’t add unknown libraries without due diligence. 🛡️ 𝐔𝐬𝐞 𝐬𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐭𝐨𝐨𝐥𝐢𝐧𝐠: Use dependency scanning tools to catch malicious packages. 🛡️ 𝐄𝐝𝐮𝐜𝐚𝐭𝐞 𝐚𝐧𝐝 𝐚𝐝𝐚𝐩𝐭: 👨💻 𝐓𝐫𝐚𝐢𝐧 𝐝𝐞𝐯𝐬: Slopsquatting + LLM hallucinations are real threats. 🧠 𝐌𝐢𝐧𝐝𝐬𝐞𝐭: Trust AI… but always verify. 🤖 𝐏𝐫𝐨 𝐭𝐢𝐩: Ask the AI if a package is real- it’ll admit it made it up. 𝐁𝐨𝐭𝐭𝐨𝐦 𝐋𝐢𝐧𝐞: AI coding assistants boost productivity, but integrity is key. Slopsquatting shows a single hallucinated dependency can open the door to malware.🔒💡 𝘍𝘰𝘳 𝘔𝘰𝘳𝘦 𝘐𝘯𝘧𝘰: csoonline.com securityweek.com 𝐈𝐥𝐥𝐮𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐀𝐭𝐭𝐚𝐜𝐡𝐞𝐝: 🚨 Attack Flow: Bad actor uploads fake, AI-hallucinated package (Pkg X). Dev blindly trusts LLM, installs it. 💥 Malicious code gets in. ISACA New York Metropolitan Chapter Tim Wei Teena Eugene Christina Alyssa
-
As an industry, we’ve poured billions into #ZeroTrust for users, devices, and networks. But when it comes to software - the thing powering every modern business, we’ve made one glaring exception: OPEN SOURCE SOFTWARE! Every day, enterprises ingest unvetted, unauthenticated code from strangers on the internet. No questions asked. No provenance checked. No validation enforced. We assume OSS is safe because everyone uses it. But last week’s #npm attacks should be a wake-up call. That’s not Zero Trust. That’s blind trust. If 80% of your codebase is open source, it’s time to extend Zero Trust to the software supply chain. That means: • Pin every dependency. • Delay adoption of brand-new versions. • Pull trusted versions of OSS libraries where available. #Google's Assured OSS offering is a good one for this. • Assess health and risk of malicious behavior before you approve a package. • Don’t just scan for CVEs—ask if the code is actually exploitable. Use tools that give you evidence and control, not just noise. I wrote more about this in the blog linked 👇 You can’t have a Zero Trust architecture while implicitly trusting 80% of your code. It's time to close the gap and mandate Zero Trust for OSS. #OSS #npmattacks #softwaresupplychainsecurity