LLM Agents Exploiting Cybersecurity Vulnerabilities

OpenAI claimed in their GPT-4 system card that it isn't effective at finding novel vulnerabilities. We show this is false. AI agents can autonomously find and exploit zero-day vulnerabilities. Zero-day vulnerabilities are particularly dangerous since they aren’t known ahead of time. They’re also challenging since the agent doesn’t know what to exploit. Our prior work on agents gets confused when switching tasks in the zero-day setting. To resolve this, we introduce a new technique HPTSA, hierarchical planning and task-specific agents. The planner explores the website and dispatches to other agents that perform the exploit. HPTSA can hack over half of the vulnerabilities in our benchmark, compared to 0% for open-source vulnerability scanners and 20% for our previous agents. Our results show that testing LLMs in the chatbot setting, as the original GPT-4 safety assessment did, is insufficient for understanding LLM capabilities. We anticipate that other models, like Claude-3 Opus and Gemini-1.5 Pro will be similarly capable but were unable to test at the time of writing. Paper: https://lnkd.in/ecRUthcM Medium: https://lnkd.in/euZCPssz

'AI models, the subject of ongoing safety concerns about harmful and biased output, pose a risk beyond content emission. When wedded with tools that enable automated interaction with other systems, they can act on their own as malicious agents. Computer scientists affiliated with the University of Illinois Urbana-Champaign (UIUC) have demonstrated this by weaponizing several large language models (LLMs) to compromise vulnerable websites without human guidance. Prior research suggests LLMs can be used, despite safety controls, to assist [PDF] with the creation of malware. Researchers Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang went a step further and showed that LLM-powered agents – LLMs provisioned with tools for accessing APIs, automated web browsing, and feedback-based planning – can wander the web on their own and break into buggy web apps without oversight. They describe their findings in a paper titled, "LLM Agents can Autonomously Hack Websites." "In this work, we show that LLM agents can autonomously hack websites, performing complex tasks without prior knowledge of the vulnerability," the UIUC academics explain in their paper.' https://lnkd.in/gRheYjS5

A team of Carnegie Mellon University researchers, working with Anthropic, has demonstrated that large language models (LLMs) are capable of autonomously planning and executing complex network attacks, shedding light on emerging capabilities of foundation models and their implications for cybersecurity research. https://lnkd.in/gQM7ce5j