Global AI Safety Collaboration

"Following the Seoul AI Safety Summit, we have seen the announcement of a substantial network of state-run AI Safety Institutes (AISIs) across the globe. What progress has been made? How do their plans and motivations differ? And what can we learn about how to set up AISIs effectively? This brief analyses the development, structure, and goals of the first wave of AISIs. Key findings: Diverse Approaches: Countries have adopted varied strategies in establishing their AISIs, ranging from building new institutions (UK, US) to repurposing existing ones (EU, Singapore). Funding Disparities: Significant variations in funding levels may impact the relative influence and capabilities of different AISIs. The UK leads with £100 million secured until 2030, while others like the US face funding uncertainties. International Cooperation: While AISIs aim to foster global collaboration, tensions between national interests and international cooperation remains a challenge for AI governance. Efforts like the UK-US partnership on model evaluations highlight potential for effective cross-border cooperation. Regulatory Approaches: There’s a spectrum from voluntary commitments (UK, US) to hard regulation (EU), with ongoing debates about the most effective approach for ensuring AI safety while fostering innovation. Focus Areas: Most AISIs are prioritising AI model evaluations, standard-setting, and international coordination. However, the specific risks and research areas vary among institutions. Future Uncertainties: The evolving nature of AI technology and relevant geopolitical factors create significant uncertainties for the future roles and impacts of AISIs. Adaptability will be key to their continued relevance and effectiveness." This work from The International Center for Future Generations - ICFG is quite helpful for understanding the existing institutes and their overlaps and differences. Link in comments.

International AI Safety Report 2025 A report on the state of advanced AI capabilities and risks – written by 100 AI experts including representatives nominated by 33 countries and intergovernmental organizations The International AI Safety Report is the world’s first comprehensive synthesis of current literature of the risks and capabilities of advanced AI systems. Chaired by Turing-award winning computer scientist, Yoshua Bengio, it is the culmination of work by 100 AI experts to advance a shared international understanding of the risks of advanced Artificial Intelligence (AI). The Chair is supported by an international Expert Advisory Panel made up of representatives from 30 countries, the United Nations (UN), European Union (EU), and Organization for Economic Cooperation and Development (OECD). The report does not make policy recommendations. Instead it summarises the scientific evidence on the safety of general-purpose AI to help create a shared international understanding of risks from advanced AI and how they can be mitigated. —   The report is concerned with AI risks and AI safety and focuses on identifying these risks and evaluating methods for mitigating them. It summarises the scientific evidence on 3 core questions –  - What can general-purpose AI do?  - What are risks associated with general-purpose AI?  - What mitigation techniques are there against these risks? —  Focus On - Provide scientific information that will support informed policymaking – it does not recommend specific policies - Facilitate constructive and evidence-based discussion about the uncertainty of general-purpose AI and its outcomes - Contribute to an internationally shared scientific understanding of advanced AI safety —   The report was written by a diverse group of academics, guided by world-leading experts in AI. There was no industry or government influence over the content. The secretariat organised a thorough review, which included valuable input from global civil society and industry leaders.

AI is rewriting the playbook for national-security technology. Unlike radar, nuclear systems, or the early internet, all technologies built largely by governments, frontier AI is being developed primarily by the private sector, and advancing at a pace traditional policy frameworks were never designed to manage. In this new reality, democratic societies need new institutions and new forms of cooperation. One of the most promising developments is the emergence of AI Safety Institutes (AISIs), independent research bodies that evaluate advanced AI systems and help governments understand their risks and capabilities. In this article by OpenAI's head of national security policy Sasha Baker, she lays out how a global network of AISIs could become the backbone of democratic AI governance - strengthening safety research, improving international cooperation, and helping governments responsibly navigate the Intelligence Age.

Last week we published the first International AI Safety Report, supported by 30 nations, the OECD, UN, and EU. It’s the result of a massive collaboration involving over 100 independent experts—including Nobel laureates and Turing Award winners. As a co-author who worked on the “Risks of Copyright” section, I’m especially grateful to the variety of experts that I learned from. The writing leads went to great length to incorporate many perspectives, ultimately strengthening the report’s balance and depth. This report digs into a broad spectrum of risks: 🦺 Misuse ��️ deepfakes, mis/disinformation, cyber threats, CBRNs. 🛠️ Malfunctions ➡️ reliability, bias, lack of control. 🔭 Systemic risks (my personal favourite) ➡️ privacy, competition, labor markets, copyright, and environmental impacts. The report doesn’t stop at outlining challenges. It also discusses the range of technical solutions. Each risk area is supported by carefully curated citations to help readers dive deeper. Looking ahead to the AI Action Summit in Paris, I hope this report sets the stage for thoughtful discussions on AI reliability and safety. If you’re looking for an entry point into these multifaceted topics, this is it. A special thank you to the leads Yoshua Bengio Daniel Privitera Sören Mindermann, and fellow writers and advisors I worked with directly Danielle Goldfarb, Lee Tiedrich, Stephen Casper, and Tobin South.

In my role as Chair of the International AI Safety Report, an effort backed by over 30 countries and international organisations including the European Union, OECD - OCDE and United Nations, I work with 100 researchers to help policymakers understand the capabilities and risks of general-purpose AI. The field is clearly changing far too quickly for a single annual report to suffice. That’s why today we’re introducing Key Updates: shorter, focused reports on critical developments in AI that will be published between editions of the full report. Our first Key Update focuses on advancements in AI capabilities, and what they mean for AI safety. You can read it here: https://lnkd.in/eKVGF7dy Some of the key findings it covers include: ➡️ Impressive performance improvements. Several AI systems can now solve International Mathematical Olympiad problems at gold medal level and complete a majority of problems in several databases of real-world software engineering tasks. ➡️ The rise of “reasoning” models. Recent gains have come mainly from training and deployment techniques that allow AI models to generate interim steps before producing final answers. This demonstrates that AI capabilities can advance significantly through post-training techniques and additional computing power at inference time, not just through scaling model size. ➡️ Some signals of real-world adoption. In a recent StackOverflow survey, a majority of software developers report using AI tools daily to help design experiments, process data, and write reports. Yet we still don’t know much about AI use in many other domains, nor crucially about how AI use affects productivity overall. ➡️ Stronger safeguards from developers. Leading AI developers recently activated enhanced protections on their most capable models as a precautionary measure, given possibilities like misuse to build weapons. ➡️ Emerging oversight challenges. AI models increasingly demonstrate an ability to distinguish evaluation tasks from real-world tasks, possibly complicating our ability to reliably test their capabilities before deployment. These developments raise further questions about control, monitoring, and governance as AI systems become more capable.

The Annual AI Governance Report 2025 by the International Telecommunication Union (ITU) provides a comprehensive overview of how nations, institutions, and innovators are guiding AI towards a responsible global impact. The Rise of AI Agents:- AI Agents have transitioned from copilots to autonomous digital workers, engaging in tasks such as booking trips, coding, and negotiating purchases. This shift raises critical questions about traceability, liability, and visibility. Governance frameworks are rapidly evolving, proposing agent identifiers, activity logs, and safe-harbour regimes to ensure accountability. Bridging the AI Divide:- As AI transforms industries, many nations still lack adequate computing resources. The report notes that over 150 countries do not have significant AI compute hubs, highlighting the urgent need for inclusive AI infrastructure, skills, and standards that allow broader participation beyond the Global North. The Global Governance Mosaic:- International coordination is accelerating through initiatives like the Bletchley, Seoul, and Paris AI Summits, along with regional collaborations (ASEAN, AU, GCC, EU). However, challenges remain in policy interoperability and the establishment of shared safety infrastructure. Ten Pillars for AI Governance:- The report concludes with a framework focused on transparency, inclusion, environmental sustainability, compute governance, and agile regulation, setting the stage for the UN Global Dialogue on AI Governance in 2026. ⛵ “We do not need to sail in the same ship, or at the same speed, but we do need to navigate the same oceans by the same compass.” — Doreen Bogdan-Martin, ITU Secretary-General Read the attached full report for deep insights into the evolving landscape of AI governance across agents, safety, and standards. #AIGovernance #AIForGood #ResponsibleAI #AIStandards #AgenticAI #AI2025 #GlobalAI #Inclusion #EthicalAI #DigitalCooperation

The International AI Safety Report captures is led by experts from more than 30 nations, and it surveys how artificial intelligence is advancing. TLDR: capability progress must be matched by safety and governance. What the report outlines • General purpose AI systems now reason through complex problems in mathematics, coding, and science with near human proficiency. Models have solved International Mathematical Olympiad problems at the gold medal level and complete over 60 percent of tasks on the SWE bench Verified benchmark. • Progress is driven by smarter post training methods and greater compute at inference rather than simple model scaling. • Systems show early autonomy, including planning, tool use, and multi-step operation with limited supervision. • Multimodal capability across text, image, video, and audio expands applications while increasing misuse potential. Why this matters • AI systems now perform sensitive tasks once limited to domain experts, creating governance and accountability challenges. • Knowledge in biology and cybersecurity is increasingly accessible through general purpose systems, lowering barriers to dual use misuse. Key risks and practices • Biological risk: Laboratory evaluations and emerging tools indicate that advanced systems could assist aspects of harmful biological design or troubleshooting. Developers have introduced heightened safety levels and deployment mitigations for frontier systems. • Cybersecurity risk: In controlled tests one AI system identified 77 percent of synthetic software vulnerabilities and patched 61 percent across 54 million lines of code, and a national authority judges it almost certain that AI will make cyber offence more effective by 2027. The same techniques can also accelerate defensive patching. • Labor disruption: Adoption is uneven. Fifty one percent of professional developers report using AI tools daily, yet many remain mistrustful, and success on realistic workplace tasks often stays below 40 percent. • AI companions: Some services report tens of millions of active users, raising concerns about dependence, harmful advice, and privacy. Who should act • Developers embedding safety evaluation and transparency in training and deployment. • Governments harmonizing international standards, enforcing disclosure, and auditing high risk uses. • Researchers expanding behavioral and interpretability studies under realistic and adversarial conditions. • International bodies coordinating oversight of dual use capabilities. Action items • Move beyond benchmarks to real world stress testing and misuse scenarios. • Establish a shared AI Safety Level framework aligning disclosure and mitigation across jurisdictions. • Fund independent evaluation and safety research so accountability is not confined to private labs. • Build safety by default systems that keep humans in charge of consequential actions.

AI Safety Institutes represent a new era for AI governance In the past year, AI Safety Institutes have been established by the UK, Japanese and U.S. governments. These new bodies pursue advanced research and engagement on frontier AI risk, safety and governance. A key focus is model evaluations and red-teaming. A new Institute for AI Policy and Strategy (IAPS) report provides an overview of these bodies and their work. Below, I list the key resources, research and updates, published by the UK, Japan and U.S. AI Safety Institutes. UK AI Safety Institute 🤖 Mission: The AI Safety Institute's mission is to minimise surprise to the UK and humanity from rapid and unexpected advances in AI. ➡️ International Scientific Report on the Safety of Advanced AI 🔗 https://lnkd.in/eASatJwT ➡️ AI Safety Institute approach to evaluations 🔗 https://lnkd.in/eFvFhpj8 ➡️ Advanced AI evaluations: technical update 🔗 https://lnkd.in/eEWgPbFU ➡️ Safety cases at AISI 🔗 https://lnkd.in/eDXhNc5S ➡️ Inspect: open-source framework for LLM evaluations 🔗 https://lnkd.in/enemuc_i ➡️ Should AI systems behave like people? 🔗 https://lnkd.in/e7n6pf82 ➡️ Early Insights from Developing Question-Answer Evaluations for Frontier AI 🔗 https://lnkd.in/eApf3_Uw Japan AI Safety Institute 🤖 Mission: Our mission is to equip governments with an empirical understanding of the safety of advanced AI systems. ➡️ Guide to Red Teaming Methodology on AI Safety (summary) 🔗 https://lnkd.in/eWskb-SK ➡️ Guide to Red Teaming Methodology on AI Safety (full report) 🔗 https://lnkd.in/ePahDRRC ➡️ Guide to Evaluation Perspectives on AI Safety (summary) 🔗 https://lnkd.in/e_ZdR3ry ➡️ Guide to Evaluation Perspectives on AI Safety (full report) 🔗 https://lnkd.in/eyQ7VzhN U.S. AI Safety Institute 🤖 Mission: AISI’s mission is to help define and advance the science of AI safety. ➡️ The United States Artificial Intelligence Safety Institute: Vision, Mission, and Strategic Goals 🔗 https://lnkd.in/eY-XKFug ➡️ Managing Misuse Risk for Dual-Use 4 Foundation Models 🔗 https://lnkd.in/em-_MYUF ➡️ Notice: Safety Considerations for Chemical and/or Biological AI Models 🔗 https://lnkd.in/e6KAY2JH

Are you curious how others are managing AI risks? 🤔 I recently came across the International AI Safety Report, a collaborative effort between the Department for Science, Innovation and Technology and the AI Security Institute. This report showcases techniques employed by developers and risk management professionals worldwide to enhance the reliability of AI models and systems, while also mitigating the risk of misuse. The report covers various topics, including defense-in-depth strategies, limiting undesired behaviors, deploying safeguards, post-deployment monitoring, and early governance approaches, among others. At the end of the report, they highlight how developers are using "assurance tools to substantiate claims" about the capabilities of their AI products. As someone with an assurance background, I appreciate the good-faith effort that technical resources are making not only to build safety into these technologies but also to measure their level of safety and report on it periodically. I believe this space is ripe for further clarity and standardization in the coming months and years. #ai #airisk #aisafety #riskmanagement

Great news in Australia overnight as the government announces it will establish the Australian AI Safety Institute as a core component of its upcoming National AI Capability Plan. Tim Ayres, Andrew Charlton, National AI Centre's Lee Hickin and Beth Worrall + Responsible AI team. The institute will: ▪️carry out rigorous assessment and evaluation of emerging AI capabilities to identify risks and inform timely action. ▪️engage internationally; building partnerships, joining networks and contributing to global AI safety efforts. ▪️publish research and policy guidance; supporting industry, academia and the public to understand and respond to AI challenges. ▪️coordinate across sectors and regulators to ensure that AI governance is fit‑for‑purpose and aligned with Australia’s interests. Read more via: ▪️Justin Hendry for InnovationAus.com: https://lnkd.in/graVwK5r ▪️Noah Y. for The Australian: https://lnkd.in/gwCNdZS7 Earlier in the year I joined the inaugural AI Governance Summer Fellowship by Good Ancestors. Our team (Archana Atmakuri, Joshua Krook) wrote a white paper recommending Australia pursue a 'narrow-but-deep' AI safety strategy, and establish an AI Safety and Security Institute via three-phase approach to: ▪️ coordinate and support Australia’s existing AI safety capability ▪️ establish a dedicated Australian AI Security Institute (AISI) ▪️ pursue expanded Asia-Pac leadership through a regional AI safety initiative Read more about that: https://lnkd.in/gpJKrZee Australia possesses excellent technical AI safety talent (Qinghua Lu, Liming Zhu, Javen Qinfeng Shi + teams and many more). I think this initiative will augment that strong foundation with the governance, policy and international engagement capability to allow Australia to play a more influential role in the global AI conversation. [+Greg Sadler, Gareth Kindler, David Hua, Arshia Jain, T Nang Seng Pan, Rickard Vikström].