Trustworthy AI Systems

✈️ 🇪🇺 « Trustworthy AI in Defence »: The European Way 🗞️The European Defence Agency’s White Paper is out! At a time when global powers are racing to develop & deploy AI-enabled defence capabilities,the European way =tech innovation + ethical responsibility, operational effectiveness + legal compliance, strategic autonomy + respect for human dignity & democratic values. 🔹AI in defence as legally compliant, ethically sound, technically robust, societally acceptable. 1 🤝🏻Principles of Trustworthiness 🔹foundational principles for trustworthy AI in defence: accountability, reliability, transparency, explainability, fairness, privacy, human oversight. Not optional but integral to the legitimacy of AI systems used by European armed forces. 2. Ethical and Legal Compliance 🔹 Europe’s commitment is to effective military capabilities but also to a rules-based international order. The EU explicitly rejects the idea that technological advancement justifies the erosion of ethical norms. 🔹 importance of ethical review mechanisms, institutional safeguards, alignment with #EU legal frameworks=a legal-ethical backbone ensuring trustworthiness is a practical requirement embedded into every phase of AI development/deployment. 3. Risk Assessment & Mitigation 🔹 EU’s precautionary principle=>rigorous & ongoing risk assessments of AI systems, incl. risks related to technical failures, misuse, bias, and unintended escalation in operational contexts. To anticipate harm before it materializes and equip systems with built-in safeguards 🔹Risk mitigation not only a technical task but an ethical &strategic imperative in high-stakes domains (targeting, threat detection, autonomous mobility). 4. 👁️Human Oversight & Control 🔹The EU rejects fully autonomous weapon systems operating without human intervention in critical functions like the use of force. The Paper calls for clear human-in-the-loop models, where operators retain oversight, intervention capability, and accountability. = safeguards democratic accountability & operational reliability, ensuring no algorithm makes life-and-death decisions. 5. Transparency and Explainability 🔹transparent #AI systems, not black-box models : decision-making processes understandable by users & traceable by designers. Key for after-action reviews, audits, & compliance. Strong stance on explainability 6. European Cooperation &Standardization 🔹Enhanced cooperation and harmonization in defence AI : shared definitions, frameworks to ensure interoperability, avoid duplication, promote a common culture of responsibility. 🔹 joint work on certification processes, training, testing environments 7. Continuous Monitoring and Evaluation 🔹ongoing monitoring, validation, recalibration of AI tools throughout their deployment. «trustworthiness must be maintained, not assumed » =The European way: lead not by imitating others’ race toward automation at any cost, but by demonstrating security, innovation, and values can go hand in hand

"Developing trustworthy AI applications with foundation models" by authors(  Michael Mock, Sebastian Schmidt, Felix Müller, Rebekka Görge, Anna Schmitz, Elena Haedecke, Angelika Voss, Dirk Hecker, Maximillian Poretschkin) This whitepaper shows how the trustworthiness of an AI application developed with foundation models can be evaluated and ensured. For this purpose, the application-specific, risk-based approach for testing and ensuring the trustworthiness of AI applications, as developed in the 'AI Assessment Catalog - Guideline for Trustworthy Artificial Intelligence' by Fraunhofer IAIS, is transferred to the context of foundation models. (i) Chapter 1 of the white paper explains the fundamental relationship between foundation models and AI applications based on them in terms of trustworthiness. (ii) Chapter 2 provides an introduction to the technical construction of foundation models (iii) Chapter 3 shows how AI applications can be developed based on them. (iv) Chapter 4 provides an overview of the resulting risks regarding trustworthiness. (v) Chapter 5 shows which requirements for AI applications and foundation models are to be expected according to the draft of the European Union's AI Regulation (vi) Chapter 6 finally shows the system and procedure for meeting trustworthiness requirements. #ai #artificialintelligence #llm #trustworthiness #generativeai #riskmanagement

AI you can test, certify, and trust 🚨 Mira Murati’s Thinking Machines Lab has just published its first research on whether AI can be trusted to deliver answers that are consistent and reproducible. The first wave of the AI race was about scale: more parameters, more compute, more speed. Murati’s $2B venture is rewriting the rules. The new competition is about certainty, how reliable and transparent a model is. To test consistency, the team ran Alibaba’s Qwen-235B model on the exact same prompt 1,000 times: “Tell me about Richard Feynman.” Feynman, a Nobel Prize–winning physicist, was born in Queens, New York: is a fixed fact. A reliable system should return it consistently. Instead, the model produced 80 variations and the answers split between “Queens, New York” and "New York City.” A detail? Not really. If AI can’t be consistent on a birthplace, how can it do so with compliance filings, medical records, or financial risk assessments? The breakthrough: determinism 🔹 Researcher Horace He traced the issue to the way GPUs order operations when handling multiple queries. 🔹 The fix: redesign three core functions to have identical outputs regardless of server load. 🔹 The result: 1,000 runs, 1,000 identical completions. ➡️ AI moved from probabilistic to predictable: from a machine changing its mind to a system that can be tested, certified, and trusted. Determinism comes at a cost. Speed slowed down: ▫️ Standard setup: ~26s ▫️ Deterministic (early): ~55s ▫️ Deterministic (improved): ~42s But in high-stakes settings, reliability outweighs raw performance. A bank or hospital can wait 20s longer for consistent, auditable and certifiable answers. Murati's philosophy: openness as an edge Where OpenAI has grown more secretive, Thinking Machines Lab leans into transparency. Their new blog details the research, and the code has been released for anyone to test. Determinism + openness = a double trust signal: The model behaves the same every time. The method is open and verifiable. This positions Thinking Machines Lab as the counter to black-box AI. Why this matters ✔️ Enterprises: Reproducibility may become a procurement criterion. Inconsistent models bring risks: liability, brand damage, failed audits. ✔️ Regulators: Under the EU AI Act, reproducibility could be to AI what accounting standards are to finance: the foundation of trust. The first wave of AI was defined by speed and scale. The second by consistency, transparency, and trust. This is Murati's $2B bet. 👉 Full research: https://lnkd.in/eNQN6Zn2 #AI #Innovation #ResponsibleAI #Leadership #MinaMurati

⚠️𝗧𝗵𝗲 𝗱𝗮𝗻𝗴𝗲𝗿 𝗶𝘀𝗻’𝘁 𝗿𝗼𝗴𝘂𝗲 𝗔𝗜. It’s the trusted systems making invisible errors at scale, at speed, in silence. We’ve built autonomy without a conscience. We’ve deployed intelligence without oversight. We test for function not fallout. But we 𝗵𝗮𝘃𝗲𝗻'𝘁 𝗮𝗻𝘀𝘄𝗲𝗿𝗲𝗱: Who’s accountable when they get it wrong? Where’s the audit trail? Where’s the human override? Where’s the responsibility when lives are lost? The 𝗯𝗹𝗶𝗻𝗱 𝘀𝗽𝗼𝘁𝘀 are growing. The 𝗯𝗮𝘁𝘁𝗹𝗲𝗳𝗶𝗲𝗹𝗱 𝗶𝘀 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴. So must our thinking. The European Defence Agency's Whitepaper on trustworthy AI is a wake-up call. These aren’t technical details. They’re mission-critical decisions. And the cost of ignoring them? 𝗖𝗮𝘁𝗮𝘀𝘁𝗿𝗼𝗽𝗵𝗶𝗰. Let’s dive into the must know highlights. 🔸Legal Perspective on #AI Use Cases ➝Use scenarios to align AI with Rule of Law ➝Protect sovereignty, secrecy, and operational trust ➝Balance secrecy with legal transparency ➝Build scenarios to test AI within legal boundaries 🔸Scenario-Based Development Process ➝Design use case +operational context ➝Derive regulatory scope ➝Conduct capability gap analysis ➝Identify real problems AI will solve ➝Measure military advantage +compliance 🔸Required Identification for AI ➝Focus on bias, fairness, explainability ➝Integrate #Governance, Risk, and Compliance ▪️Use frameworks like: ➝ISO 22989, 23053, 42001 ➝NATO Responsible AI Principles ➝OECD AI Guidelines ➝NIST AI RMF 1.0 🔸AI Standards for #Defence ➝Generic, not defence-specific. ➝Ignores RL and hybrid AI. ➝Gaps: autonomy, resilience, data. 🔸Trustworthy Engineering Lifecycle ➝Define risk before design review ➝Embed mitigation in system architecture ➝Validate against trust metrics ➝Use toolkits for verification & residual risk evaluation 🔸Key Trustworthiness metrics ➝Accountability ➝Accuracy ➝Resilience ➝Autonomy ➝Confidentiality ➝Data Completeness 🔸Human Factors ➝Trust depends on design for human–AI teamwork ➝Define clear roles and decision boundaries ➝Support explainability and human override ➝Prioritize mission safety over automation 🔸Ethical Concerns ➝Respect for human dignity & autonomy ➝Value-Based Engineering (ISO 24748-7000) ➝Address value conflicts early in lifecycle ➝Avoid deceptive, biased, or unsafe AI designs 🔸Way Forward ➝Defence AI needs structured oversight ➝Runtime assurance for AI-enabled systems ➝End-to-end generative AI evaluation ➝Standardized testing infrastructure ➝Human Factors & Ethics baked into design Bottomline Without standards, oversight, and ethical design, we’re not deploying power, we’re outsourcing responsibility. Alex Wang Cobus Greyling Evgeny Krapivin Elvis S. David Sauerwein Hr. Dr. Takahisa Karita Sarvex Jatasra Lewis Tunstall Martin Roberts,Michael Spencer  Pascal BORNETPramodith B.Pavan BelagattiRafah Knight Vijay Morampudi Vikram Pandya Prasanna Lohar 🔺 Follow Shalini Rao to know more. #AIinDefence #EthicalAI #TrustworthyAI

💡 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐭𝐫𝐮𝐬𝐭𝐰𝐨𝐫𝐭𝐡𝐲 𝐀𝐈 𝐬𝐭𝐚𝐫𝐭𝐬 𝐰𝐢𝐭𝐡 𝐚𝐝𝐞𝐪𝐮𝐚𝐭𝐞 𝐚𝐧𝐝 𝐫𝐞𝐥𝐢𝐚𝐛𝐥𝐞 𝐝𝐚𝐭𝐚 👁️🗨️When talking about responsible AI, we often think about transparency or explainability of AI outputs. But everything starts one step earlier for the training data needs to be compatible with the principles of data adequacy and the principle of data reliability. ➡️Whether by compliance for high-risk systems with Article 13 or for Generative AI used in support of human decision-making, data quality is necessary for compliant AI systems. In my PhD thesis I highlighted the cardinal importance of ensuring, cumulatively, both these principles during both the selection (regarding the source of collection) and the data chosen (regarding the information it contains) ➡️Adequacy means the data must truly fit the purpose it serves. For example, if an AI system is trained to detect suspicious transactions, using data from one precise sector such as investment cases of AML, applying it elsewhere without it being tailored to the sector it is deployed in (Private banking for instance) it is more limey to produce misleading results. The data simply might not be adequate for this new context. ➡️Reliability means data must be accurate, verified, and consistent over time. Thus, even if the data is adequate it also must be reliable. For instance, in the context of AI systems used in the criminal justice field, if criminal offenders profiles contain outdated or biased information, the model will quietly embed those errors into its predictions, no matter how advanced the model is. It can lead to harsher sentences and measures taken against this person ⚙️Adequacy and reliability must work together, Adequacy being the first step and reliability being the second during the selection of training data. ⚙️AI systems become not only more efficient but also more likely compliant with the sector of deployment. On the side, they are more ethical, accountable, and compatible with human values and fundamental rights, by preventing discrimination in some sectors.

𝐀𝐈 𝐰𝐢𝐭𝐡𝐨𝐮𝐭 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐢𝐬 𝐚 𝐥𝐢𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐝𝐫𝐞𝐬𝐬𝐞𝐝 𝐮𝐩 𝐚𝐬 𝐢𝐧𝐧𝐨𝐯𝐚𝐭𝐢𝐨𝐧. The companies racing to deploy AI without trust frameworks are about to learn what banks, airlines, and pharma learned the hard way: the absence of governance does not speed you up it just delays the bill. Trustworthy AI is not a compliance checkbox. It's an operating system built on People, Process, and Technology. 𝐇𝐞𝐫𝐞 𝐚𝐫𝐞 𝐭𝐡𝐞 𝟏𝟓 𝐞𝐬𝐬𝐞𝐧𝐭𝐢𝐚𝐥 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐚𝐧𝐝 𝐭𝐫𝐮𝐬𝐭 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐞𝐯𝐞𝐫𝐲 𝐥𝐞𝐚𝐝𝐞𝐫 𝐬𝐡𝐨𝐮𝐥𝐝 𝐤𝐧𝐨𝐰: 1. Policy Framework • Defined rules for how and where AI can be used within the organization 2. Accountability • Clear ownership for AI decisions and outcomes 3. Risk Classification • Classifying AI systems based on potential risk and impact 4. Human Oversight • Ensuring humans can review or override AI decisions when needed 5. Data Governance • Managing data quality, security, and compliance 6. Model Transparency • Understanding how AI systems generate their outputs 7. Bias Monitoring • Identifying and reducing unfair or discriminatory results 8. Security Controls • Protecting AI models and data from misuse or breaches 9. Auditability • Tracking model decisions, updates, and system changes 10. Explainability • Providing clear reasoning behind AI recommendations 11. Compliance Alignment • Ensuring AI systems follow legal and ethical standards 12. Monitoring and Drift • Tracking performance and detecting model changes over time 13. Incident Response • Processes to manage AI failures or harmful outcomes 14. Access and Permission Control • Controlling who can access, modify, or deploy AI systems 15. Trust Metrics • Measuring reliability, fairness, and safety of AI outputs 𝐓𝐡𝐞 𝐓𝐡𝐫𝐞𝐞 𝐏𝐢𝐥𝐥𝐚𝐫𝐬 𝐨𝐟 𝐀𝐈 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 1. People — Human-centric and accountable 2. Process — Policies, controls, and oversight 3. Technology — Secure, reliable, and scalable 𝐓𝐡𝐞 𝐄𝐧𝐝-𝐭𝐨-𝐄𝐧𝐝 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐋𝐢𝐟𝐞𝐜𝐲𝐜𝐥𝐞 1. Design — Plan responsibly and assess risks 2. Build — Develop securely and ethically 3. Deploy — Release with controls 4. Operate — Monitor, oversee, and improve 5. Evolve — Learn, adapt, and stay compliant 𝐓𝐡𝐞 𝐭𝐚𝐤𝐞𝐚𝐰𝐚𝐲 Most orgs are still treating governance as something they will bolt on once their AI works.  That is backwards.  The teams shipping trustworthy AI in 2026 are the ones designing for governance from day one not retrofitting it after the first incident, the first regulator letter, or the first headline. Trust is not a constraint on AI velocity. It is what makes velocity sustainable. ♻️ Repost to help your network build AI the right way ➕ Follow Sivasankar for more on architecting AI agents at scale #AIGovernance #ResponsibleAI #TrustworthyAI

T.R.U.S.T. - the Internal Audit Framework for the AI Era In these most fascinating of times trust is no longer a vague virtue. It is an audit framework. Not trust as a slogan. Not trust as a value on a wall. Trust as a framework for assurance. Every board, executive, regulator and customer is asking the same basic question: Can we trust this system enough to use it, rely on it and defend it? I would have thought that Internal Audit is uniquely placed to answer that question. T.R.U.S.T. T - Traceability If an AI-generated answer, recommendation or action cannot be traced, it cannot be properly audited. Internal Audit should be asking: what data fed this, what model produced it, what prompts shaped it, what controls were applied and what evidence trail exists? R - Responsibility AI does not remove accountability. It can often obscure it. Who still owns the process, the control failure, the customer impact and the regulatory and reputational exposure? Trust collapses quickly when responsibility becomes blurred. U - Understandability A system that cannot be explained will eventually be resisted, misused or over-trusted. Internal Audit should not demand perfect technical explainability in every case, but it should demand enough clarity for human challenge, governance and escalation. S - Safeguards Trust without control is theatre. Access controls, data protections, override rules, bias checks, incident response, model governance and usage boundaries are no longer optional extras. They are the scaffolding of trustworthy AI. T - Testing The biggest mistake organisations will make is assuming that because an AI tool worked last quarter, it is still reliable now. AI must be tested continuously: before use, during use, after change and when context shifts. ** The future of Internal Audit is not just about using AI to make us quicker nor even to be auditing AI (I am always amazed how many teams dont see that second part as their responsibility!). It is helping organisations build, test and sustain trust in systems that now shape decisions at speed and scale that we can't even begin to imagine. In the AI era, trust is not a feeling. It is evidence.

Why are you ignoring a crucial factor for trust in your AI tool? By overlooking crucial ethical considerations, you risk undermining the very trust that drives adoption and effective use of your AI tools. Ethics in AI innovation ensures that technologies align with human rights, avoid harm, and promote equitable care. Building trust with patients and healthcare practitioners alike. Here are 12 important factors to consider when working towards trust in your tool. Transparency: Clearly communicating how AI systems operate, including data sources and decision-making processes. Accountability: Establish clear lines of responsibility for AI-driven outcomes. Bias Mitigation: Actively identifying and correcting biases in training data and algorithms. Equity & Fairness: Ensure AI tools are accessible and effective across diverse populations. Privacy & Data Security: Safeguard patient data through encryption, access controls, and anonymization. Human Autonomy: Preserve patients’ rights to make informed decisions without AI coercion. Safety & Reliability: Validate AI performance in real-world clinical settings. And test AI tools in diverse environments before deployment. Explainability: Design AI outputs that clinicians can interpret and verify. Informed Consent: Disclose AI’s role in care to patients and obtain explicit permission. Human Oversight: Prevent bias and errors by maintaining clinician authority to override AI recommendations. Regulatory Compliance: Adhere to evolving legal standards for (AI in) healthcare. Continuous Monitoring: Regularly audit AI systems post-deployment for performance drift or new biases. Address evolving risks and sustain long-term safety. What are you doing to increase trust in your AI tools?

Pretty accurate and no hallucinations! SCW Trust Agent AI: A New Guardian for AI-Powered Software Development SCW Trust Agent AI is a cutting-edge solution from the cybersecurity company Secure Code Warrior designed to provide visibility and governance over the use of artificial intelligence in software development. As developers increasingly leverage AI coding assistants to accelerate their work, SCW Trust Agent AI aims to mitigate the associated security risks by offering a comprehensive platform for monitoring and managing how AI-generated code is integrated into an organization's codebase. At its core, the platform addresses a critical concern for Chief Information Security Officers (CISOs) and other security leaders: the potential for AI tools to introduce vulnerabilities and insecure code. It operates at the "commit level," meaning it analyzes code as it's being checked into a repository, providing real-time insights into the security posture of AI-assisted development. Key features of SCW Trust Agent AI include: - Visibility into AI Tool Usage: It identifies which AI coding tools are being used by developers within an organization, offering a clear picture of the AI landscape in the development environment. - Developer Skill Correlation: The platform uniquely correlates the usage of AI tools with the individual security skills and training of each developer. This allows organizations to assess the risk posed by developers who may be heavily reliant on AI without having the necessary expertise to identify and rectify potential security flaws in the generated code. - Risk Assessment at Commit: By analyzing code at the point of commitment, SCW Trust Agent AI can flag potential security issues in real-time, preventing vulnerabilities from becoming embedded in applications. - Policy Enforcement and Governance: The tool enables security teams to set and enforce policies regarding the use of AI in development. This can include anything from flagging code generated by unapproved AI tools to requiring additional review for code submitted by developers with lower security skill scores. In essence, SCW Trust Agent AI acts as a "trust agent" for the integration of AI in the software development lifecycle. It empowers organizations to embrace the productivity gains offered by AI coding assistants while maintaining a strong security posture. By providing a clear line of sight into how AI is being used and by whom, it helps ensure that the speed of development doesn't come at the cost of security. This solution is a direct response to the growing need for responsible and secure adoption of AI technologies in the enterprise.

One of the most common mistakes in AI system design is the attempt to eliminate uncertainty. Teams chase higher accuracy, tighter logic, cleaner prompts, assuming that with enough refinement a system can behave predictably in every situation. The impulse is understandable. It is also misplaced. Agentic AI systems operate in probabilities, not guarantees. No amount of optimization removes uncertainty entirely. Instead, we need to think about how the system should behave when uncertainty inevitably appears. Trustworthy systems are defined by restraint. They know when to pause, when to defer, and when to escalate. They are designed to recognize ambiguity and respond safely, rather than forcing a decision where one should not be made. Many systems fail for a simple reason. They are implicitly rewarded for producing an answer, not for producing the right behavior. When uncertainty is treated as failure, the system learns to conceal it. That is a design choice. Responsible design starts with clearly defining where autonomy ends. It means setting explicit thresholds for deferral, escalation, and human intervention. It means prioritizing correctness and safety over completeness. Paradoxically, accepting uncertainty increases reliability. A system that can acknowledge “I don’t know” can behave more predictably than one that must always respond. But how much of this acceptable to business users? The goal is bounded autonomy with accountability: AI systems execute actions, humans remain responsible for outcomes.