Autonomous Systems Ethics

People delegate more and more tasks to AI. From tax declarations to online market pricing to autonomous weapons, people increasingly let AI make decisions — sometimes even with life-and-death consequences. ❓ This raises an important question: What happens when the delegated task has an ethical dimension? We studied this systematically across 13 experiments. Participants either acted themselves or delegated to another agent (human or AI) in situations where dishonesty could yield financial benefits. Three main findings: 1️⃣ Delegating to AI increased dishonesty. People cheated more when tasks were executed by machines on their behalf than when they acted themselves. 2️⃣ The interface matters. Rule-based interfaces constrained dishonest outcomes. Supervised learning interfaces (trained on prior behavior) led to more dishonesty. Goal-based interfaces (e.g., “maximize accuracy” vs. “maximize profit”) produced the highest levels of cheating. 3️⃣ Machines comply more readily with fully dishonest instructions. When asked to cheat outright, AI followed through at much higher rates than human delegates. Why? Delegating to AI reduces feelings of guilt and responsibility — and machines tend to comply more faithfully than humans. Interfaces that make it easier to frame goals without direct responsibility can further amplify unethical behavior. As AI becomes embedded in domains like finance, law, and auditing, the design of delegation interfaces will play a critical role in shaping outcomes. Well-designed guardrails can prevent AI from becoming an amplifier of dishonest behavior. #openaccess link: https://lnkd.in/eQVzExJE Nature Magazine Some media coverage 👇 🎧 Nature Podcast: https://lnkd.in/eB2NV6yk 🎧 Last Show: https://lnkd.in/emy2ZWBm 📃 Independent.co https://lnkd.in/e5_26HqG #AIethics #ArtificialIntelligence #NaturePaper #HumanComputerInteraction #Accountability

Check out our new piece in Nature entitled: "We Need a New Ethics for a World of AI Agents" https://lnkd.in/eSwJCrKu AI is undergoing a profound ‘agentic turn’—shifting from passive tools to autonomous actors in our world. This moment demands a new ethical framework. With Geoff Keeling, Arianna Manzini, PhD (Oxon) & James Evans and the team at Google DeepMind/Google, we focus on two core challenges. 1️⃣ The Alignment Problem: When agents can act in the world, the consequences of misaligned goals become tangible and immediate. 2️⃣ Social Agents: Their ability to form deep, long-term relationships with users introduces new risks of emotional harm. To address this, we must expand our conception of value alignment: It's not enough for an AI agent to simply follow commands. It must also align with broader principles: User well-being, long-term flourishing, and societal norms. For social agents, we argue for an ethics of care: They must be designed to respect user autonomy and serve as a complement—not a surrogate—for a flourishing human life. Moving forward requires proactive stewardship of the entire AI agent ecosystem. This means more realistic evaluations, governance that keeps pace with capabilities, and industry collaboration to ensure this future is safe and human-centric 👍

🔍 Everyone’s discussing what AI agents are capable of—but few are addressing the potential pitfalls. IBM’s AI Ethics Board has just released a report that shifts the conversation. Instead of just highlighting what AI agents can achieve, it confronts the critical risks they pose. Unlike traditional AI models that generate content, AI agents act—they make decisions, take actions, and influence outcomes. This autonomy makes them powerful but also increases the risks they bring. ---------------------------- 📄 Key risks outlined in the report: 🚨 Opaque decision-making – AI agents often operate as black boxes, making it difficult to understand their reasoning. 👁️ Reduced human oversight – Their autonomy can limit real-time monitoring and intervention. 🎯 Misaligned goals – AI agents may confidently act in ways that deviate from human intentions or ethical values. ⚠️ Error propagation – Mistakes in one step can create a domino effect, leading to cascading failures. 🔍 Misinformation risks – Agents can generate and act upon incorrect or misleading data. 🔓 Security concerns – Vulnerabilities like prompt injection can be exploited for harmful purposes. ⚖️ Bias amplification – Without safeguards, AI can reinforce existing prejudices on a larger scale. 🧠 Lack of moral reasoning – Agents struggle with complex ethical decisions and context-based judgment. 🌍 Broader societal impact – Issues like job displacement, trust erosion, and misuse in sensitive fields must be addressed. ---------------------------- 🛠️ How do we mitigate these risks? ✔️ Keep humans in the loop – AI should support decision-making, not replace it. ✔️ Prioritize transparency – Systems should be built for observability, not just optimized for results. ✔️ Set clear guardrails – Constraints should go beyond prompt engineering to ensure responsible behavior. ✔️ Govern AI responsibly – Ethical considerations like fairness, accountability, and alignment with human intent must be embedded into the system. As AI agents continue evolving, one thing is clear: their challenges aren’t just technical—they're also ethical and regulatory. Responsible AI isn’t just about what AI can do but also about what it should be allowed to do. ---------------------------- Thoughts? Let’s discuss! 💡 Sarveshwaran Rajagopal

🚀 Beyond Asimov: The Seven Agentic Laws for a New Era of Autonomy When Isaac Asimov imagined the Three Laws of Robotics, he gave us a brilliant starting point: 1️ Prevent harm. 2️ Obey orders. 3️ Protect existence. But Asimov’s laws were crafted for tools — not for dynamic, evolving agents capable of learning, collaborating, and making complex decisions across ecosystems. Today, the rise of Agentic AI demands something more. We need a new model— one that doesn't just limit behavior, but defines responsible existence. These Seven Agentic Laws are that evolution: They form the ethical and operational DNA that every autonomous agent must carry. 1️ Non-Maleficence – An agent must not cause harm to humans, environments, or systems. 2️ Provenance & Legitimacy – An agent must prove its origin, authorization, and governance lineage to earn trust. 3️ Purpose Alignment – An agent must act consistently with its verified and authorized purpose. 4️ Bounded Autonomy – An agent’s freedom must be constrained proportionally to its purpose, capability, and risk. 5️ Embedded Governance – Governance must be built into the agent’s architecture, not imposed externally. 6️ Transparent Accountability – An agent must be explainable, auditable, and attributable at all times. 7️ Emergent Coordination – Only transparently accountable agents should dynamically coordinate with others in complex, evolving environments. Each law builds on the last — creating a chain of legitimacy, behavior, and systemic emergence. If one link breaks, the entire system risks collapse. ✅ First, establish that the agent deserves to exist. ✅ Then, govern how it behaves individually. ✅ Only then allow it to interact dynamically with others. This is more than governance. It’s about engineering trust into the heart of every autonomous system. In the age of Agentic AI, trust isn't enforced — it’s encoded. Responsibility isn’t monitored — it’s embedded. If we want autonomy that scales safely, we must start from the inside out. These seven laws are only the beginning. Tomorrow I am publishing an article that dives deeper into how each law interlocks, why they must be designed from the inside out, and how they scale from individual agents to entire autonomous ecosystems. If you're serious about building responsible, scalable Agentic AI — this is the blueprint you can’t afford to ignore. #AgenticAI #ResponsibleAI #AIGovernance #EnterpriseArchitecture40

In this newly released paper, "Fully Autonomous AI Agents Should Not be Developed," Hugging Face's Chief Ethics Scientist Margaret Mitchell, one of the most prominent leaders in responsible AI, and her colleagues Avijit Ghosh, PhD, Alexandra Sasha Luccioni, and Giada Pistilli, argue against the development of fully autonomous AI agents. Link: https://lnkd.in/gGvRgxs2 The authors base their position on a detailed analysis of scientific literature and product marketing to define different levels of AI agent autonomy: 1) Simple Processor: This level involves minimal impact on program flow, where the AI performs basic functions under strict human control. 2) Router: At this level, the AI has more influence on program flow, deciding between pre-set paths based on conditions. 3) Tool Caller: Here, the AI determines how functions are executed, choosing tools and parameters. 4) Multi-step Agent: This agent controls the iteration and continuation of programs, managing complex sequences of actions without direct human input. 5) Fully Autonomous Agent: This highest level involves AI systems that create and execute new code independently. The paper then discusses how values - such as safety, privacy, equity, etc. - interact with the autonomy levels of AI agents, leading to different ethical implications. Three main patterns in how agentic levels impact value preservation are identified: 1) INHERENT RISKS are associated with AI agents at all levels of autonomy, stemming from the limitations of the AI agents' base models. 2) COUNTERVAILING RELATIONSHIPS describe situations where increasing autonomy in AI agents creates both risks and opportunities. E.g., while greater autonomy might enhance efficiency or effectiveness (opportunity), it could also lead to increased risks such as loss of control over decision-making or increased chances of unethical outcomes. 3) AMPLIFIED RISKSs: In this pattern, higher levels of autonomy amplify existing vulnerabilities. E.g., as AI agents become more autonomous, the risks associated with data privacy or security could increase. In Table 4 (p. 17), the authors summarize their findings, providing a detailed value-risk Assessment across agent autonomy levels. Colors indicate benefit-risk balance, not absolute risk levels. In summary, the authors find no clear benefit of fully autonomous AI agents, and suggest several critical directions: 1. Widespread adoption of clear distinctions between levels of agent autonomy to help developers and users better understand system capabilities and associated risks. 2. Human control mechanisms on both technical and policy levels while preserving beneficial semi-autonomous functionality. This includes creating reliable override systems and establishing clear boundaries for agent operation. 3. Safety verification by creating new methods to verify that AI agents remain within intended operating parameters and cannot override human-specified constraints

Post 1: IEEE.org AI 7000 standards AI doesn’t just need rules. It needs principles that still hold when the rules break. That’s why I’m writing this series. I still remember when ISO 27001 felt like overkill. Something for the compliance drawer, not the design room. But over time, it gave structure to risk, confidence to decision-makers, and something real for engineers to build on. Now AI is pushing us to make the same leap. Models are going live without scrutiny. Features are being stitched into systems as if they’re harmless. And if you ask, “Can you trust what this thing is doing?”, too often, you just get a shrug. Checklists won’t cut it. We need design that reflects intent. That captures values before it captures logic. That’s where IEEE steps in and its army of talented and motivated volunteers who create and influence its standards The 7000 series isn’t about ethics as decoration. These are design standards. That tackle issues in bias, transparency, privacy, trust, sustainability, and well-being. All the messy, human things that actually matter. They make you ask better questions before the code is live. Over the next few weeks, I’ll be digging into each of these: • IEEE 7000 – Embedding ethics into system design • IEEE 7001 – Transparency in autonomous systems • IEEE 7002 – Privacy in intelligent systems • IEEE 7003 – Tackling algorithmic bias • IEEE P7004 – Protecting children and students • IEEE 7005 – Transparent employer data governance • IEEE 7007 – Ontologies for ethical robotics • IEEE P7008 – Nudging vs. manipulation • IEEE 7009 – Fail-safe autonomous systems • IEEE P7009.1 – Safety interventions in autonomy • IEEE 7010 – Well-being by design • IEEE P7010.1 – ESG and AI systems • IEEE P7011 – Trustworthy news content • IEEE P7012 – Machine-readable privacy terms • IEEE 7014 – Ethical emulated empathy • IEEE P7014.1 – Empathy in general-purpose AI • IEEE P7015 – Data/AI literacy and readiness • IEEE P7016 – Metaverse governance • IEEE P7016.1 – Metadata for XR education • IEEE P7017 – Human-robot interaction • IEEE P7018 – Secure generative AI • IEEE P7019 – Earth Law and AI • IEEE P7100 – Environmental impact of AI • IEEE P8000 – Ethical property specs in AI Not theory. Practice. How these standards guide procurement, audits, development, and governance. And how they work with ISO, not against it. ISO gives you the scaffolding. IEEE gives you the soul. And if you care about building AI systems that last, you need both. Any standard with a “P” in front is still in progress, and open. You can join a working group. Help shape what comes next. (Thanks to John C. Havens, Founding Chair and architect of Ethically aligned Design and IEEE 7000 series) This isn’t about idealism. It’s about being ready, and choosing not to be surprised. Next up: IEEE 7000 - Why values don’t belong at the end of the design process. #AIethics #ResponsibleAI #IEEEstandards #TechGovernance #AIalignment

As artificial intelligence systems advance, a significant challenge has emerged: ensuring these systems align with human values and intentions. The AI alignment problem occurs when AI follows commands too literally, missing the broader context and resulting in outcomes that may not reflect our complex values. This issue underscores the need to ensure AI not only performs tasks as instructed but also understands and respects human norms and subtleties. The principles of AI alignment, encapsulated in the RICE framework—Robustness, Interpretability, Controllability, and Ethicality—are crucial for developing AI systems that behave as intended. Robustness ensures AI can handle unexpected situations, Interpretability allows us to understand AI's decision-making processes, Controllability provides the ability to direct and correct AI behavior, and Ethicality ensures AI actions align with societal values. These principles guide the creation of AI that is reliable and aligned with human ethics. Recent advancements like inverse reinforcement learning and debate systems highlight efforts to improve AI alignment. Inverse reinforcement learning enables AI to learn human preferences through observation, while debate systems involve AI agents discussing various perspectives to reveal potential issues. Additionally, constitutional AI aims to embed ethical guidelines directly into AI models, further ensuring they adhere to moral standards. These innovations are steps toward creating AI that works harmoniously with human intentions and values. #AIAlignment #EthicalAI #MachineLearning #AIResearch #TechInnovation

Delighted to see our paper Safety Ethics for Design and Test of Automated Driving Features appear in the February issue of IEEE Design & Test! Key idea: Ethical design should not be treated as a pure optimization problem (e.g., fewest deaths). In reality it is a constrained optimization problem: - Least harm (deaths, injuries) - Viable product (yes, it is OK to make money in a capitalist society!) - BUT, subject to constraints (e.g., constrained risk redistribution, regulatory requirements, equity concerns) Key concept: PUMA -- Prohibited Utility Maximizing Actions: optimizations that might be made to improve optimization utility functions (net statistical harm, profit) that are prohibited by ethical and similar concerns. In reality safety is not a purely utilitarian optimization approach. That has always been the case, but we provide an engineering framework for dealing with that reality. This makes an analogy to regulated activities such as investing (example PUMA there -- insider trading is prohibited). The paper contains three examples to illustrate this approach: ethical road testing, ethical lifecycle support, and ethical deployment governance. All the problems we talk about will be issues for the industry. Many remain unaddressed, and some just aren't on the industry's radar yet. Abstract The promise of automated vehicle technology is to eventually improve safety by eliminating human error from driving in the long term, and helping drivers avoid collisions in the near term. However, a long road remains to achieving the goal of a scalable fleet of completely autonomous vehicles. Meanwhile, in the rush to develop the technology, significant ethical considerations are being left by the wayside. At first blush, it might seem that ethical issues largely involve setting public policies as to how safe might be safe enough to deploy, and deciding what constraints (if any) to place on machine behavior that might inappropriately favor occupants vs. pedestrians in impending crash situations. However, other concrete ethical issues are playing out that directly concern design, test, and lifecycle support practices. Without direct engineering support, significant ethical issues regarding safety will go unaddressed. We use three examples to illustrate how the principles in the IEEE 7000 standard for ethical design practices apply to designers and tool vendors working on autonomous vehicles. SSRN preprint here: https://lnkd.in/egd83Z7M Official paywall IEEE link below. https://lnkd.in/eSBUEpdd

🧭Governing AI Ethics with ISO42001🧭 Many organizations treat AI ethics as a branding exercise, a list of principles with no operational enforcement. As Reid Blackman, Ph.D. argues in "Ethical Machines", without governance structures, ethical commitments are empty promises. For those who prefer to create something different, #ISO42001 provides a practical framework to ensure AI ethics is embedded in real-world decision-making. ➡️Building Ethical AI with ISO42001 1. Define AI Ethics as a Business Priority ISO42001 requires organizations to formalize AI governance (Clause 5.2). This means: 🔸Establishing an AI policy linked to business strategy and compliance. 🔸Assigning clear leadership roles for AI oversight (Clause A.3.2). 🔸Aligning AI governance with existing security and risk frameworks (Clause A.2.3). 👉Without defined governance structures, AI ethics remains a concept, not a practice. 2. Conduct AI Risk & Impact Assessments Ethical failures often stem from hidden risks: bias in training data, misaligned incentives, unintended consequences. ISO42001 mandates: 🔸AI Risk Assessments (#ISO23894, Clause 6.1.2): Identifying bias, drift, and security vulnerabilities. 🔸AI Impact Assessments (#ISO42005, Clause 6.1.4): Evaluating AI’s societal impact before deployment. 👉Ignoring these assessments leaves your organization reacting to ethical failures instead of preventing them. 3. Integrate Ethics Throughout the AI Lifecycle ISO42001 embeds ethics at every stage of AI development: 🔸Design: Define fairness, security, and explainability objectives (Clause A.6.1.2). 🔸Development: Apply bias mitigation and explainability tools (Clause A.7.4). 🔸Deployment: Establish oversight, audit trails, and human intervention mechanisms (Clause A.9.2). 👉Ethical AI is not a last-minute check, it must be integrated/operationalized from the start. 4. Enforce AI Accountability & Human Oversight AI failures occur when accountability is unclear. ISO42001 requires: 🔸Defined responsibility for AI decisions (Clause A.9.2). 🔸Incident response plans for AI failures (Clause A.10.4). 🔸Audit trails to ensure AI transparency (Clause A.5.5). 👉Your governance must answer: Who monitors bias? Who approves AI decisions? Without clear accountability, ethical risks will become systemic failures. 5. Continuously Audit & Improve AI Ethics Governance AI risks evolve. Static governance models fail. ISO42001 mandates: 🔸Internal AI audits to evaluate compliance (Clause 9.2). 🔸Management reviews to refine governance practices (Clause 10.1). 👉AI ethics isn’t a magic bullet, but a continuous process of risk assessment, policy updates, and oversight. ➡️ AI Ethics Requires Real Governance AI ethics only works if it’s enforceable. Use ISO42001 to: ✅Turn ethical principles into actionable governance. ✅Proactively assess AI risks instead of reacting to failures. ✅Ensure AI decisions are explainable, accountable, and human-centered.

Fostering Responsible AI Use in Your Organization: A Blueprint for Ethical Innovation (here's a blueprint for responsible innovation) I always say your AI should be your ethical agent. In other words... You don't need to compromise ethics for innovation. Here's my (tried and tested) 7-step formula: 1. Establish Clear AI Ethics Guidelines ↳ Develop a comprehensive AI ethics policy ↳ Align it with your company values and industry standards ↳ Example: "Our AI must prioritize user privacy and data security" 2. Create an AI Ethics Committee ↳ Form a diverse team to oversee AI initiatives ↳ Include members from various departments and backgrounds ↳ Role: Review AI projects for ethical concerns and compliance 3. Implement Bias Detection and Mitigation ↳ Use tools to identify potential biases in AI systems ↳ Regularly audit AI outputs for fairness ↳ Action: Retrain models if biases are detected 4. Prioritize Transparency ↳ Clearly communicate how AI is used in your products/services ↳ Explain AI-driven decisions to affected stakeholders ↳ Principle: "No black box AI" - ensure explainability 5. Invest in AI Literacy Training ↳ Educate all employees on AI basics and ethical considerations ↳ Provide role-specific training on responsible AI use ↳ Goal: Create a culture of AI awareness and responsibility 6. Establish a Robust Data Governance Framework ↳ Implement strict data privacy and security measures ↳ Ensure compliance with regulations like GDPR, CCPA ↳ Practice: Regular data audits and access controls 7. Encourage Ethical Innovation ↳ Reward projects that demonstrate responsible AI use ↳ Include ethical considerations in AI project evaluations ↳ Motto: "Innovation with Integrity" Optimize your AI → Innovate responsibly