Debates on Artificial General Intelligence as a Milestone

New on AI Snake Oil: Arvind Narayanan and I argue that AGI will not lead to rapid economic effects, the race to AGI is not relevant for great power competition, we won't know AGI when we have built it, and AGI does not imply impending superintelligence. In other words, AGI is not a milestone: https://lnkd.in/exDQbafU 1) Even if general-purpose AI systems reach some agreed-upon capability threshold, we will need many complementary innovations that allow AI to diffuse across industries to realize its productive impact. Diffusion occurs at human (and societal) timescales, not at the speed of tech development. 2) Worries about AGI and catastrophic risk often conflate capabilities with power. Once we distinguish between the two, we can reject the idea of a critical point in AI development at which it becomes infeasible for humanity to remain in control. 3) The proliferation of AGI definitions is a symptom, not the disease. AGI is significant because of its presumed impacts but must be defined based on properties of the AI system itself. But the link between system properties and impacts is tenuous, and greatly depends on how we design the environment in which AI systems operate. Thus, whether or not a given AI system will go on to have transformative impacts is yet to be determined at the moment the system is released. So a determination that an AI system constitutes AGI can only meaningfully be made retrospectively. 4) Businesses and policy makers should take a long-term view. Businesses should not rush to adopt half-baked AI products. Rapid progress in AI methods and capabilities does not automatically translate to better products. Building products on top of inherently stochastic models is challenging, and businesses should adopt AI products cautiously, conducting careful experiments to determine the impact of using AI to automate key business processes. A “Manhattan Project for AGI” is misguided on many levels. Since AGI is not a milestone, there is no way to know when the goal has been reached or how much more needs to be invested. And accelerating AI capabilities does nothing to address the real bottlenecks to realizing its economic benefits. We plan to keep writing on this topic, and have a series of essay planned on the theme of AI as Normal Technology. Follow the AI Snake Oil substack for more.

I’m thrilled and honored to be the lead author of BCG’s first-ever publication on Artificial General Intelligence (AGI) and the topic of human-level intelligence, written in collaboration with my colleagues Dr. Lars Littig and Sophie Geraerdts. Is human-level AI around the corner? Is it going to be doomsday for us all? Or are we even ever going to get there? Perhaps there's a brighter version of the story? Ever since Generative AI—which talks and creates like humans—spiked a couple of years ago, the debate about AGI has only intensified. Some of the most influential voices in the industry have fueled the idea that human-level intelligence is right around the corner. A few months ago, I had the privilege of participating in a roundtable with some of the brightest minds in AI—both from academia and industry. The central question? What exactly is AGI? We quickly found common ground: while some traits seem logical indicators of human-level intelligence, from a purely academic perspective, it’s still too early to tell. There are fundamental scientific and research challenges that need solving before we can definitively say what AGI will look like or when it will arrive. Now, I’m an optimist. Yes, as AI evolves toward AGI, we’ll face new risks and ethical dilemmas that demand attention. But let’s not allow the negative narratives to overshadow the incredible potential AI holds to elevate humanity. There’s too much good that can come from this journey—too much opportunity to solve real problems and create meaningful change. I invite you to read the full white paper, where we break down these complexities and explore the possibilities. What do you think about AGI and the future of AI? I’d love to hear your thoughts. White paper link: https://lnkd.in/da9gPCYy #AI #ArtificialGeneralIntelligence #AGI Boston Consulting Group (BCG) [Disclaimer: the views expressed in this post do not necessarily reflect the views of BCG]

The Argument for an AGI Horizons Team In early 2023, shortly after the release of ChatGPT, a major software company reached out to me to try and understand what was happening with Gen AI and how it might impact their product roadmap and business strategy. We talked through how ChatGPT worked, compared notes on what it could mean to their product development plans, and I shared insights into what else the AI labs were working on that might affect their company and its customers in the coming years. One of my key recommendations for them was to form what I termed an “AGI Horizons Team” tasked with monitoring advancements toward artificial general intelligence (AGI) and assessing potential threats and opportunities. While there continues to be a lack of agreement on how to define AGI, I consider it an AI system that is generally capable of outperforming the average human at most cognitive tasks (e.g. ChatGPT being able to perform 90% or more of marketing tasks better than an average marketer). For more than 70 years, researchers have pursued this idea of human-like general intelligence. They were driven by a belief that we could give machines the ability to think, reason, understand, create, and take actions in the digital and physical worlds. But, progress was often slow, and the impact on our professional lives was minimal. Then, everything changed—and accelerated—with the release of ChatGPT in November 2022, and the rise of Gen AI. By early 2023, the tone and positioning on AGI from the leading AI labs had changed. They no longer talked about AGI as something that might be possible in a decade or more. They were conveying increasing confidence that there was a clear path to achieving AGI within 3 - 5 years. My point to the software company was that while everyone was racing to understand the impact of the current forms of AI on their business, far smarter and more generally capable models were on the horizon that might force them to reimagine and reinvent their products and business model. That if AGI was unlocked by OpenAI, Google Deepmind or another AI lab, it would change everything. And while the probability of that occurring in the 3 - 5 year window was relatively unknown, it certainly wasn’t zero. In other words, there was a potential transformative event (maybe even an extinction-level event for their company) possible within half of a decade. I felt that it was worth putting a team of their best people together to assess, along with outside advisors who can be more objective about the path forward. I now believe there is a greater than 50% chance of an AI lab claiming they have achieved AGI with 1 - 2 years. What that means to your business and industry is unknown. What that means to society and the economy is also unclear. But, I think it’s significant enough that we should all be doing more to consider the possibilities. Maybe it’s time for an AGI Horizons Team in your organization.

We’ve all seen the flashy headlines—“AI Will Replace Humans!” or “AGI Is Here!”—but here’s the reality check: we still don’t even agree on what intelligence is, let alone how to replicate it in machines. Historically, from Francis Galton in the 1800s trying to measure human IQ, to modern AI that can beat us in chess and Go, our definition of “intelligence” has never stopped evolving. We realized animals like dolphins and octopuses show remarkable smarts, so it’s clear we don’t have a one-size-fits-all definition. And if we can’t pin down what intelligence really means, claiming we’ve created a “general” version is, at best, wildly premature—and at worst, just a brilliant marketing scheme. Yes, AI breakthroughs are incredible. But take a closer look at the benchmarks we use to measure progress: Turing Test: beaten by chatbots that use tricks, not genuine understanding. ARC-AGI, MLE-Bench, and other cutting-edge tests: they highlight how far we are from machines truly “thinking” in a human-like way. So why are we bombarded with AGI headlines? Hype sells—simple as that. It grabs attention, fuels investments, and generates buzz. What do you think—does calling AI “general intelligence” trivialize the complexities intelligence?

𝗪𝗵𝗮𝘁 𝗲𝘅𝗮𝗰𝘁𝗹𝘆 𝗶𝘀 𝗔𝗚𝗜, 𝗮𝗻𝗱 𝗵𝗼𝘄 𝗰𝗹𝗼𝘀𝗲 𝗮𝗿𝗲 𝘄𝗲, 𝗿𝗲𝗮𝗹𝗹𝘆? For years, Artificial General Intelligence (AGI) has been an ever-moving target. Each time AI breaks a boundary, from winning Go to passing bar exams, we quietly shift the goalpost. A new paper, 'A Definition of AGI' (Hendrycks et al., 2025), proposes a long-awaited quantifiable framework to finally pin it down. 💡 Core idea:- 𝗔𝗚𝗜 = 𝗔𝗻 𝗔𝗜 𝘁𝗵𝗮𝘁 𝗺𝗮𝘁𝗰𝗵𝗲𝘀 𝘁𝗵𝗲 𝗰𝗼𝗴𝗻𝗶𝘁𝗶𝘃𝗲 𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗹𝗶𝘁𝘆 𝗮𝗻𝗱 𝗽𝗿𝗼𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 𝗼𝗳 𝗮 𝘄𝗲𝗹𝗹-𝗲𝗱𝘂𝗰𝗮𝘁𝗲𝗱 𝗮𝗱𝘂𝗹𝘁. To operationalize this, the authors adapt the Cattell-Horn-Carroll (CHC) theory of human cognition, the same model used in psychometrics, and break intelligence into 10 measurable domains:- 🧠 𝗚𝗲𝗻𝗲𝗿𝗮𝗹 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 📖 𝗥𝗲𝗮𝗱𝗶𝗻𝗴 & 𝗪𝗿𝗶𝘁𝗶𝗻𝗴 ➗ 𝗠𝗮𝘁𝗵𝗲𝗺𝗮𝘁𝗶𝗰𝗮𝗹 𝗔𝗯𝗶𝗹𝗶𝘁𝘆 🔍 𝗢𝗻-𝘁𝗵𝗲-𝗦𝗽𝗼𝘁 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 🧩 𝗪𝗼𝗿𝗸𝗶𝗻𝗴 𝗠𝗲𝗺𝗼𝗿𝘆 💾 𝗟𝗼𝗻𝗴-𝗧𝗲𝗿𝗺 𝗠𝗲𝗺𝗼𝗿𝘆 𝗦𝘁𝗼𝗿𝗮𝗴𝗲 🧠 𝗟𝗼𝗻𝗴-𝗧𝗲𝗿𝗺 𝗠𝗲𝗺𝗼𝗿𝘆 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 👁️ 𝗩𝗶𝘀𝘂𝗮𝗹 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 👂 𝗔𝘂𝗱𝗶𝘁𝗼𝗿𝘆 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 ⚡ 𝗦𝗽𝗲𝗲𝗱 Each domain contributes 10 % to an overall “AGI Score.” 📊 Results: GPT-4 → 27 % GPT-5 → 57 % In other words, we’re halfway there, but not evenly. Current AIs show a jagged cognitive profile:- 'Superhuman in math and language, yet near-zero in long-term memory storage and integrated reasoning'. The authors warn of 'capability contortions', where we mistake clever workarounds (like long context windows or retrieval tools) for genuine learning and memory. 🚧 The true bottlenecks:- 1. Continual learning (memory formation). 2. Reliable recall without hallucinations. 3. Visual-spatial reasoning. 4. Cross-modal integration (seeing, hearing, remembering together). This paper doesn’t just measure progress, it redefines the map of what “general” intelligence means. Paper - https://lnkd.in/di9dEfzb Curious, which of these 10 domains, do you think will be cracked next? #AGI #ArtificialIntelligence #AIResearch #CognitiveScience #AIDevelopment #GPT5 #AIProgress #MachineLearning #ResponsibleAI #AIFuture #AIEthics #AIInnovation #AILeaders #DeepLearning

In recent months, I have had the pleasure of contributing to the International Scientific Report on the Safety of Advanced AI, a project of the UK government's Department for Science, Innovation and Technology (DSIT) and AI Safety Institute. This report sets out an up-to-date, science-based understanding of the safety of advanced AI systems. The independent, international, and inclusive report is a landmark moment of international collaboration. It marks the first time the international community has come together to supports efforts to build a shared scientific and evidence-based understanding of frontier AI risks. The intention to create such a report was announced at the AI Safety Summit in November 2023 This interim report is published ahead of the AI Seoul Summit next week. The final report will publish before the AI Action Summit in France. The interim report restricts its focus to a summary of the evidence on general-purpose AI, which have advanced rapidly in recent years. The report synthesizes the evidence base on the capabilities of, and risks from, general-purpose AI and evaluates technical methods for assessing and mitigating them. Key report takeaways include: 1️⃣ General-purpose AI can be used to advance the public interest, leading to enhanced wellbeing, prosperity, and scientific discoveries. 2️⃣ According to many metrics, the capabilities of general-purpose AI are advancing rapidly. Whether there has been significant progress on fundamental challenges such as causal reasoning is debated among researchers. 3️⃣ Experts disagree on the expected pace of future progress of general-purpose AI capabilities, variously supporting the possibility of slow, rapid, or extremely rapid progress. 4️⃣ There is limited understanding of the capabilities and inner workings of general-purpose AI systems. Improving our understanding should be a priority. 5️⃣ Like all powerful technologies, current and future general-purpose AI can be used to cause harm. For example, malicious actors can use AI for large-scale disinformation and influence operations, fraud, and scams. 6️⃣ Malfunctioning general-purpose AI can also cause harm, for instance through biassed decisions with respect to protected characteristics like race, gender, culture, age, and disability. 7️⃣ Future advances in general-purpose AI could pose systemic risks, including labour market disruption, and economic power inequalities. Experts have different views on the risk of humanity losing control over AI in a way that could result in catastrophic outcomes. 8️⃣ Several technical methods (including benchmarking, red-teaming and auditing training data) can help to mitigate risks, though all current methods have limitations, and improvements are required. 9️⃣ The future of AI is uncertain, with a wide range of scenarios appearing possible. The decisions of societies and governments will significantly impact its future. #ResponsibleAI #GenerativeAI #ArtificialIntelligence #AI #AISafety

As we witness unprecedented advancements in artificial intelligence (AI), the concept of AI singularity has moved from a speculative theory to a serious discussion. AI singularity refers to the moment when machines surpass human intelligence, possibly reaching a point where AI improves itself autonomously, potentially triggering an "intelligence explosion." While this remains a theoretical concept, the innovations we see today, from hyper-intelligent chatbots to autonomous robots, make the idea increasingly relevant. The question is: what will happen when AI reaches a level where it surpasses our cognitive abilities? #AI singularity poses both incredible promise and considerable risks. On the one hand, we might witness technological prosperity—AI could automate routine tasks, freeing up human creativity and increasing our leisure time. Imagine a world where superintelligent systems solve complex global issues like poverty or disease, or where brain-computer interfaces enhance our cognitive abilities. The potential to revolutionize fields like medicine, space exploration, and even our understanding of consciousness is immense. Technologies like DeepMind's AlphaFold are already pushing the boundaries of medical research, and the idea of AI helping us understand the cosmos is no longer just a dream. However, the road to singularity is fraught with dangers. The rapid pace of AI development could lead to unforeseen consequences, especially if we lose control of these superintelligent systems. AI could pursue goals misaligned with human values, creating harmful outcomes. For instance, we've seen how AI-driven algorithms in social media have exacerbated the spread of misinformation. Imagine that on a larger scale, with systems influencing global economics or national security. The complexity of these technologies could also introduce new risks—information hazards, where the very knowledge AI uncovers might be too dangerous for us to handle. Despite these challenges, AI's trajectory seems unavoidable. Some experts, like Ray Kurzweil, predict that by 2045, machines could surpass human intelligence. However, many still believe that AI singularity might not be feasible due to the limitations of current technology, like the inability to replicate the complexities of human consciousness. Others argue that ethical and societal challenges might halt progress before reaching this tipping point. In any case, whether we're heading towards a singularity or not, it's clear that AI will continue to reshape our world. As we progress, ensuring that AI development aligns with human values will be crucial to reaping its benefits while managing its risks.

Einstein framed today’s core AI debate in 1953, before the term even existed, and gave us the answer. We just haven’t been listening. Correlation versus causality. Today’s debate between connectionists (NNs) and symbolists (logic, reasoning) in AI echoes this much older and deeper question. This distinction isn’t new. In a 1953 letter, Einstein put it succinctly, “The development of Western science is based on two great achievements: the invention of the formal logical system (in Euclidean geometry) by the Greek philosophers, and the discovery of the possibility to find out causal relationships by systematic experiment (during the Renaissance).” The classical Greek tradition, through Plato and Aristotle, placed explanation, causality, and first principles at the center of understanding. Much of that orientation was later eclipsed during the medieval period, when authority, superstition, analogy, and metaphor often substituted for systematic causal inquiry, until the Renaissance reawakened causal and logical reasoning. Einstein’s point: causality is what allowed us to move from patterns to meaning, from observation to understanding, from blind correlation to causal reasoning. This isn’t just a philosophical debate; it’s a structural one. In 1969, McCarthy, who coined the term "Artificial Intelligence" in 1956, argued that for a machine to truly "understand," it must possess Epistemological Adequacy. It isn’t enough for a machine to produce a correct output; it must have an internal representation of the world that understands why things happen. Without this foundation, as Pearl argues, AI remains stuck on the first rung of the "Ladder of Causation". It excels at Association (seeing patterns), but it cannot master Intervention (predicting the effect of actions) or Counterfactuals (reasoning about what would have happened if conditions were different). In AI today, the same question persists. Some argue that correlation, scaled by data and compute, is sufficient. It isn’t. If this sounds abstract, the data now makes it unavoidable. The Empirical Proof: Two major studies from 2025 prove that simply "scaling up" pattern-matchers is hitting a wall. Apple’s 2025 research (“The Illusion of Thinking”) demonstrates that as reasoning tasks grow more complex, model accuracy collapses, even as models generate longer “thinking traces.” A late 2025 study on scientific discovery finds the same plateau, models perform well on surface benchmarks but fail at iterative reasoning, hypothesis refinement, and genuine discovery. Correlation can predict; it cannot explain. Intelligence requires causal models and logical reasoning, the ability to explain, not just predict. Until AI can represent causes, test counterfactuals, and justify its inferences, it will remain powerful pattern recognition masquerading as cognition. Causality and logical reasoning are the foundations of human intelligence, and they remain the missing foundations of today’s AI.  #AI

I’ve spent a lot of time recently trying to get my arms around the state of play with AI. I have more questions than answers. Research-implementation mismatch: There’s a chasm between cutting-edge AI research results, & practical implementation in organizations that were designed & built in the industrial age. When & how does this chasm shrink? Trillion-dollar AI infrastructure investments: These proposals come with staggering energy demands, yet there’s little discussion of end states, comprehensive cost-benefit analyses, or the opportunity costs of such massive commitments. Export controls on the AI stack: As these grow increasingly complex, tracking & implementing all of them will become extraordinarily difficult. Have we fully considered the unanticipated & unintended consequences, or how they’ll fare against the relentless pace of technological change? Catastrophic AGI fears: Warnings about runaway AI, killer drones, and scheming/deceptive AGI models dominate headlines, yet development races ahead at max speed. Good, or bad?🤷♂️ Socio-economic impacts: Lots of talk, little action. Where are the concrete plans to address job displacement, economic inequality, and the ethical dilemmas posed by highly advanced AI? AGI as a near-term end state: Treating AGI as a destination overlooks the long-term, arduous journey of innovation diffusion, and the infrastructure, education, training, and retooling required for society-wide adoption. AI arms race rhetoric: Referring to AI as an arms race ignores its dual- and multi-use nature—a technology born primarily outside the national security enterprise, with implications far beyond militarization. Shifting narratives: Are we about to pivot from one unhelpful bookend—p(doom), fearing AI��s existential risks—to the other—p(screw regulations), dismissing responsible AI, governance, and ethical frameworks in the name of unconstrained competition? The meaning of “all in” on AI: What does it truly mean for governments, organizations, or nations to go all in on AI? Are we even aligned on what this should look like? Zero-sum game: Viewing AI purely through the lens of untrammeled competition misses the opportunity to define areas for international cooperation—whether through frameworks like “AI for good” or multilateral agreements on governance, safety, and lawful and ethical use. AGI: Revolution or house of cards? Is AGI the secret to global dominance, or are we investing in an Enron-like mirage of overhyped promises & underwhelming results? Put 10 of the world’s AI experts in the same room, you get at least 5 different answers to this question. I’m a techno-optimist. I believe in AI’s transformative potential and its promise to reshape industries, strengthen economies, bolster national security, solve global challenges, and spark a digital revolution. But we need national & global conversations to address these complex, often contradictory, questions in a comprehensive rather than one-off ways.

There’s growing optimism that GenAI agents will soon handle complex, multi-step tasks autonomously. Some predict that in 6–12 months, 90% of coding will be automated. But how realistic is this? At the core of this debate is planning and reasoning which is an agent’s ability to decompose problems, structure workflows, and self-correct over long horizons. This is central to autonomy. Here are the most critical weaknesses of llm-based agents: • Hierarchical task structuring – Most AI agents still approach planning linearly, rather than breaking tasks into hierarchical components. •Long-Term Planning and Reasoning –Current agentic systems built on LLMs struggle with long-horizon planning. • Hallucinations and error accumulation – AI models often generate incorrect or inconsistent information. When planning across multiple steps, these errors compound. • Lack of memory and context persistence – Most LLMs operate within a fixed context window, meaning they gradually lose track of earlier decisions. • Inefficiency in execution – Today’s AI agents generate plans step-by-step, often in a reactive rather than strategic manner. Some of these weaknesses are show stoppers. For example, benchmarks testing LLMs in interactive environments (e.g., Minecraft, MiniWoB) indicate that LLM-only agents succeed in fewer than 10% of complex tasks due to hallucinations and poor execution tracking. So what’s the solution? Let’s look deeper at different planning methods: 1. Task Decomposition – AI agents break down complex problems into smaller, manageable steps and then solve (e.g. chain of thought, Higging GPT) 2. Multi-Plan Selection – Instead of committing to a single plan, agents generate multiple possible plans and select the most optimal one using ranking mechanisms like Monte Carlo Tree Search (MCTS) or heuristics. 3. External Planner-Aided Planning – LLMs often lack structured reasoning, so they leverage symbolic (e.g., PDDL-based planners) or neural-based planners to refine their strategies. 4. Reflection and Refinement – Agents improve their plans through self-feedback loops, adjusting their reasoning iteratively (e.g., Reflexion, CRITIC). 5. Memory-Augmented Planning – AI agents store and retrieve past experiences to enhance long-term planning. Some studies show that using hybrid methods in constrained environments achieves hp to 98% success rate which is significangly higher than llm-based reasoning. Most off-the-shelf solutions for agentic AI focus primarily on task decomposition and memory-augmented reasoning, while largely neglecting other critical planning methods. More importantly, no existing product offers a robust framework for dynamically selecting and integrating multiple planning strategies. For now, the most reliable AI agent implementations still rely on custom-tailored planning methods, designed for constrained environments and well-defined task domains, where control and predictability outweigh generalization.