Understanding AI Systems

AI is rapidly moving from passive text generators to active decision-makers. To understand where things are headed, it’s important to trace the stages of this evolution. 1. 𝗟𝗟𝗠𝘀: 𝗧𝗵𝗲 𝗘𝗿𝗮 𝗼𝗳 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗙𝗹𝘂𝗲𝗻𝗰𝘆 Large Language Models (LLMs) like GPT-3 and GPT-4 excel at generating human-like text by predicting the next word in a sequence. They can produce coherent and contextually appropriate responses—but their capabilities end there. They don’t retain memory, they don’t take actions, and they don’t understand goals. They are reactive, not proactive. 2. 𝗥𝗔𝗚: 𝗧𝗵𝗲 𝗔𝗴𝗲 𝗼𝗳 𝗖𝗼𝗻𝘁𝗲𝘅𝘁-𝗔𝘄𝗮𝗿𝗲 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 Retrieval-Augmented Generation (RAG) brought a major upgrade by integrating LLMs with external knowledge sources like vector databases or document stores. Now the model could retrieve relevant context and generate more accurate and personalized responses based on that information. This stage introduced the idea of 𝗱𝘆𝗻𝗮𝗺𝗶𝗰 𝗸𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗮𝗰𝗰𝗲𝘀𝘀, but still required orchestration. The system didn’t plan or act—it responded with more relevance. 3. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜: 𝗧𝗼𝘄𝗮𝗿𝗱 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 Agentic AI is a fundamentally different paradigm. Here, systems are built to perceive, reason, and act toward goals—often without constant human prompting. An Agentic system includes: • 𝗠𝗲𝗺𝗼𝗿𝘆: to retain and recall information over time. • 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴: to decide what actions to take and in what order. • 𝗧𝗼𝗼𝗹 𝗨𝘀𝗲: to interact with APIs, databases, code, or software systems. • 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆: to loop through perception, decision, and action—iteratively improving performance.    Instead of a single model generating content, we now orchestrate 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗮𝗴𝗲𝗻𝘁𝘀, each responsible for specific tasks, coordinated by a central controller or planner. This is the architecture behind emerging use cases like autonomous coding assistants, intelligent workflow bots, and AI co-pilots that can operate entire systems. 𝗧𝗵𝗲 𝗦𝗵𝗶𝗳𝘁 𝗶𝗻 𝗧𝗵𝗶𝗻𝗸𝗶𝗻𝗴 We’re no longer designing prompts. We’re designing 𝗺𝗼𝗱𝘂𝗹𝗮𝗿, 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 capable of interacting with the real world. This evolution—LLM → RAG → Agentic AI—marks the transition from 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 to 𝗴𝗼𝗮𝗹-𝗱𝗿𝗶𝘃𝗲𝗻 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲.

Microsoft just released a 35-page report on medical AI - and it’s a reality check for healthcare. The paper, “The Illusion of Readiness”, tested six of the most popular models (OpenAI, Gemini, etc)… across six multimodal medical benchmarks. And the verdict? The models scored high on medical exams. But they’re not even close to being real-world ready. Here’s what the stress tests revealed: ▶ 1. Shortcut learning Models often answered correctly even when key information, like medical images, was removed. They weren’t reasoning - they were exploiting statistical shortcuts. That means benchmark wins may hide shallow understanding. ▶ 2. Fragile under small changes Making small tweaks caused big swings in predictions. This fragility shows how unreliable model reasoning becomes under stress. In visual substitution tests, accuracy dropped from 83% to 52% when images were swapped - exposing shallow visual–answer pairings. ▶ 3. Fabricated reasoning Models produced confident, step-by-step medical explanations - but many were medically unsound… or entirely fabricated. Convincing to the eye, dangerous in practice. And more importantly, healthcare isn’t a multiple-choice exam. It’s uncertainty, incomplete data, and high stakes. So Microsoft’s team calls for new standards: - Stress tests that expose fragility - Clinician-guided guidelines that profile benchmarks - Evaluation of robustness and trustworthiness - not just leaderboard scores The takeaway is simple: Medical AI may ace tests today. But until it proves reliable under stress, it’s not ready for the clinic. When do you think popular LLMs will be clinic-ready? #entrepreneurship #healthtech #AI

In a MAJOR ruling for European copyright law, the Munich Regional Court has sided with Germany’s music rights society GEMA against OpenAI, finding that the company’s ChatGPT model unlawfully used copyrighted song lyrics in its training and responses. The decision, issued this morning, marks the first major European court judgment holding an AI company liable for using protected works without a licence. I got into AI through being Director of Legal Affairs and Regulatory Compliance in IMRO, the Irish counterpart of GEMA - and I know the people in GEMA - so this is very interesting to me. The case centred on GEMA’s allegation that OpenAI trained ChatGPT on its repertoire of German song lyrics, allowing the chatbot to reproduce works by artists such as Helene Fischer and Herbert Grönemeyer. The court agreed, concluding that the model’s ability to reproduce lyrics word for word demonstrated that the works had been used in training. It ruled that OpenAI is liable for copyright infringement and prohibited ChatGPT from reproducing lyrics from GEMA-represented artists unless a licence is obtained. The court also held that the European Union’s Text and Data Mining exceptions cannot shield generative AI systems that “memorise” and reproduce copyrighted material. This reasoning undermines one of the primary legal defences AI developers have relied upon in Europe. While damages will be determined in a separate proceeding, the court’s finding of liability alone sets a powerful precedent. OpenAI has announced plans to appeal. The 42nd Civil Chamber of the Munich Regional Court had indicated its position in September, when it observed that the model’s outputs could not be explained without training on copyrighted material. The final judgment confirmed that assessment. For the wider AI sector, the ruling suggests that AI companies operating in the European Union may need explicit licences for any copyrighted content used in model training or risk litigation. The decision also has regulatory implications. It aligns with growing momentum within the EU to enforce transparency and rights-holder protections under the AI Act and the Copyright in the Digital Single Market Directive. The GEMA v OpenAI ruling diverges sharply from Bartz v Anthropic in the United States. In Bartz, Judge Alsup found that AI training on copyrighted material could qualify as fair use, meaning no licence is required when the use is deemed transformative and non-substitutive. He viewed training as an analytical process that teaches the model general patterns rather than reproducing expression. The Munich court took the opposite view, holding that using protected works in AI training without permission constitutes reproduction requiring a licence. This illustrates the growing divide between the U.S. model, where fair use can exempt AI developers from licensing duties, and the European approach, which treats copyright as an enforceable economic right demanding prior authorisation.

🤖 Best chance to have well-informed discussions on AI : #AI Bible accessible for free ! 🗞️ The Cambridge Handbook on the Law, Ethics, and Policy of Artificial Intelligence, 2025 👓 contributions from experts 👓 theoretical insights and practical examples of AI applications The Handbook examines: 🔹the legal, ethical, and policy challenges of AI & algorithmic systems esp. in #Europe 🔹the societal impact of these technologies 🔹the legal frameworks that regulate them 📚 18 chapters 🎓 I : AI, ETHICS AND PHILOSOPHY 1 AI: A Perspective from the Field 2 Philosophy of AI: A Structured Overview 3 Ethics of AI: Toward a "Design for Values" Approach 4 Fairness and Artificial Intelligence 5 Moral Responsibility and Autonomous Technologies: Does AI Face a Responsibility Gap? 6 AI, Power and Sustainability ⚖️ II : AI, LAW AND POLICY 7 AI Meets the GDPR: Navigating the Impact of Data Protection on AI Systems 8 Tort Liability and AI 9 Al and Competition Law 10 Al and Consumer Protection 11 Al and Intellectual Property Law 12 The European Union's AI Act 🤖 III AI ACROSS SECTORS 13 Al and Education 14 Al and Media 15 Al and Healthcare Data 16 Al and Financial Services 17 Al and Labor Law 18 Legal, Ethical, and Social Issues of AI and Law Enforcement in Europe: The Case of Predictive Policing 👏🏼 Edited by Nathalie Smuha legal scholar at KU Leuven who specializes in AI’s impact on human rights, democracy, and the rule of law. 🔗 Cambridge University Press & Assessment .

Ever wondered where the future of AI is being built? I just visited the data centre in Finland that's making it happen. Nebius’ data centre is the powerhouse where AI models are trained. Thousands of GPUs working in unison. It’s expanding to host up to 60,000 GPUs dedicated to intensive AI workloads. They’re building a full-stack AI cloud platform. Here’s what I learned: 1. There is a scarcity of GPUs in the US • Clusters are being sold in massive packages • People who need smaller requirements can’t find them 2. Nebius are building a self-serve platform • Cover infrastructure requirements from a single GPU to big GPU clusters • They’re not a GPU reseller—they’re designing the servers and the racks from the ground up 3. Applications • Helped Mistral train their multimodal models • Provide full-stack infrastructure for AI model development Something else that was unique about the visit. Nebius cools the servers in Finland using the outside air. The heat that’s generated from the servers is then shipped back into the grid. This means Nebius not only heats the onsite building, But it also heats homes nearby, benefitting the local community. They’re able to recover 70% of the heat generated. And it’s the first in the world to have this heat reuse application connected to the local municipal grid. They’re now investing over $1B in AI data centres in Europe. I feel the future of AI depends on infrastructure like this that balances performance with sustainability. Follow me Alex Banks for daily AI highlights & insights.

We’re used to thinking of Big Tech as a cage match: Apple vs Google, Meta vs OpenAI, Amazon vs Microsoft. But go deeper down the stack and these rivalries dissolve into something less cinematic - inter-dependent supply chains. Consider just this past week: ➰ Meta signed a $10B+ cloud deal with Google, its fiercest rival in digital advertising. ➰ OpenAI is feeding ChatGPT with Google Search results (via SerpAPI) and renting its GPUs, while trying to make Google Search obsolete. There’s an entangled web of interdependencies in AI: your most threatening competitor is often your most critical vendor. Everyone sells the shovel, even to the guy digging their grave. So, what gives? 1. Moats are now Rentable. And often leased to the very people trying to cross them. What used to be a moat - distribution (iOS/Android), data (Search), or compute (GPUs at hyperscale) - is increasingly sold as a SKU. If your “defensive asset” can be metered, it will be monetized… even to your rivals. That sounds contradictory until you realize the real moat isn’t the resource - it’s the flywheel that replenishes it. Google can lease GPUs and still deepen its Gemini feedback loops. 2. Infrastructure is too Expensive to own Alone. The modern AI stack is fractured and expensive: - Compute (GPUs, interconnects, custom silicon) - Indexing (web crawlers, real-time feeds, proprietary corpora) - Modeling (foundation models, adapters, RAG) - Orchestration (retrievers, agents, tool use) - Distribution (hardware, OS defaults, app interfaces) No single company can win all five. So they do what every industry does when vertical integration becomes unscalable: they trade. AI isn’t owned. It’s assembled - by companies renting from rivals they’d love to replace. 3. Market Power comes from Volume. Take Meta. It’s signed deals with every major cloud provider: AWS, Azure, Oracle, CoreWeave, and now Google Cloud. This isn’t loyalty; it’s pricing arbitrage and regional hedging. At that scale, cloud is a commodity and power comes from being the customer that can move someone else’s earnings call. 4. Time-to-Quality > Ideological Purity. If the fastest path to product quality is to buy accuracy while you build your own index, you do both. You can always replace a vendor. You can’t buy back time. Months matter. In AI, months are market share. Meanwhile, Google selling compute to OpenAI is not charity; it’s toll collection on a rival’s growth curve. 5. Optics matter Turning your enemies into customers is not just good business - it’s good politics. Each hyperscaler that lands a rival as a marquee customer bolsters its narrative: To Wall Street: “We grow no matter who wins.” To regulators: “We’re not a monopoly, we power our competitors.” The stack is too entangled, too capital-intensive, and too unevenly distributed for anyone to play lone wolf. In this economy, independence is expensive and rivalry is mostly theater. Monetize your enemy’s ambition. The best revenge is recurring revenue.

The AI gave a clear diagnosis. The doctor trusted it. The only problem? The AI was wrong. A year ago, I was called in to consult for a global healthcare company. They had implemented an AI diagnostic system to help doctors analyze thousands of patient records rapidly. The promise? Faster disease detection, better healthcare. Then came the wake-up call. The AI flagged a case with a high probability of a rare autoimmune disorder. The doctor, trusting the system, recommended an aggressive treatment plan. But something felt off. When I was brought in to review, we discovered the AI had misinterpreted an MRI anomaly. The patient had an entirely different condition—one that didn’t require aggressive treatment. A near-miss that could have had serious consequences. As AI becomes more integrated into decision-making, here are three critical principles for responsible implementation: - Set Clear Boundaries Define where AI assistance ends and human decision-making begins. Establish accountability protocols to avoid blind trust. - Build Trust Gradually Start with low-risk implementations. Validate critical AI outputs with human intervention. Track and learn from every near-miss. - Keep Human Oversight AI should support experts, not replace them. Regular audits and feedback loops strengthen both efficiency and safety. At the end of the day, it’s not about choosing AI 𝘰𝘳 human expertise. It’s about building systems where both work together—responsibly. 💬 What’s your take on AI accountability? How are you building trust in it?

Neural networks might speak English, but they think in shapes. Understanding their rich neural geometry is key to understanding how they work, and to debugging and controlling them with precision. Starting today, our research team is publishing a new series on the hidden shapes inside AI models. Much of interpretability treats a model's concepts as arrows: linear directions in activation space. But that view flattens the richer structure that models actually learn. Neural networks build complex inner worlds with geometry that reflects the structure of reality. Days of the week form a circular loop in language models. The tree of life appears as a complex structure in a genomics model. We found a novel class of Alzheimer's biomarkers as a clean curve in an epigenomic model. This pattern shows up across models, modalities, and domains. Neural geometry lets us both understand models more deeply and also control their behavior more effectively. Steering often fails when we treat concepts as linear, but succeeds when we follow the geometric structures that models actually use. We think understanding neural networks is the most important problem and opportunity in AI, and research like this is a big part of how we get there. Huge credit for the incredible work behind this series to: Atticus Geiger, Ekdeep Singh Lubana, Daniel Wurgaft, Noah Goodman, Can Rager, Thomas Fel, Matthew Kowal, Vasudev Shyam, Sheridan Feucht, Usha Bhalla, Tal Haklay, Eric Bigelow, Raphaël Sarfati, Tom McGrath, Owen Lewis, Jack Merullo, and Michael Byun.

The NYT just reported that patients are uploading entire medical records into chatbots - but the risks are not what most people think. Patients are pasting labs, imaging, clinical notes, and oncology reports directly into LLMs. • 26-year-old told her labs “most likely” indicated a pituitary tumor. MRI: normal • 63-year-old advised to escalate to catheterization. Found ~85% LAD stenosis Because of how the chatbot responds, many assume the AI reasons about their symptoms and medical record the same way a clinician does. But AI systems are capable of both meaningful help and serious error, without any calibration signal visible to the user. Most worry about wrong AI recommendations. But the bigger risk is what the AI does not say. 📊 Harm preprint study A new Stanford-Harvard study (David Wu, MD, PhD, Fateme (Fatima) Nateghi, Adam Rodman, Jonathan H. Chen et al.) evaluated 31 models on 100 real outpatient eConsult cases across 10 specialties: - 4,249 management actions - 12,747 expert ratings Severe harm per 100 cases: - Best models: ~12–15 - Worst models: ~40 ~77% of severe harms were omissions: - Not ordering a critical test - Missing a needed referral - Neglecting follow-up suggestions 🔷 Additional findings: 1) Top models outperformed generalists using conventional resources (though these were difficult eConsult cases that PCPs were posing to specialists) 2) No link between safety and model size, recency, “reasoning modes,” or standard benchmarks 3) Multi-agent + RAG approaches reduced harm; heterogeneous ensembles had ~6× higher odds of top-quartile safety 📌 Implications When a patient asks AI for medical advice, the primary risk is not incorrect recommendations. It's neglecting critical actions a clinician might suggest (notably, humans also make a lot of mistakes). ⚠️ Why this matters 1) 2/3 of US physicians report using LLMs, and millions of patients. Errors will become more subtle as models get better. Both harms of omissions and commission will become harder for clinicians (and especially patients) to detect. 2) Sampling a few outputs is not enough: clinical AI evaluation needs explicit, systematic harm measurement on real cases, not just performance or accuracy on knowledge benchmarks. 3) If we don’t measure omission harms, we will systematically underestimate risk. 🔴 Open Call: State of Clinical AI Report (Jan 2026) The ARISE Network (Stanford + Harvard) is compiling a State of Clinical AI Report for 2026. Audience: health system leaders, clinicians, researchers, tech/pharma, media, investors 2025 peer reviewed and preprint studies within scope: • Clinical AI (doctor- or patient-facing) • Benchmarks, evaluations, real-world deployments, prospective trials • Workflow, outcomes, and implementation studies 📅 Submission deadline: Dec 21, 2025 - Comment with study link + 1–2 sentences on key findings and why it matters - We will follow up with a one-slide reference example for invited submissions

This week I found four papers on Google Scholar “written” by me and my co-authors. Except we didn’t write them. They were AI-generated fake citations. I see multiple risks associated with that happening: - Misinformation risks if fakes get referenced further, in academic research, policy, funding proposals, or practical guidelines. Especially in fields that impact people’s lives directly. - Erosion of trust in academic research: real research becomes harder to find; claims are harder to verify. - Collateral damage to journals that never published the research but are now cited as if they did. - Distorted journal and author metrics: fake citations inflate impact factors, h-indexes, and other performance indicators. - Reputational harm to the real authors falsely cited. - Legal exposure if harmful claims are falsely attributed to you. The same way countries are trying to figure out how to protect voices and faces to fight deepfakes and artworks to fight copyright fraud, we need knowledge and author protection in academic publishing. Until then, document and report such cases - because the more visible we make this problem, the harder it will be ignored. What else can be done? Have it ever happened to you? #academicintegrity #academia #informationsystems Electronic Markets - The International Journal on Networked Business Journal of Information Technology (JIT)