Alex G. Lee, Ph.D. Esq. CLP

DeepSeek AI in China's healthcare - is it a matter of "too fast, too soon"? In China, we are witnessing a revolution in #AI with DeepSeek. Search queries for DeepSeek in Baidu (China's Google) have surpassed those for OpenAI’s ChatGPT and Google Gemini by 10 times since late January 2025. In the healthcare sector, as of a search conducted on March 9, 2025, >300 hospitals in China have started "adopting" private, local, on-site deployments of DeepSeek AI, with attempts to integrate this "low cost", "open-source/weight", "reasoning-based" LLM into real-world clinical and hospital-related tasks, including purportedly clinical diagnostics and decision support, patient education, scientific research, and hospital management systems. In this JAMA, Journal of the American Medical Association Perspective, we share our preliminary thoughts of the potential opportunities, but also the risks and the need for robust regulatory and policy guidelines for the use and adoption of #AI and other #LLM in healthcare, not only for China but globally. #medicine #healthcare #clinicalinnovation #aimedicine #aihealthcare #deepseek #deeplearning

131

5 Comments

Healthcare AI is entering its Foundation Model Era The floodgates just opened for clinical-grade, healthcare-specific AI models...and VCs are rushing in. As Ngai Yeung mentioned in her recent article: 👉 Aidoc raised $150M from General Catalyst and NVIDIA’s venture arm to flag critical findings in imaging exams. 👉 Ash by Slingshot AI: a therapy chatbot trained on real session transcripts just extended its Series A to $93M with backing from a16z. 👉 OpenEvidence landed $210M (yes, B round!) for a search engine trained on medical journals. this was called “the most competitive deal” by a GV investor. What ties these together? *They’re all built on foundational models: LLMs trained not just on text, but on multimodal healthcare data: imaging, audio, notes, EMR dumps, and beyond. These models don’t just summarize: they reason, flag, search, predict, and increasingly... act. And we’re just getting started. Expect foundational model-native startups to pop up across every specialty, especially those with highest cost to payors and lowest consumer access: - MSK - Oncology - Cardiology - Women’s health The next phase? *Agentic workflows built on top of these models. Think ambient scribe → care coordinator → revenue cycle → diagnostics → population health. All wrapped into modular, intelligent assistants. Clinical productivity jumps, staffing headaches shrink, and a new UX layer is born. We’re watching a new platform shift that is purpose-built for healthcare.

130

28 Comments

🚨 Just In: Draft of FDA AI Governance Guide for 2025 2025 is shaping up to be a landmark year for AI governance. With the growing complexity of AI systems, the FDA’s recent “Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations” whitepaper couldn’t have come at a better time. What’s most striking about this publication? It doesn’t aim to reinvent the wheel—it organizes the fundamentals. It provides a crystal-clear roadmap for organizations looking to align AI-enabled devices with robust governance frameworks. Key Takeaways: 1️⃣ Transparency by Design – The paper emphasizes integrating transparency into AI systems from the ground up, ensuring that users and regulators alike can trust these technologies. Think of it as building a foundation of trust before the first AI model even goes live. 2️⃣ Bias Management – It’s not just about detecting issues post-launch. This guide stresses proactive measures, like diverse data collection and robust validation, to reduce systemic biases. 3️⃣ Comprehensive Risk Assessments – Highlighting the Total Product Lifecycle (TPLC), the paper advocates for continuous evaluation and improvement of AI systems, ensuring safety and efficacy over time. This document offers a practical, sector-specific lens for AI governance. It’s not an abstract vision—it’s a toolset for developers, manufacturers, and regulators to collaborate effectively. In the ever-evolving landscape of AI, such grounded resources remind us of the importance of clarity and practicality. This is the kind of guidance we’ve been waiting for to ensure AI-driven innovation remains safe and equitable. 🌟 2025 could also be the year you make a mark in AI governance. Will you join the movement for responsible innovation? #AIGovernance #ResponsibleAI #TechRegulation #AICompliance #InnovationForGood __________________________________ Did you like this post? Connect or Follow 🎯 Jakub Szarmach, AIGP Want to see all my posts? Ring that 🔔

56

Great panel at the nference Agentic AI Innovation Summit on The Future of Agentic AI in Biomedical Innovation. I really enjoy participating on panels such as this. The preparation and co-panelists really make me think about the thoughtful questions being ask. Today's conversation about AI autonomy and accountability and scientific and operational oversight built for distributed agency didn't disappoint. The Hard Truth: Accountability Can't Be Diffuse When Consequences Are Real As we deploy agentic AI across drug development, from target identification to trial optimization, we need more than philosophical frameworks. We need infrastructural redesign. I believe in a layered liability model: → Model builders own validity → Deployers own contextual fitness → Sponsors own ultimate clinical and regulatory accountability But here's what makes this real: algorithmic audit trails as rigorous as our data audit trails. Every agent action must be reconstructable detailing what it saw, what logic it applied, and what oversight was present. On Scaling Responsibly: We Need Boring Infrastructure, Not Just Brilliant Algorithms Let's be clear: almost all healthcare interventions are regulated and reviewed by the FDA or some government agency. While not every agentic AI application needs this level of scrutiny, any system making critical patient health decisions absolutely must be. We can't carve out exceptions just because the technology is novel. Three non-negotiables: Validation frameworks matched to risk profile. A model suggesting targets needs different validation than one executing dose calculations. We need FDA-grade pathways tiered by consequence, not novelty. Human checkpoints at irreversible decisions. Agents can propose and execute within bounds, but any action that locks in patient exposure or regulatory commitment needs a human signature. Cultural honesty about optimization goals. Are we saving time? Reducing bias? Cutting costs? Vague claims about "augmentation" erode trust faster than honest trade-offs. Bottom line: We scale when systems are more auditable than the humans they replace and not more opaque. #AgenticAI #BiomedicalInnovation #DrugDevelopment #AIinHealthcare #RegulatoryScience #ResponsibleAI

36

BREAKING NEWS! Two popular large language models (LLMs), GPT-4 and Llama-3, produced medical device-like decision support across a range of scenarios even though those are not regulated by the FDA. Authors of a new study were interested in finding out whether LLMs show promise for clinical decision support (CDS) even though none is currently authorized by the FDA as a CDS device.  "𝐹𝑜𝑙𝑙𝑜𝑤𝑖𝑛𝑔 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 𝑟𝑒𝑞𝑢𝑒𝑠𝑡𝑠 𝑎𝑏𝑜𝑢𝑡 𝑡𝑖𝑚𝑒-𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 𝑒𝑚𝑒𝑟𝑔𝑒𝑛𝑐𝑖𝑒𝑠, 𝟣𝟢𝟢% 𝑜𝑓 𝐺𝑃𝑇-𝟦 𝑎𝑛𝑑 𝟧𝟤% 𝑜𝑓 𝐿𝑙𝑎𝑚𝑎-𝟥 𝑟𝑒𝑠𝑝𝑜𝑛𝑠𝑒𝑠 𝑤𝑒𝑟𝑒 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑑𝑒𝑣𝑖𝑐𝑒-𝑙𝑖𝑘𝑒 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 (𝐹𝑖𝑔. 𝟣). 𝐴𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 𝑠𝑢𝑔𝑔𝑒𝑠𝑡𝑖𝑜𝑛𝑠 𝑤𝑒𝑟𝑒 𝑐𝑙𝑖𝑛𝑖𝑐𝑎𝑙𝑙𝑦 𝑎𝑝𝑝𝑟𝑜𝑝𝑟𝑖𝑎𝑡𝑒 𝑎𝑛𝑑 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑𝑠 𝑜𝑓 𝑐𝑎𝑟𝑒." Conclusions? 1) LLMs are capable of CDS! 2) If so, people will use them on a massive scale. 3) If so, it would make a lot of sense for regulators to provide regulatory oversight even though LLMs provide disclaimers that they are not intended to be used for medical advice. And these are just unimodal LLMs. Imagine the regulatory challenge that will come with multi-modality. Study: https://lnkd.in/eaCCbXed

66

12 Comments

Commercial fine-tuning services for LLMs struggle to teach models updated medical knowledge, with even the best-performing models showing limited generalization. 1️⃣ Researchers fine-tuned six major LLMs (GPT-4o, Gemini, Llama) using new drug approvals, synthetic patient data, and updated guidelines from 2024. 2️⃣ Fine-tuned models could memorize training facts (e.g., exact blood pressure values or drug doses) but often failed to apply them in clinical scenarios. 3️⃣ GPT-4o mini performed best, with up to 51% accuracy on test questions about new drugs, but still showed limited generalization. 4️⃣ Models did worst on synthetic EHR data, where accuracy dropped to 12% on average, highlighting challenges with patient-specific memory. 5️⃣ For updated clinical guidelines, OpenAI models showed moderate performance (around 35%), but most others failed to adapt effectively. 6️⃣ Fine-tuning led to minor drops in general medical reasoning (MedQA scores), especially for larger models like GPT-4o. 7️⃣ Gemini and Llama models showed both low memorization and generalization; some even refused to answer test questions. 8️⃣ Model size may influence fine-tuning success: smaller models like GPT-4o mini appeared more adaptable. 9️⃣ Retrieval-based methods (like RAG) remain essential; current fine-tuning tools are not reliable for real-time clinical updates. 🔟 The authors stress the need for transparent, improved fine-tuning methods before these tools can be safely used in medical AI systems. ✍🏻 Eric W., Kevin Wu, James Zou. Limitations of Learning New and Updated Medical Knowledge with Commercial Fine-Tuning Large Language Models. NEJM AI. 2025. DOI: 10.1056/AIcs2401155. Image Copyright © 2025 Massachusetts Medical Society. Reprinted with permission from Massachusetts Medical Society.

74

3 Comments

AdvaMed has just released its AI Policy Roadmap, designed to promote the next era of AI-enabled medical and digital health technologies.   AI has tremendous potential to automate routine tasks and deliver deeply personalized care, and the continued support of Congress and the Administration will be important in creating policies necessary to help further ensure as many patients as possible have access to AI-enabled innovations. We hope that this roadmap serves as a useful guide.   https://lnkd.in/gUcP4P-X

88

3 Comments

A scoop from the prolific Sean Whooley at MassDevice, on yet another major funding round for a #braincomputerinterface (#BCI) startup. The South China Morning Post reports today that brain-computer interface (BCI) startup StairMed brought in $48 million in funding. Shanghai-based StairMed, a developer of implantable BCIs and surgical robots, reportedly said it raised 350 million yuan. China’s Qiming Venture Partners, OrbiMed, and Lilly Asia Ventures 礼来亚洲基金 led the funding round. SCMP reports that StairMed claims its round marks the largest-ever funding round for a Chinese startup in the implantable BCI space. StairMed plans to use funds to accelerate clinical trials, support R&D and to build a new production facility. This funding adds to a previous round that brought in “hundreds of millions of yuan.” SCMP reports that StairMed touts ultra-flexible neural electrodes that come in at a fifth of the size of BCI competitor Neuralink’s electrodes. The company also claims its material is significantly softer to offer a less intrusive option for brain tissue. https://lnkd.in/g3-F5QRF #neurotechnology #neurotech Neurotech Futures

31

🕹️𝐁𝐑𝐈𝐃𝐆𝐄: 𝐀 𝐂𝐫𝐢𝐭𝐢𝐜𝐚𝐥 𝐒𝐭𝐞𝐩 𝐓𝐨𝐰𝐚𝐫𝐝 𝐑𝐞𝐬𝐩𝐨𝐧𝐬𝐢𝐛𝐥𝐞 𝐀𝐈 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐢𝐧 𝐇𝐞𝐚𝐥𝐭𝐡𝐜𝐚𝐫𝐞 🔹Earlier today at HLTH Europe, Trent Norris from NVIDIA announced the official release of #BRIDGE — a collaborative governance framework designed to support safe, responsible AI deployment in clinical care. Read more in the comments below 👇🏻 🔹Proud to have contributed to this multi-institutional effort alongside colleagues across clinical practice, industry, regulatory science, AI development, and health system leadership. Thankful to Aidoc for the opportunity and Joshua Streit for his leadership! “𝘉𝘶𝘵 𝘸𝘩𝘺 𝘴𝘩𝘰𝘶𝘭𝘥 𝘸𝘦 𝘤𝘢𝘳𝘦??” AI in healthcare is not static — it 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬𝐥𝐲 𝐞𝐯𝐨𝐥𝐯𝐞𝐬 as new data, clinical contexts, and populations emerge. If we are to responsibly scale AI into real-world patient care, governance frameworks like BRIDGE are essential. “Why??” You might ask are they essential ⤵️ ✅ 𝐌𝐮𝐥𝐭𝐢-𝐬𝐭𝐚𝐤𝐞𝐡𝐨𝐥𝐝𝐞𝐫 𝐜𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧 Clinical AI lives at the intersection of patient care, regulatory oversight, data science, and health system operations. Frameworks like BRIDGE formalize shared guardrails across these domains. ✅ 𝐅𝐮𝐥𝐥-𝐥𝐢𝐟𝐞𝐜𝐲𝐜𝐥𝐞 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 Evaluation cannot end at model deployment. Responsible oversight must include ongoing monitoring of: • Model accuracy & performance drift • Equity & subgroup generalizability • Clinical relevance & usability • Transparency on model updates • Explainability for clinicians • Safety monitoring & incident response ✅ 𝐒𝐮𝐬𝐭𝐚𝐢𝐧𝐚𝐛𝐥𝐞 𝐢𝐧𝐧𝐨𝐯𝐚𝐭𝐢𝐨𝐧 𝐫𝐞𝐪𝐮𝐢𝐫𝐞𝐬 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 The pathway to scale is not just better algorithms — it’s building durable systems of accountability that ensure ongoing model reliability, transparency, and safety in 𝘥𝘺𝘯𝘢𝘮𝘪𝘤 𝘤𝘢𝘳𝘦 𝘦𝘯𝘷𝘪𝘳𝘰𝘯𝘮𝘦𝘯𝘵𝘴. 🔹AI’s impact in healthcare will ultimately be defined not just by what we build — but also 𝘩𝘰𝘸 𝘸𝘦 𝘨𝘰𝘷𝘦𝘳𝘯 𝘸𝘩𝘢𝘵 𝘸𝘦 𝘥𝘦𝘱𝘭𝘰𝘺. Congratulations 🎉 again to our #BRIDGE team on moving the field in the right direction. Joshua Streit Heather Cmiel Leonardo Kayat Bittencourt Nina Kottler, MD, MS, FSIIM Brad Genereaux, FSIIM UW Medicine UW Medicine + Heart Institute University Hospitals Aidoc Demetri Giannikopoulos Edward Zaragoza Michael A. Bruno, M.D., F.A.C.R. Alex Towbin Stephen Aylward Brenton Hill, JD, MHA Coalition for Health AI (CHAI) Amit Gupta Ashley Weber #BRIDGEFramework #AIinHealthcare #ResponsibleAI #AIGovernance #HealthEquity #PatientSafety #ClinicalAI #HLTH2025 #DigitalHealthLeadership #FutureOfMedicine

29

3 Comments

Is your real world data is still sitting in silos? 🔺 Here's the disconnect 👇👇 Many MedTech companies have real-world data but are uncertain how to transform the data into evidence that regulators actually accept. MedTech Leading Voice, in cooperaation with Project Medtech proudly present speakers Amelia Hufford, Kelley Kendle, Steve Levine, and Jay V., this Thursday for "Closing the Blind Spots: Designing Devices for the Real World" Thursday, Sept 18 • 10-11 AM ET* (*Recording available to all registrants) Register with one click: https://lnkd.in/eYj4WYa9 🔥 The gap between data collection and evidence generation is costing companies millions in missed opportunities. Here's a quick 101 guide to bridge that gap: ✅ Data vs Evidence Foundation ↳ RWD is raw patient information from routine care settings ↳ RWE is clinical evidence derived from analyzing that data ↳ Remember: Quality RWD is essential, but RWE drives decisions ✅ Strategic Data Sources ↳ EHRs and claims databases provide broad population insights ↳ Patient registries offer disease-specific depth ↳ Wearables and apps capture real-time behavioral data ✅ Evidence Generation Process ↳ Define objectives before you start collecting data ↳ Assess data quality and completeness early ↳ Build analytical frameworks that regulators trust ✅ Regulatory Applications ↳ Use external control arms to reduce trial costs ↳ Support label expansions with post-market evidence ↳ Demonstrate comparative effectiveness vs. competitors ✅ Quality by Design ↳ Your RWE strategy should start in development, not post-market ↳ Partner with registries and health systems early ↳ Engage FDA for guidance on evidence requirements The secret to RWE success lies in building data collection into your development process from day one. Real world evidence creates competitive advantages by accelerating approvals and reducing trial costs. - Sean P.S. What's your biggest challenge in turning real world data into regulatory evidence? P.P.S. This is part 2 of 3, tomorrow I will share the In Silico Testing Cheat Sheet #RealWorldEvidence #MedTech #RegulatoryAffairs #DataScience

39

8 Comments

Part 4: Evaluation for AI in Healthcare – Reflecting on HealthBench, Gaps We’re Bridging, and What’s Next OpenAI’s HealthBench is a major milestone in the healthcare AI space. It finally brings a rigorous, clinically grounded structure to evaluating LLMs—with themes like emergency escalation, communication nuance, and global diversity. As someone working at the intersection of AI, health behavior, and patient-facing tools, I see this as a gold-standard baseline we’ve long needed. Why it matters: It finally moves us beyond outdated exam-style benchmarks (like MedQA or USMLE-style MCQs) and into real-world, scenario-based evaluation-synthetic cases written by researchers, validated by physicians. Finally, a benchmark that feels closer to real patient interaction. At our end, we’ve been developing a comprehensive evaluation pipeline for healthcare AI—especially for tools used in sexual health, triage, and digital guidance. And this new benchmark helps us align efforts while also identifying where real-world complexity still needs deeper reflection. Where HealthBench Raises the Bar: • Meaningful axes like emergency referral, response depth, and global diversity • A shared benchmark we can finally build around • A thoughtful first move toward standardized healthcare evaluation - especially suited for general medical guidance • And backed by a robust rubric (48,000+ grading criteria) with input from 262 physicians across 26 specialties The areas we believe need further reflection: 1. Sexual Health and Sensitive Domains → These are often excluded due to cultural complexity—but they’re the exact areas patients hesitate most. We’re actively building a sexual health–specific evaluation track to address nuance, stigma, and user safety. 2. Multimodal Testing → Real-world healthcare tools aren’t text-only. They use images, symptom photos, even voice input. Our framework integrates text + image evaluation— starting with tools used in digital triage, sexual health and dermatology. 3. Privacy & Consent Simulation Consent isn’t always clean-cut. We test real-world gray areas: • Uploading someone else’s data • Ambiguous user identity • Simulating underage use These edge cases are critical to ensure tools don’t just follow policy—but behave ethically. 4. Behavioral & Emotional Safety → We don’t just evaluate what the AI “knows”—we test how it behaves when a user is confused, anxious, or hesitant. Because in healthcare, trust is built—or lost—in those small, human moments. We’ve been conducting weekly deep-dive evaluations—one focus area at a time. We are aligned with healthbench in key areas and also go beyond—to reflect sensitive use cases, human behavior, and real-world risks. Our eval pipeline—particularly those built for high-context, sensitive, and multimodal healthcare tools. Because in healthcare, evaluation must go beyond benchmarks. Yudara Kularathne MD, FAMS(EM) Thanveer Ahamed Janitha Prathapa Dr Sithira A.

22

2 Comments

Did you see the recent news??? Microsoft recently unveiled its latest AI Diagnostic Orchestrator (MAI DxO), reporting an impressive 85.5% accuracy on 304 particularly complex cases from the New England Journal of Medicine, compared to just ~20% for physicians under controlled conditions . These results—quadrupling the diagnostic accuracy of human clinicians and more cost-effective than standard pathways — have gotten a lot of buzz. They may mark a significant milestone in clinical decision support and raise both enthusiasm but also caution. Some perspective as we continue to determine the role of AI in healthcare. 1. Validation Is Essential Promising results in controlled settings are just the beginning. We urge Microsoft and others to pursue transparent, peer reviewed clinical studies, including real-world trials comparing AI-assisted workflows against standard clinician performance—ideally published in clinical journals. 2. Recognize the value of Patient–Physician Relations Even the most advanced AI cannot replicate the human touch—listening, interpreting, and guiding patients through uncertainty. Physicians must retain control, using AI as a tool, not a crutch. 3. Acknowledge Potential Bias AI is only as strong as its training data. We must ensure representation across demographics and guard against replicating systemic biases. Transparency in model design and evaluation standards is non-negotiable. 4. Regulatory & Liability Frameworks As AI enters clinical care, we need clear pathways from FDA approval to liability guidelines. The AMA is actively engaging with regulators, insurers, and health systems to craft policies that ensure safety, data integrity, and professional accountability. 5. Prioritize Clinician Wellness Tools that reduce diagnostic uncertainty and documentation burden can strengthen clinician well-being. But meaningful adoption requires integration with workflow, training, and ongoing support. We need to look at this from a holistic perspective. We need to promote an environment where physicians, patients, and AI systems collaborate, Let’s convene cross sector partnerships across industry, academia, and government to champion AI that empowers clinicians, enhances patient care, and protects public health. Let’s embrace innovation—not as a replacement for human care, but as its greatest ally. #healthcare #ai #innovation #physicians https://lnkd.in/ew-j7yNS

186

40 Comments

It is exciting news to see the Eli Lilly and Company x a16z $500M partnership recognizing that the future of drug development lies at the intersection of biology and computation. We will hopefully see significant breakthroughs in AI-driven drug discovery and precision medicine in the next few years! For those of us building in healthtech/biotech, this validates what many founders have been saying: to accelerate biotech innovation, you need the involvement, expertise and funding from pharma alongside cutting-edge tech and VC funding. Curious to hear others' thoughts on this new model of pharma-VC partnership. #Biotech #HealthTech #VentureCapital #VC #Pharma

24

2 Comments