Yanni Pandelidis

BREAKING NEWS! Two popular large language models (LLMs), GPT-4 and Llama-3, produced medical device-like decision support across a range of scenarios even though those are not regulated by the FDA. Authors of a new study were interested in finding out whether LLMs show promise for clinical decision support (CDS) even though none is currently authorized by the FDA as a CDS device.  "𝐹𝑜𝑙𝑙𝑜𝑤𝑖𝑛𝑔 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 𝑟𝑒𝑞𝑢𝑒𝑠𝑡𝑠 𝑎𝑏𝑜𝑢𝑡 𝑡𝑖𝑚𝑒-𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 𝑒𝑚𝑒𝑟𝑔𝑒𝑛𝑐𝑖𝑒𝑠, 𝟣𝟢𝟢% 𝑜𝑓 𝐺𝑃𝑇-𝟦 𝑎𝑛𝑑 𝟧𝟤% 𝑜𝑓 𝐿𝑙𝑎𝑚𝑎-𝟥 𝑟𝑒𝑠𝑝𝑜𝑛𝑠𝑒𝑠 𝑤𝑒𝑟𝑒 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑑𝑒𝑣𝑖𝑐𝑒-𝑙𝑖𝑘𝑒 𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 (𝐹𝑖𝑔. 𝟣). 𝐴𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 𝑠𝑢𝑔𝑔𝑒𝑠𝑡𝑖𝑜𝑛𝑠 𝑤𝑒𝑟𝑒 𝑐𝑙𝑖𝑛𝑖𝑐𝑎𝑙𝑙𝑦 𝑎𝑝𝑝𝑟𝑜𝑝𝑟𝑖𝑎𝑡𝑒 𝑎𝑛𝑑 𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑡 𝑤𝑖𝑡ℎ 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑𝑠 𝑜𝑓 𝑐𝑎𝑟𝑒." Conclusions? 1) LLMs are capable of CDS! 2) If so, people will use them on a massive scale. 3) If so, it would make a lot of sense for regulators to provide regulatory oversight even though LLMs provide disclaimers that they are not intended to be used for medical advice. And these are just unimodal LLMs. Imagine the regulatory challenge that will come with multi-modality. Study: https://lnkd.in/eaCCbXed

66

12 Comments

I’m thrilled to announce our 510(k) submission for Photonova Spectra,* GE HealthCare’s next-generation photon-counting CT system with eight bins of energy resolution, powered by our proprietary Deep Silicon detector technology (510(k) pending with the U.S. FDA; not available for sale).   Why does this matter? Traditional CT converts X-rays into light before measuring them. Photon counting directly counts individual photons and measures their energy, enabling the potential for high resolution and impressive tissue characterization. Our design goals for Photonova Spectra include:   - Multi-energy data for higher spectral and spatial resolution and impressive tissue characterization  - Ultra-high-definition imaging to help reveal fine anatomical details - Quantitative material maps designed to help clinicians characterize diseases with confidence   Our Deep Silicon detector design with edge-on geometry and eight bins of energy resolution is designed to minimize noise and charge sharing, helping deliver sharp images even. Combined with advanced algorithms, Photonova Spectra is engineered to help clinicians visualize subtle tissue differences, small lesions and vascular structures.   This is more than technology, it’s a bold step toward precision care. From cardiology to oncology to musculoskeletal imaging, the potential is enormous.   We innovate with intention. And today, that intention is clear: to empower clinicians and transform care through science-driven innovations.   Read the full announcement in the first comment. #PhotonovaSpectra #GEHealthCare #PCCT #photoncounting #RSNA25   *Photonova Spectra is 510(k)-pending with the U.S. FDA. Not CE Marked. Not available for sale.

556

8 Comments

Based on an analysis of over 1400 research papers, this paper provides a comprehensive and systematic review of Context Engineering techniques for LLMs and introduces a unifying taxonomy for describing the field. #llm https://lnkd.in/g78gC5fM

8

🧠📄 AI Reviewers Are a Symptom of a Broken Publishing System 🤖 A recent arXiv study (Sahoo et al., 2025 - https://lnkd.in/ez2dBR9Q) shows that LLM-based scientific reviewers can be manipulated: hidden instructions in a manuscript can flip an AI decision from reject to accept, without improving the science. This is not just an AI problem. It exposes a deeper failure in current publishing policies! 😱 Peer review takes months, not because it is inefficient, but because expert reviewers are overloaded, unpaid, and undervalued. Journals rely on free academic labour, while publication volumes and profit margins continue to rise. The predictable consequences: • fewer qualified reviewers • human reviewers using LLMs as shortcuts • journals tempted to automate review to reduce costs If scientific output starts being validated by AI, we risk replacing judgement with pattern matching and accountability quietly disappears. And once this becomes normal in journals, grant evaluation will be next… AI can assist with formatting and checks. Scientific merit must remain a human responsibility and that responsibility must be properly rewarded. Using AI reviewers doesn’t fix peer review. It reveals how broken the system already is. #innovation #research #AI #PromptInjection #review #science

69

2 Comments

Delighted to share our new work integrating propensity scores with the self-adapting mixture (SAM) prior to dynamically and efficiently borrow information from historical data in hybrid-control randomized trials. This method can be easily implemented using the R package “SAMprior”. https://lnkd.in/gd-g-R8c

141

2 Comments

Most AI is entering clinical research through the side door. Eligibility algorithms screen charts before investigators ever see them. Imaging models generate endpoints. Monitoring systems decide who gets a safety call first. Yet the documentation supporting these tools is often thinner than what we expect for a single lab assay. If an algorithm can change who’s recruited, what outcomes are measured, or when a trial stops early, it deserves the same level of scrutiny as any other core component of the protocol. That means being explicit about scope, data, performance, transportability, drift management, consent, oversight, and exit. This cheatsheet is designed as a one-page checklist sponsors, investigators, and IRBs can lift directly into review templates. If an AI‑enabled tool can’t clear these questions, it isn’t ready to sit inside your trial. Follow Rizwan Tufail for evidence‑based playbooks on governing AI in trials, from contracts and validation packages to monitoring, drift, and regulatory readiness.

43

12 Comments

ARPA-H just launched a $55 billion program to build autonomous AI for cardiovascular care. Over 150 organizations posted teaming profiles. I analyzed them all. Here’s what jumped out: 𝗧𝗵𝗲 𝗺𝗶𝘀𝘀𝗶𝗼𝗻: Deploy AI agents that can prescribe medications and manage heart failure 24/7. Why? 46% of U.S. counties have zero cardiologists. 𝗪𝗵𝗼’𝘀 𝗮𝗽𝗽𝗹𝘆𝗶𝗻𝗴: ∙ 37% AI startups (Doctronic already practices medicine legally in Utah) ∙ 13% academic medical centers (all seeking industry AI partners) ∙ 6% Big Tech (all seeking clinical partners) ∙ 7% health systems (surprisingly few) ∙ 8% defense contractors (bringing DARPA safety-critical systems expertise) 𝗧𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺: ADVOCATE targets rural America. The teaming profiles cluster in Boston (17%), San Francisco (13%), and NYC (10%). Only 6 organizations explicitly focus on rural healthcare. We’re building urban solutions for a rural problem. 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗮𝗽𝗽𝗿𝗼𝗮𝗰𝗵𝗲𝘀 𝗮𝗿𝗲 𝗮𝗹𝗹 𝗼𝘃𝗲𝗿 𝘁𝗵𝗲 𝗺𝗮𝗽: Multi-agent systems simulating physician consensus. Digital twins modeling individual physiology. Speech-based heart failure detection requiring no wearables. Foundation models vs. specialized cardiovascular AI. Nobody knows which approach wins yet. 𝗧𝗵𝗲 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗰 𝗯𝗹𝗶𝗻𝗱𝘀𝗽𝗼𝘁: Everyone’s competing to build the patient-facing agent (TA1). Fewer teams are focused on the “supervisory agent” (TA2) which must be disease-agnostic and will become the oversight infrastructure for all future clinical AI. That’s the operating system layer. Whoever wins TA2 controls the platform. 𝗪𝗵𝗮𝘁 𝗰𝗼𝗻𝗰𝗲𝗿𝗻𝘀 𝗺𝗲 𝗺𝗼𝘀𝘁: Big Tech has the models. Startups have the regulatory pathways. Academic centers have the clinical credibility. 𝘉𝘶𝘵 𝘩𝘢𝘳𝘥𝘭𝘺 𝘢𝘯𝘺 𝘩𝘦𝘢𝘭𝘵𝘩 𝘴𝘺𝘴𝘵𝘦𝘮𝘴 𝘢𝘳𝘦 𝘱𝘢𝘳𝘵𝘪𝘤𝘪𝘱𝘢𝘵𝘪𝘯𝘨 𝘢𝘯𝘥 𝘵𝘩𝘦𝘺’𝘳𝘦 𝘵𝘩𝘦 𝘰𝘯𝘦𝘴 𝘸𝘩𝘰 𝘢𝘤𝘵𝘶𝘢𝘭𝘭𝘺 𝘩𝘢𝘷𝘦 𝘵𝘰 𝘥𝘦𝘱𝘭𝘰𝘺 𝘵𝘩𝘪𝘴. Technology isn’t the bottleneck. Implementation is. 𝗪𝗵𝗮𝘁 𝘁𝗵𝗶𝘀 𝗺𝗲𝗮𝗻𝘀 𝗳𝗼𝗿 𝗽𝗵𝘆𝘀𝗶𝗰𝗶𝗮𝗻𝘀: The systems being designed now will define cardiovascular care for decades. Every major tech company is explicitly seeking clinical partners. The choice isn’t whether clinical AI happens—it’s whether it happens with adequate physician input. As both a cardiologist and someone building in this space, I think we have a responsibility to engage. The alternative is systems built without us, deployed into workflows they don’t understand, serving patients whose needs weren’t prioritized.

60

10 Comments

Penn Engineering Engineers Unveil Generative AI Model that Designs New Antibiotics: What if generative AI could design life-saving antibiotics, not just art and text? In a new Cell Press biomaterias paper, Penn researchers introduce AMP-Diffusion, a generative AI tool used to create tens of thousands of new antimicrobial peptides (AMPs) — short strings of amino acids, the building blocks of proteins — with bacteria-killing potential. In animal models, the most potent AMPs performed as well as FDA-approved drugs, without detectable adverse effects.  “Nature’s dataset is finite; with AI, we can design antibiotics evolution never tried,” says César de la Fuente Presidential Associate Professor in Bioengineering (BE) and in Chemical and Biomolecular Engineering in the University of Pennsylvania School of Engineering and Applied Sciences,  in Psychiatry and Microbiology in the University of Pennsylvania Perelman School of Medicine, and in Chemistry at the University of Pennsylvania, Graduate Division of the School of Arts & Sciences, and the papers senior co author. “We’re leveraging the same AI algorithms that generate images, but augmenting them to design potent new molecules,” adds Pranam Chatterjee Assistant Professor in BE and in Computer and Information Science within Penn Engineering, and the paper’s other senior co-author, who began work on the project while at Duke University. So, is this the future of Generative AI in structural Biology and in computational protein and antibody design? Tell me what you think 🙏 I'll share the paper published in Cell in the comments

8

2 Comments

Stanford and Harvard just released their first "State of Clinical AI" report. The findings should concern every healthcare product team. Even the best-performing AI models make 12 to 15 severe clinical errors per 100 patient cases. The worst exceed 40. The report by the ARISE interdisciplinary research network evaluated 31 large language models against 100 real primary care consultations spanning 10 specialties. The gap between controlled research settings and real clinical environments is massive. Here's what matters for healthcare UX: the report explicitly calls for prioritizing human-computer interaction design in clinical decision support trials as much as primary outcomes. Translation: how clinicians interact with AI recommendations is just as important as the AI's accuracy. The researchers found a "jagged frontier" in AI capabilities. Some models show superhuman performance on narrow tasks. Those same models fail to recognize when they don't know something. That uncertainty gap is a design problem. When an AI is confidently wrong 15% of the time, the interface needs to help clinicians catch it. That means designing for appropriate trust, not blind trust. Showing reasoning. Making it easy to override. Building in friction at the right moments. The report's core argument: regardless of how fast the technology moves, the field needs to move more deliberately, with measurement focused on outcomes that matter in real-world care. The bottleneck is designing systems where humans and AI actually work well together. Source: https://lnkd.in/g-j5JVDF Learn Healthcare UX: https://lnkd.in/gaNDBRTc Scholarships: https://lnkd.in/gyM3Shqi Book a chat: https://HXRlabs.com

219

12 Comments

TEAM-AI Research Update - We have been investigating the use of language models for clinical information retrieval and computational phenotyping for over a decade through the OHNLPIR project. Understanding the science behind generative language models is necessary when translating AI for real-world applications. Excited to share this research led by Andrew Wen is now published! 📚 Context matching is not reasoning when performing generalized clinical evaluation of generative language models https://lnkd.in/g3PjtH7F @SN and #SNAuthor.

71

5 Comments

FDA rolls out generative AI tool ‘Elsa’ to speed up reviews and streamline regulatory tasks >> 💊The FDA is rolling out Elsa, a secure generative AI tool that helps staff accelerate clinical reviews, summarize adverse events, compare drug labels, and even generate code for internal systems 💊Elsa is built on a large language model and housed in a high-security GovCloud environment, ensuring sensitive regulatory data stays in-house and not trained on by external models 💊Early results from pilot testing with FDA scientific reviewers were positive, leading to the accelerated, under-budget deployment across all centers (original target launch date was June 30th) 💊Elsa’s debut is seen as the first step in a broader AI integration strategy that will expand to include advanced analytics and further generative AI use cases 💊FDA leadership is positioning AI as a lever to boost performance without compromising scientific rigor, describing Elsa as a tool that “enhances and optimizes the potential of every employee.” 💊Elsa launches amid a proposed 4% FDA budget cut and loss of up to 3,500 staff, potentially helping offset pressure on review timelines #digitalhealth #ai #pharma

146

7 Comments

A good demonstration of how graph data science can uncover hidden structure beyond what traditional correlation matrices show. In healthcare, we likewise wrestle with "latent relationships" - for example, how patients, comorbidities, treatments, care pathways, and social determinants of health interact in ways linear models often miss. https://lnkd.in/gpCRCh8i Maia Kennedy Hannah MacDonald Ryan Farhat-Sabet

21

AI agents are built on three fundamental scientific breakthroughs concerning Large Language Models (LLMs): 1. **Scaling Test-Time Compute:** Models exhibit enhanced performance when more computations are carried out during inference. (Source: [https://lnkd.in/gFpx-8Ck)) 2. **Decomposing Complex Tasks:** Breaking down intricate tasks into simpler components enhances model performance. (Source: [https://lnkd.in/gKwNTDqn)) 3. **Utilizing Knowledge Databases and Tools:** Grounding a model's responses on knowledge databases or tools reduces hallucinations and enhances factuality. (Source: [https://lnkd.in/gnZ85wa4)) These discoveries have shown substantial performance improvements, rather than just incremental advancements, and have undergone extensive validation. The emergence of Agentic AI is a concrete reality with a lasting impact! This is why we have embraced an agentic Virtual Clinician decentralized framework at The BlueScrubs ( https://thebluescrubs.ai ) to facilitate reliable and trustworthy AI in medicine.

8

1 Comment

New Mount Sinai AI study finds model-driven bias in stress test of nine large language models (LLMs). Despite identical clinical details, AI models altered recommendations based on a patient's socioeconomic and demographic profile. The authors urge for more stringent model validation and refinement of AI tools to ensure they uphold the highest ethical standards and treat all patients fairly. “Socio-Demographic Biases in Medical Decision-Making by Large Language Models: A Large-Scale Multi-Model Analysis.”  April 7, 2025, online issue of Nature Medicine https://lnkd.in/eTe98Kin #AL #LLM #bias #AIassurance #healthequity #mountsinai

23

5 Comments