Daniel Han

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and Aymeric Roucher, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here! https://lnkd.in/g2Td5WqD

3,507

131 Comments

Can clinical AI summaries be not just useful... but fun? We built something unusual at Abstractive Health: a free simulation game called Clinical Time Machine. Try for free here -> https://lnkd.in/epeuph9m Why a game? Because less than 1% of doctors have ever seen an actual AI summary of a real patient. Not because they don’t want to - it’s an access problem. Our summaries are really good on real patients - but it takes time to verify new users are doctors. And synthetic data doesn’t cut it. So we thought: what if we used famous historical patients instead? The first case we built is King George III - whose condition was long labeled as "madness." No spoilers if you know what it really was... We tracked down his original handwritten records from 200+ years ago, ran OCR on them, and fed them through our AI summary engine. Then we layered in RAG and agent-based reasoning to build a playable simulation. Doctors can now interact with the case like it’s their own: ask clinical questions, submit a differential, choose treatments, and watch the outcome. It’s built on top of our real product, and early testers said: “This made the board exams fun.” Hence Abstractive decided to build the first clinical AI game with chart summaries 🙂  Shout out to Caroline Reiner for the original idea! #HealthcareAI #ClinicalSimulation #FutureOfMedicine Doctors can diagnose King George III with our AI:

104

8 Comments

📣 Introducing RRE-1: A New Model for Retrieval Evaluation & Reranking for RAG based applications. Retrieval-Augmented Generation (RAG) has massively helped transform LLM-powered applications, but let’s face it—retrieval quality is still the weakest link in many pipelines. Poor retrieval is one of the major contributors of lower accuracy of your LLM applications. 🧐 That’s why we at AIMon built RRE-1, a powerful retrieval evaluation and reranker model designed to optimize ranking quality at scale. 🎯 👉 Why does this matter? Traditional retrieval methods like vector search are powerful, but they have limitations in effectiveness, especially when handling complex queries and domain specific knowledge. RRE-1 addresses this by offering: ✅ Offline evaluation of retrieval effectiveness, helping teams benchmark and refine their pipelines. ✅ Real-time low-latency reranking to ensure the most relevant context reaches the LLM. 👉 Who is this for? If you’re working on LLM agents, enterprise RAG systems, or AI copilots, this is for you. Our model ensures your retrieval pipeline is as optimized for your generative model—because great responses start with great retrieval. 🔎 Want to see it in action? Read more about AIMon RRE-1 in our blog post (link in comments). If you’re tackling retrieval challenges in your AI stack, we would love to chat! #AI #RAG #Retrieval #LLMs #MachineLearning

67

4 Comments

dbt Labs announcement of #dbt (con)Fusion has a lot of things that we as a data community need to discuss. dbt Fusion is a complete rewrite of dbt Core in Rust. Unlike dbt Core, which is completely free and open source under Apache 2.0, dbt Fusion is not open source as it is under the more restrictive Elastic 2.0 license. Although Fusion is free to use, it restricts its usage in a hosted or managed offering to 3rd parties. You may think this is fine, but there are far-reaching implications. Open source is amazing because it incentivizes individuals and companies to invest in it without risk. A company can go all in on open source because no matter what the direction of the core maintainers is, it can be forked and used for whatever reason you want. A non-permissive license like Elastic will disincentivize companies from investing. Don't get me wrong, there's nothing ethically wrong about dbt Labs' decision. It may even be in their best financial interest to do so. However, I want to analyze what led to this situation and what it might mean for the future of dbt Core. What I believe this means is that dbt's strategy is to put dbt Core in "maintenance mode" to focus on Fusion and their other proprietary offerings. The wording of the announcement was very carefully selected to be vague. In particular, when referring to dbt Core support, it was only highlighted that bug fixes, security patches, and compatibility would be ongoing. According to their dbt Core roadmap, they've separated out the dbt language from the runtime. There's a specific callout that Fusion and Core will inevitably diverge because Fusion has additional capabilities that cannot be added to Core. To me, it makes sense that this is the chance for dbt Labs to invest in more restrictive and profitable software while slowly deprecating what not only made them great but is also their biggest challenge to financial growth. Ultimately, resources are finite, and companies must prioritize what makes sense for the business. Given dbt Core's foundational importance to modern data infrastructure, Analytics Engineers deserve a free, open, and continually evolving transformation platform. Otherwise, your career will be dangerously dependent on the decisions of a single company. To safeguard the continuing innovation and development of the transformation space, it may be time to start a discussion about an open standard for defining data transformations.

795

49 Comments

🤔 Could a fine-tuned LLM actually be worse at finding the right answer than the raw base model? The shortcomings of fine-tuning aren't discussed much, but even the most modern methods come with tradeoffs. Here’s what recent research by Yang Yue et al. (2025) reveals: 🔹 Base models often match – or even beat – RL-fine-tuned models on tough tasks when given enough samples (i.e., more generations) due to more effective exploration. 🔹 Even world-class models like DeepSeek AI's R1 can actually become too efficient at sampling paths and end up losing to their base models in their ability to arrive at the correct answer. 🔹 This implies that modern reasoning models are, in fact, hyper-efficient samplers – nothing more. Fine-tuning is more about efficiency than capability expansion. At Flow AI, we’ve spent countless hours fine-tuning models and pushing them to their limits. One thing has become clear: tweaking how a model samples is a high-stakes game. You chase gains, but side effects often emerge silently. Hallucinations, bias, even catastrophic forgetting can surface long after deployment. Have you had any fine-tuning experiments you'd be willing to share? (Link to the paper in the comments) #artificialintelligence #LLMs #machinelearning

49

6 Comments

Nuanced is a structured context layer for AI coding tools: call graphs, symbol maps, and static analysis that make LLMs reliable. We just launched our explainer video: 📽️ https://lnkd.in/eA8wmYeA If you’ve ever banged your head against Claude Code or Cursor, you know the pain: LLMs can write code, but they don’t understand it. They hallucinate helpers, invent imports, and break prod, because they can’t trace real structure across your repo. Nuanced gives them that structure. No more guesswork. Just deterministic edits, grounded in your codebase.

47

6 Comments

Metrics saturation is a real problem when developing a scoring system. Real fun exchange with Achint Srivastava on our Discord on how to build metrics that don't saturate and don't get gamed. Shows how hard it is to build metrics with discerning power and why Pi Labs, Inc. scoring system is such a good fit for purpose. Starts small, but tides you to very complex situations. Pasting in full here, curious to hear thoughts on folks developing benchmarks and metrics on their experience in this space. Q: Hey folks, I just got out of your workshop at the AI world fair. I am really curious on how you handle the poor statistical behavior of scores that are a linear combination of sub-metrics. I.E: Mutual information between metrics, increased noise that mask small improvements in single metrics, and most importantly how linear combinations of metrics can result in favoring saturation of "easy" metrics with exploitable bias instead of general task improvement. A: Great question! Behind the scenes we are using GAMs for combining the dimensions (sub-metrics), instead of simple linear combination — largely for the reasons you stated. That doesn't solve all the problems, but it addresses many of them: Non-linearity with Interpretability Unlike linear models, GAMs let each sub-metric have a non-linear influence on the final score — e.g., diminishing returns for "easy" metrics, or thresholds where a dimension suddenly becomes more important. But because they're additive, you still get interpretability per dimension. Less Susceptible to Saturation Bias Since each dimension's contribution is a smooth function (like a PWL), the model can learn to cap the impact of "easy to game" dimensions, so you don’t end up with a scoring system that blindly rewards maxing out one signal. Learned from Examples Our GAM weights/functions are learned from user-labeled preferences over examples (exposed through the calibration API) — which gives us a way to ground the combined score in actual subjective judgments, rather than arbitrary weights. Easier Score Auditing The shape of each dimension’s contribution can be visualized — so you can audit whether your model is behaving reasonably (e.g., does it overvalue fluency past a certain point?). Handles Mutual Information Better While GAMs don’t explicitly model interaction terms, they still reduce the problem of overweighting correlated dimensions — especially if you regularize or sparsify (prune) them well. We’re also exploring extensions that do handle key interactions (e.g., pairwise GAMs). This is still a work in progress.

40

2 Comments

🤔 How do you actually compare two (or more) LLM outputs? We’ve talked to a lot of teams building with LLMs — and one pain point keeps coming up: “I just want to know if A is better than B… without building a whole scoring system.” In our latest writeup, we explored a practical approach: ⚖️ LLM Arena-as-a-Judge — pairwise evaluations with Elo-style scoring, blind trials, randomized order, and co-prompting. 📚 Read it here: https://lnkd.in/gdrMUCwi A few lessons: ✅ Binary comparisons (“which is better?”) are often more reliable than numeric scores 🎯 You don’t need 5 metrics to test for regressions — one thoughtful comparison can go a long way 🧠 LLMs can serve as surprisingly good judges with the right setup (and some bias guardrails) This is something we’ve been building into DeepEval — not because it's shiny, but because it just works in day-to-day eval workflows. Curious how others are thinking about this: How do you evaluate model updates or prompt tweaks? Manual reviews? Metrics? Or just vibes?

33

7 Comments

💡 Gemini 2.5 Pro achieves impressive success through parallel thinking—but how can we achieve this efficiently? We propose a unified strategy: 🔥Fractured Chain-of-Thought Reasoning! 🔑 Key Insights: ✅ Truncated Chain-of-Thought (CoT) matches or exceeds full CoT performance, using dramatically fewer tokens.  ✅ Introducing Fractured Sampling—a novel inference-time strategy that optimally allocates computational effort across three dimensions: reasoning trajectories, solution diversity, and reasoning depth.  ✅ Demonstrated superior accuracy-cost trade-offs across various challenging reasoning benchmarks. 📈 Our experiments across multiple models consistently show steep log-linear scaling benefits, revolutionizing the way we think about efficiency in reasoning tasks for LLMs. 🔥 Dive deeper and explore our results: https://lnkd.in/gZha98ch  💻 Check out our open-source implementation: https://lnkd.in/g3DegW7R Let's unlock more efficient, scalable reasoning together! 🚀 Huge thanks to the collaborators!! Baohao Liao Hanze Dong Doyen Sahoo Christof Monz Junnan Li Caiming Xiong #LLM #ChainOfThought #AI #Efficiency

66

1 Comment

From dump to FHIR in a prompt 🥹 Below is a minimal HACS snippet that: 1. Converts a free-text prompt into a typed Patient + Observation 2. Stores an episodic memory block ready for future recall 3. Validates everything before it hits storage HACS is now in open beta — the core packages are already on PyPI: `pip install hacs-models hacs-openai hacs-tools`... Colab for the snippet below in comments ;) P.s.: Now integrating with LangChain with hacs-langgraph!

33

8 Comments

LLMs are dead, long live LCMs? LLMs may have hit their limit. If we take a look at LLMs today, we’ve seen evidence from startups that open-source models can achieve performance comparable to their closed-source counterparts and yet the underlying model architecture remains largely unchanged. Why? We're hitting fundamental limits: 1. Data scarcity - high-quality training material is running out. 2. Curation methods - most models use similar human-feedback methods. 3. Computational architecture - everyone relies on the same limited pool of specialized GPUs. The result? Efficiency is overtaking scale. Companies are compressing knowledge into fewer tokens and developing better engineering artifacts leading to a natural plateau where more resources yield diminishing returns. The answer may lie with large concept models (LCMs)–a new architecture that could rival LLMs in natural language understanding. LCMs promise better performance with fewer iterations, making them ideal for leaner teams. Who to pioneer this: I believe this shift towards LCMs will be led by ex-big tech founders launching startups to push these models into the mainstream. The race is on. Do you agree?

71

7 Comments

🚨 OpenAI just released HealthBench — a new gold-standard benchmark for AI in healthcare, co-developed with 262 physicians across 60 countries. HealthBench is what every serious AI builder in vertical domains should be paying attention to. It’s: 🩺 Realistic: 5,000 multi-turn health conversations across languages, specialties, and global contexts 🧠 Expert-driven: 48,000+ physician-authored grading criteria 🔍 Unsaturated: Even frontier models like GPT-4.1 and Claude 3.7 are still struggling with key scenarios 🧪 Evaluated by physicians — and shows AI+doctor combos already outperform humans alone 📊 This benchmark shows why benchmarks matter more than model size in vertical AI. If you’re building AI for healthcare, law, finance, or any high-stakes field, you need serious evaluation methods like this. Not guesses, not hype — just real expert judgment turned into structured, measurable, and reliable tests. OpenAI is raising the bar. If your “AI copilot” can’t pass this test — should it really be in the room with patients? My answer is probably not alone, but it could help a lot of doctors! #ai #health #largelanguagemodels #benchmarks

54

4 Comments

A new open-source multimodal Model LLM has been launched, offering performance comparable to GPT-4o and Gemini 2.0. BAGEL, the open-source Unified Multimodal Model, can be fine-tuned, distilled, and deployed anywhere. It provides functionality similar to proprietary systems like GPT-4o and Gemini 2.0 in an open format. BAGEL unlocks valuable image generation through a natively multimodal architecture, delivering precise, accurate, and photorealistic outputs. For more information, you can visit their website: [BAGEL AI](https://bagel-ai.org/)

46