Petar Tsankov

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and Aymeric Roucher, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here! https://lnkd.in/g2Td5WqD

3.507

131 Kommentare

🔥 MobileLLM-R1 paper and code are now available! 🚀 Can small models reason? We believe the answer is yes! 🌟 With fewer than 1B parameters and trained on just 4.2T tokens—only 1/10 of Qwen’s—MobileLLM-R1 challenges the belief that reasoning only appears in huge models trained on massive data, with the results being competitive with or outperforming Qwen3-0.6B on multiple reasoning benchmarks. Shortly after releasing the MobileLLM-R1 model weights (over 10k downloads in just two weeks), we published our technical report: 📖 MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (https://lnkd.in/gRZAg7tn) Along with the paper, we’ve open-sourced the entire training recipe—from pretraining → mid-training → post-training. 💻 Code is available on GitHub: https://lnkd.in/gYeYUiRv We hope this work sparks new exploration into how small models can achieve strong reasoning capabilities by offering a first-hand glimpse into the complex yet fascinating mechanisms behind training reasoning models from scratch. Collaborating with Ernie, Changsheng, Chia-Jung, Wei, Yuandong, et al. And thanks for the support from Raghuraman, Yangyang, Vikas

240

Why are we still training Transformers like it’s 2017? The standard Transformer architecture has been a workhorse for NLP, but at scale, it’s becoming inefficient. Every token gets the same treatment, pushed through identical layers, processed with brute force. No prioritization, no specialization. That’s where Mixture of Experts (MoE) comes in. Instead of running all data through a single dense feedforward network, MoE routes tokens to specialized subnetworks (experts). Only a fraction of them activate per token, reducing computational overhead while maintaining (or even improving) model quality. Why does this matter? - Bigger models, cheaper inference - You get the benefits of massive parameter counts without paying for every parameter on every forward pass. - Task specialization - Different experts can fine-tune on different linguistic, logical, or domain-specific patterns. - Better scalability - When OpenAI, Google, and DeepMind are all using MoE at scale, it’s a sign that traditional Transformers might not cut it for long. Of course, MoE isn’t a free lunch. There are challenges in routing, load balancing, and managing sparse activation. But one thing is clear: the age of monolithic Transformers is fading. Instead of forcing engineers to sift through scripts, dashboards, and Slack threads, Kubiya.ai intelligently routes tasks, suggests solutions, and automates workflows, reducing cognitive load and making operations seamless. Visit Your AI Teammates - https://bit.ly/3XfZxfM

150

4 Kommentare

Metrics saturation is a real problem when developing a scoring system. Real fun exchange with Achint Srivastava on our Discord on how to build metrics that don't saturate and don't get gamed. Shows how hard it is to build metrics with discerning power and why Pi Labs, Inc. scoring system is such a good fit for purpose. Starts small, but tides you to very complex situations. Pasting in full here, curious to hear thoughts on folks developing benchmarks and metrics on their experience in this space. Q: Hey folks, I just got out of your workshop at the AI world fair. I am really curious on how you handle the poor statistical behavior of scores that are a linear combination of sub-metrics. I.E: Mutual information between metrics, increased noise that mask small improvements in single metrics, and most importantly how linear combinations of metrics can result in favoring saturation of "easy" metrics with exploitable bias instead of general task improvement. A: Great question! Behind the scenes we are using GAMs for combining the dimensions (sub-metrics), instead of simple linear combination — largely for the reasons you stated. That doesn't solve all the problems, but it addresses many of them: Non-linearity with Interpretability Unlike linear models, GAMs let each sub-metric have a non-linear influence on the final score — e.g., diminishing returns for "easy" metrics, or thresholds where a dimension suddenly becomes more important. But because they're additive, you still get interpretability per dimension. Less Susceptible to Saturation Bias Since each dimension's contribution is a smooth function (like a PWL), the model can learn to cap the impact of "easy to game" dimensions, so you don’t end up with a scoring system that blindly rewards maxing out one signal. Learned from Examples Our GAM weights/functions are learned from user-labeled preferences over examples (exposed through the calibration API) — which gives us a way to ground the combined score in actual subjective judgments, rather than arbitrary weights. Easier Score Auditing The shape of each dimension’s contribution can be visualized — so you can audit whether your model is behaving reasonably (e.g., does it overvalue fluency past a certain point?). Handles Mutual Information Better While GAMs don’t explicitly model interaction terms, they still reduce the problem of overweighting correlated dimensions — especially if you regularize or sparsify (prune) them well. We’re also exploring extensions that do handle key interactions (e.g., pairwise GAMs). This is still a work in progress.

40

2 Kommentare

The Embabel JVM agent framework now implements a Google A2A server, in addition to its MCP support. Alpha quality, but then A2A itself is early. Agent federation is going to become more and more important, and standardization is essential to unlocking its value. A2A may well be that standard. Start Embabel under the a2a profile for Spring and Docker compose, as described in README. A2A client support coming soon. https://lnkd.in/gZKDAPtr https://lnkd.in/eJCKFVWV #MCP #A2A

109

2 Kommentare

TECHNICAL PERFECTIONISM: THE MOST EXPENSIVE HOBBY IN STARTUPS This team spent 4 months perfecting IPMI card configurations while their competitors made millions with bash scripts. The Perfectionism Timeline: 📅 Month 1: "Optimizing orchestration architecture" 📅 Month 2: "Implementing advanced networking protocols" 📅 Month 3: "Debugging SSL certificate automation" 📅 Month 4: "Finalizing production deployment pipelines" Customer feedback during this period: "Can I rent a GPU by tomorrow?" "Why does provisioning take 30 minutes?" "Do you have a simple pricing page?" The brutal punchline: They built a Formula 1 race car when customers needed a reliable Honda Civic. OpenStack vs. Reality: OpenStack implementation: $400K, 3 months What customers wanted: Basic VMs, predictable pricing, simple dashboard Final article reveals why boring technology beats brilliant architecture every single time. #TechnicalDebt #ProductDevelopment #TechStacks #EngineeringManagement #StartupTech https://lnkd.in/gSCeWwve

315

221 Kommentare

"With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC)" https://lnkd.in/gZU-AkKp https://lnkd.in/g9dqj_WA

173

19 Kommentare

Inference is where trust in AI is won or lost Most people focus on training, but in practice, inference decides whether your model is reliable in production. The recent K2 Vendor Verifier project shows this clearly https://lnkd.in/eAt8ZD94 - Same model, different vendors hence very different results. - Failures in schema, tool calls, or finish reasons directly impact product reliability. At trymirai.com, we’re building our own inference stack for exactly this reason. We don’t just want raw throughput - we want consistency, transparency, and control over how models behave at runtime. That’s how you build AI products people can trust. Don’t just ask what model you’re using. Ask how it’s being run.

28

Nuanced is a structured context layer for AI coding tools: call graphs, symbol maps, and static analysis that make LLMs reliable. We just launched our explainer video: 📽️ https://lnkd.in/eA8wmYeA If you’ve ever banged your head against Claude Code or Cursor, you know the pain: LLMs can write code, but they don’t understand it. They hallucinate helpers, invent imports, and break prod, because they can’t trace real structure across your repo. Nuanced gives them that structure. No more guesswork. Just deterministic edits, grounded in your codebase.

47

6 Kommentare

A few months ago, Mem0 published benchmarking numbers for MemGPT and claimed "SOTA" in memory. The weird part? I have no idea how they even could have run that benchmark, without making significant modifications to MemGPT (they did not respond to our requests for details on how the experiments were run). arXiv is not peer-reviewed, so unfortunately this has led to recent abuse as companies can post any "research" results they want for marketing. We were easily able to beat the numbers in the benchmark with simple filesystem tools - indicating the benchmark isn't particularly meaningful either (details below)

139

9 Kommentare

For DeepEval's latest release ⭐ (v3.4.8), we're releasing a brand new decision-based LLM-as-a-Judge metric for multi-turn LLM evaluation, called "Conversational DAG". The conversational DAG metric is a direct acyclic graph where each node represents an 👩🏻‍⚖️ LLM-as-a-judge, while each edge represents the verdict from these nodes. 🍃 The leaf nodes are either a hardcored score from 0 - 10, OR a custom G-Eval that you can use to evaluate conversations on any custom criteria. This means that if you are building AI conversation agents 💬, you can now deterministically evaluate whether your AI has followed whatever custom conversation flow you have set out for it. Since its beta last week, over 100 users already use it on a daily basis to figure out whether their chatbots are following SOPs that are way to specific for any other evaluation metric to take care of. In fact, an AI leader at a publicly traded pharmaceutical said this latest metric is "the most custom metric custom metrics have to offer". 👇 👀 The diagram below shows how DeepEval's conversational DAG works, docs here: https://lnkd.in/gWVDnu44

27

1 Kommentar

Two big launches from us at Anthropic today: 1. Claude 3.7 Sonnet - our most intelligent model yet. It's the first generally available hybrid reasoning model in the world. It can produce near-instant responses like a normal LLM or it can take some time and think before responding for more challenging questions. It's now available to all users in claude.ai and the Anthropic API. 2. Claude Code - a new agentic coding tool that lives in your command line. You'll get Claude-powered code assistance, file operations, and task execution directly from your terminal. We're opening up limited spots today on a first-come-first-serve basis as part of our research preview. https://lnkd.in/gb5MvsCH

351

6 Kommentare

Kicking off BAML Launch Week: Day 1! Announcing the 𝐁𝐀𝐌𝐋 𝐕𝐒𝐂𝐨𝐝𝐞 𝐋𝐋𝐌 𝐏𝐥𝐚𝐲𝐠𝐫𝐨𝐮𝐧𝐝 2.0 What is an LLM Playground? Think React hot-reload, but for LLMs! Whats in this update? 🧪 Tests for structured outputs 🌓 Dark AND light mode 🔄 Eval History 👀 More visualizations for evals 🎙️ Improved multi-modal support and more! 🔦 Added bonus, a spotlight on BAML devs: Product Hunt and GovEagle Full post below 👇

91

4 Kommentare

When we started building Atlas, we wanted to make it simple to work with maps and spatial data — right from your browser. As more people joined the platform, users would often ask us if they could use Atlas for collecting data from people who don't use Atlas. It was a fair point. If you wanted feedback from a community, or field data from volunteers, you had to juggle Google Forms, spreadsheets, location pins, and manual uploads. It didn’t make sense. So we built something new. Today, the first day our Launch Week, we announce a completely new feature in Atlas: Forms! Create a form inside your Atlas project. Share the link. Done. People can submit from any device. Each entry lands straight on your map. Use it for: ◆ Community feedback ◆ Field reports ◆ Survey responses ◆ Incident logs Available now in every Atlas account. Read more in the comments 👇

57

4 Kommentare