Bruno Del Papa

Announcing: Agentic Document Extraction! PDF files represent information visually - via layout, charts, graphs, etc. - and are more than just text. Unlike traditional OCR and most PDF-to-text approaches, which focus on extracting the text, an agentic approach lets us break a document down into components and reason about them, resulting in more accurate extraction of the underlying meaning for RAG and other applications. Watch the video for details.

23.490

851 comentários

Here's something to read this weekend! I've just finished reading the early access version of "Domain-Specific Small Language Models" by Guglielmo Iozzia, being published by Manning Publications Co. As someone deeply involved in the AI space, and particularly interested in practical, efficient deployments, I must say this book is a crucial resource for anyone looking to build powerful AI applications on a budget and with control over their data! The book addresses a significant challenge in the industry: the prevailing hype around large, general-purpose language models often masks the viable and cost-effective open-source alternatives. This book provides a highly informative deep dive into the techniques and best practices for creating domain-specific, local models that work on your unique challenges and private data. The book also highlights how to make custom LLMs run inference without the same powerful infrastructure that big tech companies have. Here's a quick breakdown of what this book covers: 📌 Understanding the SLM Advantage: Moving beyond the limitations and risks of closed-source LLMs to leverage the specialised expertise, higher accuracy, and cost-effectiveness of domain-specific models. 📌 Efficient Model Customization: Detailed guidance on data preparation for LLM customization, including fine-tuning and PEFT techniques like LoRA 📌 Optimizing for Inference: Exploring key metrics for inference performance (TTFT, TPOT, latency, throughput) and techniques to get the most from your hardware. 📌 The ONNX Standard for Portability: A comprehensive look at the ONNX format and runtime for seamless model interoperability across diverse frameworks and hardware, and its role in optimisation and quantisation. 📌 Deep Dive into Quantization: Practical examples and in-depth explanations of 8-bit and 4-bit quantization techniques to significantly reduce model size and accelerate inference while preserving accuracy. 📌 Advanced Techniques for Large Models: Introduction to cutting-edge methods like FlexGen for offloading models to CPU/disk and SmoothQuant for handling activation outliers in larger models, ensuring efficient deployment even with limited GPU memory. 📌 Real-World Applications: Hands-on examples of applying these concepts to practical problems, such as generating Python code and creating protein and crystal structures in the chemistry domain. 📌 Profiling and Insights: Tools and methods for profiling ONNX-ported models and transforming raw profiling data into actionable insights for continuous optimisation. The first nine chapters of the book have been released. If you're looking to build and deploy specialised AI models effectively and cost-efficiently, I highly recommend grabbing this book! You can find it here - https://mng.bz/Z9Ya Thank you to Guglielmo Iozzia for writing this essential guide, and I'm looking forward to the rest of the chapters!

62

2 comentários

🚨 Small number of poisoned samples can seriously compromise LLMs! I just read an eye-opening study by Anthropic, UK AISI, and The Alan Turing Institute titled “A Small Number of Samples Can Poison LLMs of Any Size.” Their core finding is astonishing: with as few as ~250 malicious documents, an attacker can successfully insert a “backdoor” into large language models (ranging from 600M to 13B parameters) — causing them to misbehave on specific trigger phrases — regardless of the total training data size or model scale. This marks a major shift from the earlier belief that an attacker must control a proportion of the dataset. Instead, this research shows that it’s the absolute number of poisoned samples that truly matters. The full technical paper — “Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples” — dives deeper into the experiments and implications. 📄 Read here: https://lnkd.in/gje8V-76 🎥 Watch a great explanation: https://lnkd.in/gMk68vj3 🙏 Thanks to Krish Naik for breaking it down so clearly!

34

2 comentários

Great survey on small language models. This 87-page survey closely examines small language models, defined as models sized between the minimum threshold for emergent abilities on specialized tasks and the maximum sustainable under resource constraints. SLMs are really worth keeping a close eye on. They are specifically useful in multi-agent systems.

168

6 comentários

You may have noticed the emergence of Leiden clustering (as opposed to Louvain clustering) a few years ago in single-cell analysis. If you are using Seurat, be careful if you decide to transition... Louvain and Leiden clustering are both community detection algorithms on (in Seurat's particular case) a shared nearest neighbor graph of your data. The Leiden algorithm, in 2019, added additional refinements upon the shoulders of Louvain. In theory, making the switch could improve your clustering schemes. But in practice... When I switched from Seurat's implementation of Louvain to Leiden using the default settings of the classic Guided Clustering Tutorial on the PBMC 3k dataset, across 20 random seeds, Leiden did not detect the platelets that Louvain easily detects. This suggested that the given clustering parameters worked for Louvain, but needed to be re-tweaked for Leiden. The image below is this initial output, with cluster centroids being visualized in a gif across the 20 runs, a visual way I developed to assess cluster stability (hence the jiggling of the yellow dots). Accordingly, when I increased the resolution, Leiden detected the platelets, but the cluster stability got worse in the T cells. Lowering the number of nearest neighbors from the default 20 to 10 also got Leiden to detect the platelets with decent stability, but I also got an extra cluster in the CD4 T compartment, similar to the one that "flickers" in on the left panel below. Note that the extra cluster is not a bug, just the result of that particular setting, which may well be a defensible subset. So there is still a bit of tweaking left to do on my end. And I'd expect you'll have to do the same on your end. In short, if you switch from Louvain to Leiden specifically in Seurat, don't assume that you can use the same default parameters for both. In contrast, Scanpy's clustering is Leiden-centric, so its defaults may work better with Leiden than with Louvain. And more generally speaking, never assume the default parameters are the right ones, for whatever you're doing! Thank you and I hope you all have a great day.

49

1 comentário

Can we build a definitive science of stories? This week, a transdisciplinary group at SFI is asking: What if we treated stories as data? As networks? As science? “Stories are storehouses of knowledge, but they are also instruments of power,” says Peter Dodds, SFI External Professor. “Fictional characters matter because real character matters. By using data to develop a science of stories, we can best identify and account for the power of stories in the real world.” https://lnkd.in/gGfJhj8V

193

14 comentários

Fine-tuning LLMs can help tailor models to specific use cases, but when should you do it? Also, I learnt about merging the model weights using different models (seems counter intuitive but works, some methods here in this notebook: https://lnkd.in/gFT9ctre) This video covers: ► LLM Training Life Cycle – Explanation of the three stages: pre-training, supervised fine-tuning, and preference alignment. ► When to Use Fine-Tuning – Comparison between prompt engineering and fine-tuning, with a decision-making framework. ► Why Enterprises Care About Open Source – Discussion on control and customizability as key factors for enterprises. ► Fine-Tuning Libraries – Overview of libraries like TRL (Hugging Face), Axolotl, and LamFactory. ► Supervised Fine-Tuning (SFT) Process – Explanation of instruction data, system prompts, user prompts, and answer formatting. ► Preference Alignment Techniques – Introduction to methods like DPO, KTO, IPO, and their role in training models. ► How to Create SFT Datasets – Importance of accuracy, diversity, and quality in dataset creation. ► Merging Model Weights – Techniques for combining model weights to improve performance or efficiency. https://lnkd.in/gWgebS8Y

31

After a short break, what a good technical paper to write about and implement. DeepSeek'sOCR takes a more modern, unified approach compared to the older stage-wise OCR setups, and this is where the phrase ‘a picture is worth a thousand words’ holds true in the case of LLMs as well. So here’s my review of DeepSeekOCR and the traditional OCR architectures. Read here: https://lnkd.in/eF7SXWi9

23

AutoGen vs LangGraph vs PydanticAI vs Google ADK vs OpenAI Agents ... Which Multi-Agent Framework should I start with? I often get versions of this question and after quite a bit of reflection, I think that the best thing to do is begin by looking at code - the same concept implemented across multiple frameworks. This process is TIME CONSUMING, so I made an interactive tool to help you review a hello world implementation across 6 (and growing) multi-agent frameworks.  Tool: https://lnkd.in/gUrRYaeE By looking at code, you can get a decent sense (though limited) of the design philosophies of each library. For example, you get low-level control with AutoGen core and LangGraph but write **a lot more code**(2-3x).  The interactive tool also shows some (limited) helpful _developer experience_ metrics like lines of code and run time (only based on a single run) which can be quite insightful (e.g., PydanticAI is snappy, AutoGen AgentChat and LlamaIndex have really good high level API abstractions to help you get started fast etc). Code: https://lnkd.in/ex6Cw5gZ Remember, comparisons are subjective and the examples probably can be further optimized. Use all of this as a starting point for your explorations. Beyond looking at code? The next step is to understand what multiagent frameworks should be doing for you [2][3]. That would be a separate post/discussion[2][3]. References [1] Helloworld Agent in 7  agentic frameworks - AutoGen , Google ADK, LangGraph, LLamaIndex, OpenAI Agents SDK, Pydantic AI, Semantic Kernel Agents  https://lnkd.in/gUrRYaeE |  Code: https://lnkd.in/ex6Cw5gZ  [2]  AutoGen vs CrewAI vs LangGraph vs PydanticAI vs Google ADK vs OpenAI Agents .... Which Multi-Agent Framework is Best? https://lnkd.in/e9_---dX  [3]  10 Ways to Critically Evaluate and Select the Right Multi-Agent Framework https://lnkd.in/eeK_Cs3S   [4] A Friendly Introduction to the AutoGen Framework (v0.4 API) https://lnkd.in/gmihrZZt

457

41 comentários

Many of us have faced last-minute issues while presenting slides. Missing fonts, media not playing, files incompatible with the host system, or simply differences between operating systems. These problems can throw you off in crucial moments like conferences, reviews, or lectures. To address this, I’ve started using Quarto with Reveal.js to create HTML-based presentations and host them on GitHub Pages. This makes my slides accessible from any browser. No need to carry files, no risk of version mismatches, and everything works exactly as intended. I’ve written a short blog post explaining how to set this up, for anyone who wants a smooth, reproducible way to present from anywhere. Blog Post: https://lnkd.in/d4cQuxpw Demo Slides: https://lnkd.in/dG-QqUHj Source Code: https://lnkd.in/d2kmzbrW Disclaimer: Slides hosted this way are publicly viewable, so it’s best suited for content you’re comfortable sharing openly.

39

Exciting new research from Anthropic introduces Internal Coherence Maximization (ICM), a breakthrough method that allows #largelanguagemodels (LLMs) to #finetune themselves using only their own outputs - potentially reducing or even replacing the need for human oversight in complex tasks. The model evaluates the consistency of its own responses and optimizes itself by comparing and correcting inconsistent statements. In benchmarks such as TruthfulQA and GSM8K, ICM achieved similar or better results than models with classic supervised fine-tuning. https://lnkd.in/d3Kbdsts

42

No pretraining. No datasets. Just pure inference-time gradient descent on the target ARC-AGI puzzle itself, solving 20% of the evaluation set. Introducing *ARC‑AGI Without Pretraining* Check out MLD PhD student Isaac Liao's technical blog post here: https://lnkd.in/eY_kvbQj

17