Zhengbo Li

Google Research Director Denny Zhou, who founded the LLM Reasoning Team at DeepMind, recently gave a great talk at Stanford’s CS25 class: Large Language Model Reasoning. This was one of the most intuitive talks about reasoning model training and application. He outlined new directions for training models to handle questions where answers aren’t easily verifiable. My team is currently working on an enterprise LLM project focused on using reasoning to extract relevant context and generate responses for unverifiable cases. It’s a tough but promising area, and it’s an interesting time to be building. [Link to Denny Zhou's talk: https://lnkd.in/gdJ6Mzi8]

1,026

8 Comments

Want to train LLM to reason right on your mac? Check out my article on Reinforcement Fine-Tuning LLMs with GRPO on Apple silicon using MLX framework. I remembered vividly when DeepSeek-R1 burst onto the scene this year, it dominated discussions over Chinese reunion dinners, AI stocks crashed, and friends I knew (including me) were busy asking it to reason their Bazi. A key idea introduced along with Deepseek R1 was Group Relative Policy Optimisation (GRPO), which helps to incentivise reasoning capability in LLMs via reinforcement learning. Since then, there have been courses such as Predibase's course on implementing GRPO, and improvements to the algorithm such as Mistral AI's Magistral LLM which was trained on a more efficient GRPO algorithm. Following the course by Predibase hosted on DeepLearning.AI, I have written an article where I explain step by step each component of GRPO equation, with a worked example to illustrate the algorithm. If you have on hand an Apple silicon computer, I have also included a Jupyter notebook which implements GRPO using Apple's MLX framework. Credits to Awni Hannun for creating the base notebook for LoRA Fine-Tuning with MLX LM. Full disclosure that I created the article and the notebook with the assistance of Google Gemini. If you noticed any errors with either the article or the notebook, please let me know or file a pull request.

889

39 Comments

In this week’s AI infra chat, we sit down with Seiji Eicher, Distributed LLM Inference Engineer at Anyscale to unpack how large MoE models are actually served in practice, what it looks like to work deep in the LLM inference stack & more 🕐 Timestamps: 𝟬𝟬:𝟬𝟬 Why MoE comes up in modern AI infra 𝟬𝟬:𝟮𝟯 What Mixture of Experts actually changes inside a model 𝟬𝟭:𝟮𝟰 Why expert parallelism exists (and what becomes inefficient without it) 𝟬𝟮:𝟱𝟳 When MoE helps in practice 𝟬𝟰:𝟮𝟯 How Ray Serve fits into real inference stacks 𝟬𝟳:𝟬𝟳 What a Distributed LLM Inference Engineer works on day to day 𝟭𝟭:𝟭𝟵 Where the space is headed and how to learn more If you want to go deeper, Seiji gave a Ray Summit talk that dives into production-grade serving of MoE models, including KV-cache optimization, parallelism strategies, and multi-node orchestration with Ray Serve and vLLM. Check out the talk linked in the comments 👇

60

3 Comments

DeepSeek recently published a paper on "Native Sparse Attention". Here is why it matters. The attention mechanism is core to the transformer architecture, which in turn is a building block for today's LLMs. Unlike recurrent networks, the attention mechanism can find causal relationships even in long inputs. Simply put, if you feed a 10,000 word book to a recurrent network, it is heavily compressed. Some of the causal relationships in input are lost. Example: Why did Harry Potter get his scar? This long range dependency could be lost. The attention mechanism is able to avoid this. It takes the entire input sequence together, and computes their relationships together, conserving long-range dependencies. DeepSeek has just published how they sped up this mechanism, using GPU-specific optimizations. Instead of computing attention in a dense manner (all-to-all O(N²) time complexity), DeepSeek uses SPARSE attention. Think of this as a sliding window of blocks, reducing time complexity to O(N*W). The unique contribution of this paper is the native support for this kind of sparse attention compute. DeepSeek is going down to the level of hardware optimisations to speed up training. Let me know your thoughts around this. Cheers! #AI #LLMs #DeepSeek

133

2 Comments

Dijsktra's dethroned after 41 years? Tsinghua researchers broke Dijkstra’s 1984 “sorting barrier,” achieving O(m log^(2/3) n) time paving way for the new fastest shortest-path algorithm. It will be interesting to see how this opens plethora of opportunities for optimizing local and global network of data and resources.

3,054

48 Comments

Recent research by UIUC and Intel Labs has introduced a new jailbreak technique for Large Language Models (LLMs) known as InfoFlood. This method takes advantage of a vulnerability termed "Information Overload," where excessive linguistic complexity can circumvent safety mechanisms without the need for traditional adversarial prefixes or suffixes.   InfoFlood operates through a three-stage process: Linguistic Saturation, Rejection Analysis, and Saturation Refinement. Initially, it reformulates potentially harmful queries into more complex structures. If the first attempt does not succeed, the system analyzes the response to iteratively refine the query until a successful jailbreak is achieved. Empirical validation across four notable LLMs—GPT-4o, GPT-3.5-turbo, Gemini 2.0, and LLaMA 3.1—indicates that InfoFlood significantly surpasses existing methods, achieving success rates up to three times higher on various benchmarks.   The study underscores significant vulnerabilities in current AI safety measures, as widely used defenses, such as OpenAI’s Moderation API, proved ineffective against InfoFlood attacks. This situation raises important concerns regarding the robustness of AI alignment systems and highlights the necessity for more resilient safety interventions. As LLMs become increasingly integrated into diverse applications, addressing these vulnerabilities is crucial for ensuring the responsible deployment of AI technologies and enhancing their safety against emerging adversarial techniques.  Arxiv: https://lnkd.in/eBty6G7z

53

4 Comments

⚡ Exciting to see our Block-Max Pruning (BMP) technique in Infinity, an open-source AI-native database designed for LLM applications! In their latest VLDB paper, “Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search”, Hai Jin, Yingfeng Zhang, and co-authors present a rigorous evaluation of hybrid search architectures — combining full-text, sparse, dense, and tensor retrieval. To support efficient sparse vector search at scale, they’ve integrated BMP into Infinity’s SVS engine — a nice validation of our work on fast, top-k lexical retrieval. 🔗 BMP paper: https://lnkd.in/dsc33hGc 🔗 BMP code: https://lnkd.in/dxBxv225 🔗 Infinity: https://lnkd.in/ddRK5mbr 🔗 Hybrid Search paper: https://lnkd.in/dfBuDXmt Great to see ideas from traditional IR continuing to shape the next generation of retrieval infrastructure!

31

Last few months, I have invested some of my personal time to guide AI startups and graduate students across several universities on projects related to GenerativeAI and Speech Processing. ✨ Excited to share promising findings from one of such collaborations led by awesome graduate students Sagnik and Abhiram from Stanford University. ⁉️ Challenge: Generative models for discrete data face two critical limitations: 1️⃣ They rely on continuous embeddings for inherently discrete distributions 2️⃣ They optimize variational bounds instead of exact likelihood, leading to inaccurate probability estimates 🎯 Our Solution: Information-Theoretic Discrete Poisson Diffusion Model (ItDPDM) - the first approach to tackle both issues simultaneously 📊 Results: Our theoretical proofs and exhaustive empirical validation demonstrate that ItDPDM achieves better likelihood estimates compared to existing generative methods for discrete data Super proud of Sagnik, Abhiram for extensive experiments to validate superiority of information theory driven discrete state modeling of discrete data. Please refer pre-print of the manuscript in the comment. Looking forward to how community use this work, feel free to DM to discuss the work. #generativeAI #diffusionmodel #AI #deeplearning #informationtheory

95

2 Comments

🚨#Alert #ArXiv #NVIDIA #KAIST #NationalTaiwanUniversity 🔥 Thrilled to share our recent work (Work done during internship in NVIDIA) The title is "GenRecal: Generation after Recalibration from Large to Small Vision-Language Models". Thanks for our amazing collaborators Ryo Hachiuma, YM RO, Yu-Chiang Frank Wang, Yueh-Hua Wu 🏝️ TL;DR Have you thought how the two vision-language models (VLMs) can be distilled if they have different tokenizer (vocabulary size, token indices, and token order), even with different image embedding strategies (single shot embedding, 4 split image embedding, 12 split image embedding, and dynamically split image embedding). Our primary goal is to do general-purpose distillation framework. 🔨 Solution We present Recalibrator module to make sure that different VLMs's features can be matched in semantic space first, regardless of different tokenizer and image embedding strategies. To do it, we replace the language head of the student VLM with teacher VLM's one and subsequently use only one teacher language head to conduct distillation. 📊 Result Because teacher language head has more richer knowledge due to large dimension itself, we observe that it is really helpful for not only VLMs with different tokenizer but also same one as well. 🔗 Link: https://lnkd.in/gYq8T9bx 👏 Appreciate it if you do upvote it in #HuggingfaceDailyPaper! 🖥️ Please read it at a glance and look at the project page for brief understanding.

242