Xiaoyang (Sam) Chen

 Reproduced GPT-2 Small from scratch in PyTorch No pre-trained weights. No high-level abstractions. I implemented and trained a full replica of the GPT-2 Small architecture end-to-end on the TinyStories dataset, with a focus on first-principles understanding of Transformer systems. What this involved - Rebuilding a 12-layer Transformer (12 heads, d_model = 768) from the ground up - Implementing multi-head self-attention with explicit Q/K/V projections + causal masking - Designing a Pre-LayerNorm residual architecture for training stability - Constructing token + positional embedding pipelines - Developing an autoregressive generation loop (sampling, temperature, context windowing) All components — including training loop, checkpointing, and inference — were implemented manually in PyTorch. Results - Trained on ~100k sequences (context length = 128) - Final loss: 1.57 - ~12 hours training on Kaggle P100 GPU - Coherent, structured short narrative generation from raw prompts Why this matters Most projects use Transformers. This project focuses on reproducing and understanding them at the systems level: - Attention as tensor operations (not black-box APIs) - Training stability (normalization, residual flow) - Sequence modeling constraints (causal masking, context limits) - Sampling strategies and their impact on output quality 🔗 What’s next - PPO-based fine-tuning (policy + value heads) - KL-regularized training vs reference models - Reward modeling and alignment pipelines 🔗 GitHub Repository: https://lnkd.in/gr4fd7jY More advanced projects coming soon insha’Allah.

17

8 Comments

𝗧𝗶𝗺𝗲 𝗦𝗲𝗿𝗶𝗲𝘀 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗠𝗔𝗧𝗟𝗔𝗕 — 𝗣𝗮𝗿𝘁 𝟭𝟬: 𝗚𝗟𝗦 𝗮𝗻𝗱 𝗛𝗔𝗖 𝗘𝘀𝘁𝗶𝗺𝗮𝘁𝗼𝗿𝘀 When residuals show heteroscedasticity or autocorrelation, classical OLS stops giving reliable inference. This example introduces two ways to handle nonspherical innovations: 🔹 𝗛𝗔𝗖 𝗲𝘀𝘁𝗶𝗺𝗮𝘁𝗼𝗿𝘀 – Adjust standard errors without changing coefficients 🔹 𝗙𝗲𝗮𝘀𝗶𝗯𝗹𝗲 𝗚𝗟𝗦 – Model the innovation covariance to re‑estimate coefficients and variances Using simulated data, the example demonstrates how nonspherical errors distort OLS variance estimates and how HAC and GLS methods affect: 🔸Standard errors 🔸Significance tests 🔸Confidence intervals It also highlights practical challenges in applied work, including choosing covariance models and tuning nuisance parameters when working with economic time series. 📖 𝗥𝗲𝗮𝗱 𝘁𝗵𝗲 𝗲𝘅𝗮𝗺𝗽𝗹𝗲: Time Series Regression X — Generalized Least Squares and HAC Estimators https://spr.ly/6046B6fU2w #MATLAB #Econometrics #TimeSeriesRegression #GLS #HAC #RobustInference 𝘓𝘰𝘰𝘬𝘪𝘯𝘨 𝘧𝘰𝘳 𝘮𝘰𝘳𝘦 𝘦𝘹𝘢𝘮𝘱𝘭𝘦𝘴? Here are the other parts of the series, which you can view in any order: 🔹 Part 1: Linear Models → https://spr.ly/6047B6fU2b 🔹 Part 2: Collinearity and Estimator Variance → https://spr.ly/6048B6fU2j 🔹 Part 3: Influential Observations → https://spr.ly/6049B6fU2d 🔹 Part 4: Spurious Regression → https://spr.ly/6040B6fU2e 🔹 Part 5: Predictor Selection → https://spr.ly/6041B6fU25 🔹 Part 6: Residual Diagnostics → https://spr.ly/6042B6fU2g 🔹 Part 7: Forecasting → https://spr.ly/6043B6fU29 🔹 Part 8: Lagged Variables and Estimator Bias → https://spr.ly/6044B6fU2i 🔹 Part 9: Lag Order Selection → https://spr.ly/6045B6fU2c

2

CLAUDE attacked by rival AI CODING MODELs After GPT5 release everybody expected what … impact launch of this model (and its open source GPT OSS siblings) will have on dominance of the #Anthropic’s Claude in coding scenarios. And market share lost Antrhopic has, indeed. But it´s coming from SURPRISING RIVALS. Neither #ChatGPT nor #Google (who lost himself) are the winners of this fight. It´s Chinese #Qwen3 Coder who has stolen the biggest share of market, as measured in part of market meeteted by OpenRouter. This data are in no means representative of whole market but give good hint on what model dynamics to watch more closely.

13

1 Comment

Short guide to understand NVIDIA Dynamo for large-scale LLM Deployments! If you worked with deep learning before transformers (pre-2017), you probably worked with ONNX, TorchScript, OpenVino, Tensorflow, or TensorRT. These are engines for common DL models. To serve these engines at scale, NVIDIA released Triton Inference Server as a production-ready framework. In the gen-AI era, engines like vLLM, SGLang, TGI, or TensorRT-LLM have emerged for transformer models. Similarly, NVIDIA has released the Dynamo Inference Framework to support these engines and handle inference in large-scale production deployments. 🗒️ Triton and Dynamo both act as higher-level, efficient distributed inference systems with support for multiple inference engines. 🔹 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗡𝗩𝗜𝗗𝗜𝗔 𝗗𝘆𝗻𝗮𝗺𝗼? ↳ 𝗗𝘆𝗻𝗮𝗺𝗼 is an AI inference-serving framework designed to maximize token revenue generation. ↳ It orchestrates and accelerates inference communication across thousands of GPUs. ↳ Features disaggregated serving, a concept introduced in the 2024 DistServe paper. TL;DR DistServe = the prefill and decode phases of the LLM Inference request are separated and parallelized across GPUs. ↳ Dynamo is built in 𝗥𝘂𝘀𝘁 (𝟱𝟱%), 𝗚𝗼 (𝟯𝟬%), and 𝗣𝘆𝘁𝗵𝗼𝗻 for flexibility and customizability. ↳ Dynamo builds on components, which are robust, modular, and independently scalable. 🔹 𝗪𝗵𝗮𝘁 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀? ↻ 𝗧𝗵𝗲 𝗚𝗣𝗨 𝗣𝗹𝗮𝗻𝗻𝗲r monitors the global workload of the entire distributed cluster and can dynamically add and remove GPUs to adapt workloads. ↻ 𝗧𝗵𝗲 𝗦𝗺𝗮𝗿𝘁 𝗥𝗼𝘂𝘁𝗲𝗿 takes care of routing specific requests to the appropriate set of workers that have the best KV Cache hit match. ↻ 𝗧𝗵𝗲 𝗗𝗶𝘀𝗮𝗴𝗴𝗿𝗲𝗴𝗮𝘁𝗲𝗱 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 module is where prefill and decode workers are distributed, handling the workload and communicating through 𝗡𝗜𝗫𝗟 (NVIDIA Inference Transfer Library). ↻ 𝗡𝗜𝗫𝗟 is a low-latency, multi-node communication library for inference payloads. It can exchange tensors, objects, and bytes across a large distributed cluster using RDMA. ↻ 𝗧𝗵𝗲 𝗘𝘃𝗲𝗻𝘁 𝗣𝗹𝗮𝗻𝗲 monitors the cluster and identifies workers' load, then it notifies the GPU planner if more GPU resources are required. ↻ 𝗠𝗲𝗺𝗼𝗿𝘆 𝗠𝗮𝗻𝗮𝗴𝗲𝗿 takes care of KV Cache offloading using a multi-tier memory system. This is similar to RadixAttention from SGLang. ↻ 𝗔𝗣𝗜 𝗦𝗲𝗿𝘃𝗲𝗿 is the frontend interface compatible with multiple APIs (OpenAI, Llama Stack, etc). 📌 I published an in-depth article, going through the Dynamo components, find it in the comments! ---- ♻️ Share this with your network. Follow me to learn more about AI Engineering and production-ready AI Systems! ----

121

12 Comments

[AI paper] GRPO has a hidden flaw in Multi-Reward settings. It’s time to decouple your normalization. 📉 As AI Architects, we are moving from single-objective RL (just "get the answer right") to multi-objective RL (accuracy + format + length + safety). We typically sum these rewards and throw them into GRPO. A new paper from NVIDIA, "GDPO: Group reward-Decoupled Normalization Policy Optimization," demonstrates that this naive summation causes Reward Signal Collapse. The Architectural Failure Mode: If you sum disparate rewards (e.g., a binary Format reward and a scalar Accuracy reward) and then normalize, different raw reward combinations can map to identical advantage values. Example: A rollout with rewards (0, 2) might yield the same normalized advantage as (0, 1) due to group statistics, effectively deleting the gradient signal for the second objective . The Solution: GDPO The fix is architectural: Decoupled Normalization. Instead of normalizing the sum, GDPO normalizes each reward objective independently within the group before aggregation. Impact: This restores the resolution of the training signal. On Tool Calling and Math Reasoning tasks (DeepSeek-R1/Qwen), GDPO converges where GRPO fails or plateaus. If you are training agents that must balance strict formatting (JSON) with reasoning quality, this is the correct objective function to use. 👇 Link to the paper in the comments. #AIArchitecture #RLHF #NVIDIA #GRPO #DeepLearning #LLM #Alignment #Research

21

1 Comment

Yann LeCun just said the quiet part out loud. After 12 years at Meta, the Turing Award winner and co-inventor of convolutional neural networks left to start a new company focused on world models — not LLMs. His message was blunt: Scaling language models will not get us to human-level AI. Why? • LLMs mostly memorize and regurgitate text • They don’t understand physics, causality, or the real world • Tokenizing reality is a dead end • True intelligence requires predicting consequences + planning actions LeCun argues that intelligence isn’t about generating better sentences — it’s about building internal models of the world that can simulate outcomes and choose actions safely. “We will absolutely never reach human-level AI by training on text alone.” Some key takeaways that stuck with me: • Video and sensory data matter more than text • A 4-year-old sees more “data” than the entire internet’s text • World models don’t recreate reality — they abstract what matters • Safety should be built into the architecture, not patched with prompts • The hardest leap isn’t human-level AI — it’s dog-level intelligence He’s optimistic — but allergic to hype. No AGI timelines. No “next year” predictions. Just engineering, abstraction, and humility. In a valley chasing the same LLM playbook, this feels like a necessary reset. Progress doesn’t always come from scaling harder. Sometimes it comes from stepping sideways. #AI #ArtificialIntelligence #MachineLearning #WorldModels #DeepLearning #TechnologyLeadership #FutureOfAI

5

If you’re an AI engineer working on fine-tuning LLMs for multi-domain tasks, you need to understand RLVR. One of the biggest challenges with LLMs today isn’t just performance in a single domain, it’s generalization across domains. Most reward models tend to overfit. They learn patterns, not reasoning. And that’s where things break when you switch context. That’s why this new technique, RLVR with Cross-Domain Verifier, caught my eye. It builds on Microsoft’s recent work, and it’s one of the cleanest approaches I’ve seen for domain-agnostic reasoning. Here’s how it works, step by step 👇 ➡️ First, you train a base model with RLVR, using a dataset of reasoning samples (x, a), and a teacher grader to help verify whether the answers are logically valid. This step builds a verifier model that understands reasoning quality within a specific domain. ➡️ Then, you use that verifier to evaluate exploration data - which includes the input, the model’s reasoning steps, and a final conclusion. These scores become the basis for training a reward model that focuses on reasoning quality, not just surface-level output. The key here is that this reward model becomes robust across domains. ➡️ Finally, you take a new reasoning dataset and train your final policy using both the reward model and RLVR again - this time guiding the model not just on task completion, but on step-wise logic that holds up across use cases. 💡 The result is a model that isn’t just trained to guess the answer, it’s trained to reason through it. That’s a game-changer for use cases like multi-hop QA, agentic workflows, and any system that needs consistent logic across varied tasks. ⚠️ Most traditional pipelines confuse fluency with correctness. RLVR fixes that by explicitly verifying each reasoning path. 🔁 Most reward models get brittle across domains. This one learns from the logic itself. 〰️〰️〰️〰️ ♻️ Share this with your network 🔔 Follow me (Aishwarya Srinivasan) for more data & AI insights

257

69 Comments

🚀 Fine-Tuning LLMs: A Quick Guide Fine-tuning allows us to adapt large language models (LLMs) for specific tasks by updating parameters with task-specific data. Here’s a breakdown: 🔹 Full Fine-Tuning: ⚡ Adjusts all parameters → best adaptation but requires huge compute & memory. 🔹 Parameter-Efficient Fine-Tuning (PEFT): ⚙️ Updates specific layers only → efficient, cost-friendly, and scalable. ✨ Popular PEFT Techniques: - LoRA (Low-Rank Adaptation) ➝ trains low-rank matrices while freezing the base model. - ▪️ LoRA+: Decouples learning rates → improved efficiency. - ▪️ QLoRA: Combines LoRA + quantization → 4-bit models with lower memory use. - Adapters ➝ small trainable modules in intermediate layers. - Prompt Tuning ➝ optimizes prompts, no weight changes. - Prefix Tuning ➝ adds trainable prefix vectors to guide responses. 📉 Efficiency Boosters: - Knowledge Distillation ➝ train a smaller student model to mimic a large teacher. - Weight Pruning ➝ remove unimportant weights. - Quantization ➝ lower precision (e.g., 8-bit) for speed & efficiency. 📑 Instruction Fine-Tuning: - Trains models on structured prompts → improves instruction-following, domain adaptation, and reasoning. ⚖️ Prompting vs Fine-Tuning: - Prompting is quick but limited. - Fine-tuning offers deeper, more consistent improvements. - Together → powerful synergy for robust performance. ⛔ Why Full Fine-Tuning is Impractical for LLMs - 💸 High cost - 🖥️ Memory intensive - ⏳ Time-consuming 🤔 Often unnecessary since pre-training already captures general-purpose knowledge 👉 Instead, PEFT + smart prompting = best of both worlds. #AI #MachineLearning #LLM #GenerativeAI #FineTuning #LoRA #QLoRA #ArtificialIntelligence Sam Ghosh

1

🎉 Excited to share that our work has been accepted to NeurIPS 2025! Generative modeling of discrete data remains a key challenge; most existing approaches either rely on continuous relaxations or approximate likelihoods. ❗️In this work, we show how a Poisson diffusion formulation combined with an information-theoretic loss can jointly achieve fully discrete latent modeling and exact likelihood training, something missing in typical Gaussian-based generative models. ✅ Our theoretical analysis guarantees tighter convergence bounds and improved training, which is validated by experiments on several synthetic datasets. 📊 Finally, we demonstrate the real-world impact of this approach through improved image and audio generation quality. Super proud of Sagnik and Abhiram for all hard work. 🚀 #generativemodels #ai #nips #diffusion #ml #research

60

1 Comment

I have always believed that being a serious AI practitioner means going well beyond simply using models at the surface level. For me, it has never been enough to call an API and accept the output at face value. I am consistently interested in understanding how LLMs work under the hood, how they form and refine representations, how they arrive at decisions, and how we can inspect, challenge, and govern those decisions with greater rigor. It is one thing to call an API and get an answer back. It is another to understand the mechanics underneath: how tokens are represented, how attention moves information, how residual connections accumulate signal, how outputs emerge layer by layer, and how we can audit model behavior in settings where trust and accountability matter. That is the motivation behind the following three hands-on labs I recently built: 1) Build LLM from Scratch Lab https://lnkd.in/dGxEMCDg In this lab, I built an LLM from scratch on my local machine, step by step, to understand the architecture from the inside out rather than treating it as a black box. It walks through the foundations in a very hands-on way, including tokenization, embeddings, self-attention, residual connections, MLP blocks, logits, training, inference, and next-token prediction. I built it around a principle I care deeply about: if we cannot build a system ourselves from first principles, we do not yet fully understand it. 2) LLM Interpretability Lab: Forensic Audit of a Suspicious Loan Denial https://lnkd.in/djr47ewm This lab is framed as a forensic audit of an LLM decision. It investigates how a loan decision forms inside the model, where that decision shifts across the network, and what safer real-world controls could look like in a high-stakes setting. This kind of work matters because AI safety, trustworthiness, explainability, and auditability become especially important when models are used in regulated environments such as financial services, insurance, and healthcare, where decisions need to be understood, challenged, and governed with rigor. 3) LLM Mechanics Lab https://lnkd.in/dz8ftTsS This repo focuses on opening up the model and tracing, as concretely as possible, how an answer is formed internally. It follows how candidate answers rise and fall across layers, how information accumulates in the residual stream, and how we can test causal hypotheses about which internal components actually influenced the final output. For me, this is where AI engineering becomes especially interesting, because it moves us beyond simply using models toward truly understanding, debugging, and governing them. These projects are part of a broader effort to strengthen the bridge between deep technical understanding and responsible real-world AI application. I believe the future belongs to practitioners who can do both: build advanced AI systems, and also explain, stress-test, and govern them responsibly.

8

w/ TurboQuant & long-context LLMs, how real to use kv cache as a document store today? to have vectorless, RAG-less search. i took qwen3.5-35b-a3b and prefilled 258K/262K context window on L4 (a budget GPU popular in prod), means ~99% of the slot is pre-computed and used as "storage", users load it on the fly in ~1s. system prompt + query append to the end, generation takes ~3K tokens, enough for search. at 99% fill rate, decoding runs ~20 tps on L4. i prepared some ego datasets (jina papers, which i know best), plus popular novels in chinese and english. the results are actually pretty good. some hallucination, but most answers are solid and well-grounded. what's more interesting is the cost: ~$0.26/h on L4 spot. single LLM. no vector database, no embedding model, no workflow/pipeline engineering. later I even made qwen3.5-35b-a3b running at 1M context on a single L4 24GB, with 905K tokens prefilled (86% full). decoding at 9 tok/s. stuffed the entire 天龙八部 (a 1.2M-word jin yong novel, 50 chapters) into KV cache as a test. good news: the infra works. generates readable answers even at this context length. bad news: hallucinates badly. it doesn't attend to anything in the middle - only the first and last few chapters. classic lost-in-the-middle at 905K depth. using kv cache as document store is nothing new, like the old CAG paper. but with quantized kv cache and modern attention (hybrid SSM-attention, GQA, MQA, MLA), the economics are changing fast. if we solve cold-prefill speed and decoding speed, and budget GPU costs keep dropping, the future of search could be vectorless. radical, but possible.

42

7 Comments

🚀 Try out one of the fastest forecasting models in the market (15–50× faster than SOTA) — TTM-R3 and TTM-R3-Lite, with state-of-the-art accuracy 📈⚡

16

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,672

190 Comments

black-forest-labs/FLUX.2-klein-9B is crazy fun! Under FLUX Non-Commercial License, it bring: - Real-time AI agents with visual reasoning. - Instant, iterative design workflows. - Low-latency production pipelines on consumer-grade hardware.  - Sub-second speed - Text-to-Image + Native Editing in one pipeline. - Fits in ~20GB VRAM (RTX 4090 ready). The 4-step distilled version is a game-changer for anyone looking to deploy high-quality vision models without the massive VRAM overhead. #GenerativeAI #MachineLearning #FLUX2 #OpenSource #AIEngineering

6

3 Comments

LIGHTSPEED LITE is a from-scratch CPU inference engine for running large language models on commodity hardware. Written entirely in C++17 with hand-tuned AVX2 kernels. No PyTorch, no ONNX, no frameworks. Just raw SIMD intrinsics and careful memory management. Today's milestone. V4 ships six integrated optimizations: → Native Q4_K superblock format that reads 23% less data from memory per token. Same precision, less bandwidth. The model file is smaller AND faster. → Work-stealing thread dispatch that naturally load-balances across fast and slow CPU cores. No more P-cores sitting idle waiting for E-cores to finish. → Batched GEMM kernels that process entire prompts through weight matrices in one pass instead of token by token. This unblocks speculative decoding, the biggest remaining multiplier. → Cosine-based early exit that skips layers when the hidden state stops changing. Easy tokens use fewer layers. Hard tokens use all of them. → A per-region profiler that shows exactly where every microsecond goes. Every claim is measured. Every dead end is documented. When a projected "5 to 10% speedup" measures at 0.01%, it gets retracted and the receipt is kept. That's the No-Vapor Protocol. If it's not measured, it doesn't count. Building in public at Marley Logic AI. More to come. #cpp #llm #inference #machinelearning #ai #buildinpublic #engineering #edgeai #quantization #cpuinference #simd #avx2 #opensource #deeplearning #mlops #startups #tech #software #performance

30

DeepSeek just dropped its new OCR model — but it’s not really about OCR. It’s a proof-of-concept that could redefine how LLMs remember and forget. 🧠 DeepSeek-OCR might’ve cracked the code to catastrophic forgetting DeepSeek-OCR: Contexts Optical Compression isn’t just reading text from images — it’s a vision–text compression experiment showing how visual encoding could manage context length and memory efficiency in LLMs. Think of it like teaching an LLM to forget gracefully: as conversations grow longer, older context gets blurred and compressed — not erased, just down-res’d, like the brain gradually deprioritizing details. 👀 The VLM bottleneck Most Vision-Language Models (VLMs) struggle with the trade-off between resolution, memory, and token length. Whether they use dual encoders, tiling, or adaptive resolution, the outcome is the same — high-res images explode token counts, overload GPUs, and slow everything down. DeepSeek wanted to break this wall. 💡 Core idea — “An image is worth a thousand words.” 📸➡️📝 The question that started it all: For a 1,000-word document, how many vision tokens do you really need to decode it? If a single image can encode the same information with fewer tokens, we unlock optical context compression — a new way to scale LLM memory without scaling cost. 🧩 DeepSeek’s innovation: The DeepEncoder At the heart of it lies the DeepEncoder — designed for aggressive yet intelligent compression. 1. Local perception: SAM-base (80M) captures fine details via window attention. 2. 16× convolutional compressor: Shrinks 4096 → 256 tokens before global attention. 3. Global knowledge: CLIP-large (300M) extracts global semantics. 4. That’s a 16× token reduction without catastrophic information loss — like squeezing a full page of text into a postcard and still reading every word. 🔬 The OCR proof-of-concept To validate it, DeepSeek paired DeepEncoder + DeepSeek-3B-MoE and tested on OCR — the perfect task to measure visual → text fidelity. Results that turned heads: 1. ~97 % precision at 10× compression. 2. ~60 % accuracy even at 20× compression. 3. Outperformed SOTA on OmniDocBench using fewer tokens — with just 100 tokens, it beats GOT-OCR 2.0 (256); in “Gundam mode” (< 800 tokens), it surpasses MinerU 2.0, which uses 6,000 + tokens/page. 🌍 What DeepSeek-OCR can do 1. Deep parsing: charts, chemical formulas → SMILES, geometric figures, and dense image captions. 2. Multilingual OCR: nearly 100 languages, including minority scripts. 🧭 Why this matters DeepSeek-OCR doesn’t just push OCR forward — it reimagines how LLMs might handle memory itself. By treating vision as a compression layer, it hints at a future of LLMs that visually “forget” — not because they can’t remember, but because they choose to. 🔗 Hugging Face model card: https://lnkd.in/g4CJVmES 🔗 GitHub Repo + Paper: https://lnkd.in/gUmUxCXN

16

1 Comment

DeepSeek’s new paper “Conditional Memory via Scalable Lookup” (Engram) — a compelling idea: scale memory (what the model can store) separately from compute (what the model has to process each token). Health conversations are pattern-heavy (med names, labs, instructions). Memory-efficient architectures can reduce cost/latency and preserve compute for reasoning. It also follows DeepSeek’s recent mHC (Manifold-Constrained Hyper-Connections) work, underscoring the rapid acceleration of China’s research capacity. 📄 Conditional Memory via Scalable Lookup paper: https://lnkd.in/dast2zsA #AI #LLM #MachineLearning #NLP 

2