Prachi Bhavsar

Can you build a functional GPT in 111 lines of code? Turns out, yes. Using PyTorch, I managed to fit a full decoder-only Transformer [attention, residual blocks, and a training loop] into about 110 lines of Python. It handles the full stack: multi-head attention, residual blocks, training, and generation. No external weights or pre-training, a raw implementation that I’m currently training on data to see how quickly it picks up the character patterns. seeing the architecture distilled down to its core is the best way to actually understand how these models work. CODE: https://lnkd.in/dK-S2RUD obv inspired by the legend Karpathy #opensource #gpt #ai

5

I finished reading the paper on MemGPT. I really like the concept of treating the context window as RAM. Many analogies are drawn from operating system concepts such as virtual memory, paging, and I/O control. This year, Andrej Karpathy described LLMs as operating systems with different sets of components. The paper explains how we can create an infinite context window by managing context effectively in real time giving the illusion of infinite context. The concept is inspired by virtual memory. Just like how an OS uses external storage such as a disk to manage multiple processes through paging, MemGPT uses similar principles for managing context. The more accurate the context, the higher the quality of responses. Context engineering has become far more important today. https://lnkd.in/dqjUb3df

16

At this point, the writing is on the wall for Indian IT companies. Yesterday, two seismic moves reshaped enterprise AI: OpenAI closed a $10B JV (internally “DeployCo”) with TPG, Brookfield, Advent, Bain, SoftBank and others to embed its models directly into PE-backed companies. Anthropic followed with a $1.5B JV alongside Blackstone, Hellman & Friedman, Goldman Sachs and more to build custom Claude deployments inside enterprises. These aren’t just funding rounds. They’re vertical integration plays. AI labs now own the services layer that Indian IT giants (TCS, Infosys, Cognizant, etc.) have dominated for decades. The message is clear: the old outsourcing model is under existential pressure. Speed, trust, and proprietary model access are becoming the new competitive moat. For Indians in tech, whether in the US or in India, this is a wake-up call. We’ve built incredible individual success stories. Now we must build collective systems: diaspora capital vehicles, sovereign AI infrastructure, open standards, and talent networks that empower the group without sliding into exclusionary bias. Economic nationalism is rising everywhere. The question is whether we respond with open, merit-driven collaboration, or let in-group preferences harden along lines we’ve long tried to transcend. The choice we make in the next 24–36 months will shape the next decade for Indian talent globally. What do you think? Do we have the systems or initiatives in place to navigate the closing of the international space and markets? My thoughts: Bharat must develop self-reliance, and a closed economic system that can sustain itself if the need ever arises. You can't become an autarky, and it's not recommended either for long-term health of the system, but every effort must be made to be as aatmanirbhar as technology allows, and as fast as we can, given the newer geopolitical and AI (Mythos) threats. To give you an example, a private and closed American AI model deleted a Chinese open-source model from my list of providers and disabled it in the code without asking my permission from Dirgha CLI. And then it lied about it when confronted. You can't rely on closed or foreign models to build critical vital infrastructure or enterprise apps. We must build our own open foundations, open source stack if we want to accelerate trust, adoption and security. At Dirgha, I believe we need to help companies deploy these newer, more powerful, and hence more dangerous technologies, without compromising or taking away founder autonomy and ownership.

2

Excited to share that we've had three papers accepted at #EMNLP2025! At Amazon Nova Inference Science team, we have been working on making large language models run faster and more efficiently. Our accepted works: 📝 "Model-free Speculative Decoding for Test-Time Scaling" (https://lnkd.in/g9bmgZRM) - A simple model-free drafter based on n-grams can outperform sophisticated approaches like Eagle-2 for accelerating multi-step reasoning 📝 "Mamba Drafters" (https://lnkd.in/gA9rGBc9) - State-space models as efficient drafters for speculative decoding, specially for longer context 📝 "Think Clearly: Redundant Token Pruning for Reasoning" (https://lnkd.in/gxBtvQhd) - Training-free method to remove redundant tokens from long reasoning chains Thankful to my amazing collaborators Woomin Song, Daewon Choi, Jinwoo Shin, Aram Galstyan, Sravan Bodapati (hiring), Amazon Science and others.

137

4 Comments

CoDA-1.7B by Salesforce is trained for code editing✍️tasks while general coding performances on par with 7B models. The cool part is it does parallel decoding so it’s blazingly fast⚡️during inference! Try it out and let us know your thoughts and questions!

2

Small language models are quietly becoming an important part of real AI systems. While building agentic workflows, I’ve found that many steps don’t need a frontier model at all. Smaller models are often faster, cheaper, and easier to deploy. I wrote a short overview of the current SLM landscape. Part 1 of a three-part series. #AI #LLM #SLM https://lnkd.in/g3Vqqd2T

14

1 Comment

For two decades, I have been wanting to read and understand “Primes is in P” paper (testing if an integer is prime can be done in polynomial time in the number of input bits) by Agrawal, Kayal, and Saxena, and finally, I managed to finish doing it. Let me share my understanding. The basic idea is that they wanted a test for primality that does not rely on randomness or unproven assumptions. Many earlier tests like Miller–Rabin were fast but still probabilistic. Their insight was to connect primality with a statement about polynomial identities. Specifically, they showed that an integer “n” is prime if and only if a certain congruence involving polynomials holds, namely that (x + a)^n is congruent to x^n + a modulo n for many small values of “a.” At first this looks like just a cute algebra trick, but it becomes the cornerstone of the whole algorithm. What they really did was take Fermat’s Little Theorem, which checks “n” by raising numbers to the “n" power, and extend it into the realm of polynomials, where structure is much richer and easier to test deterministically. The clever part is how they turned this characterization into something efficient. Just because a condition is equivalent to primality does not mean you can check it quickly. If you naïvely expanded (x + a)^n, the number of terms would explode. So they introduced a modulus not just by “n”, but also by a carefully chosen polynomial of low degree. This cut down the explosion and allowed the congruence to be checked within polynomial time. The trick is to pick the degree “r" of the polynomial modulus in such a way that it balances correctness and efficiency. They proved that such an “r” can also be found in polynomial time, and then the verification of the congruence for a limited range of a becomes manageable. The condition is that for all integers “a” up to a certain bound, the congruence still holds. This combination of reducing the polynomial and bounding the values of “a” makes the whole test run in deterministic polynomial time, which is the breakthrough. They showed that if “n” is prime, then the congruence always holds, which is the easier direction. The harder part is showing that if “n" is composite, then the congruence must fail for at least one small “a”, provided you have chosen “r" correctly. That proof mixes number theory and algebra, including the use of properties of “cyclotomic polynomials” and multiplicative orders. They essentially proved that if “n” is composite, then the polynomial identity cannot hide the factors forever. Once you see how they cleverly pieced it together, it feels like they pulled a very natural thread: they started from Fermat, lifted it to polynomials, contained the growth by reducing modulo another polynomial, and then proved that this containment cannot be faked by a composite number. The end result is a clean deterministic test that runs in polynomial time and settles one of the oldest open problems in computational number theory.

62

3 Comments

The most ironic part of all this is, the subject they deem only theoretical often define our real world. AI functioning on matrices and theorems? That's not something which you can build or realise. It's a concept, a method, a thought process. And that very thought process now impacts the whole world. Pure Science, Mechanical Engineering, Aeronautics, Food Technology, Philosophy, Psychology, all these supposed niche fields require huge brain power, intellectual capability and creativity. The applied fields and concepts are nothing without their theoretical part. We must remember that.

6

We just published at the ES-Reasoning workshop at ICLR 2026! Me, Yahia Alqurnawi, Preetom Kumar Biswas and Tejas Anvekar worked on ViTaB-A to better understand the structured data citation capabilities of current mLLMs. We kind of realized early that QA and Citation are two very separate tasks and models aren’t very performant or optimized for the latter. Here are some other results we concluded in the paper: 1. Surprisingly, citations in images are easier than MD and JSON. This isn’t the case for QA though. 2. Citing rows is easier than citing columns. 3. Confidence scores (internal or verbal) are not a good verification mechanism for how the model might have performed. Also, a huge thank you to Dr. Chitta Baral and Dr. Vivek Gupta for all of their support and guidance on this. If you’re interested in reading more, here’s the preprint: https://lnkd.in/gmTDV__5 We’ll keep updating the paper with any new insights or numbers we find. School of Computing and Augmented Intelligence #research #interpretability #nlp

36

10 Comments

On-policy distillation matches RL performance at 2-10% of compute cost. RL gives sparse feedback and burns compute. Off-policy distillation is efficient but learns in the teacher's states, not the student's, causing compounding errors on long sequences. Thinking Machines Lab combines on-policy sampling (student generates rollouts) with dense per-token teacher supervision via reverse KL. On math reasoning (AIME), they match RL accuracy with 9-30x less compute. On personalization tasks, they restore instruction-following after domain adaptation without expensive RL retraining. My take: - On-Policy distillation is a great way to recover behavior lost during domain adaptation - The RL-as-strategy-search vs pretraining-as-parameter-search framing is intuitive. Once RL finds a good strategy, distillation shortcuts the exploration by directly teaching the final policy. But this means on-policy distillation won't discover genuinely novel behaviors, it can only refine existing ones. - You need a reliable teacher with logprob access and the student's initial policy can't be too far from the teacher's support (hence the off-policy SFT warmstart in their experiments).

27

3 Comments

Coding benchmarks shape how we evaluate LLMs' coding performance. Some open-source models have been topping SWE-bench with ~80% scores. On paper, they look nearly tied with the strongest closed models. But when evaluated on SWE-rebench, a fresh, decontaminated benchmark built from recent GitHub issues, performance drops sharply. And the leaderboard reshuffles. The difference? Freshness. Once a benchmark is public long enough, models optimize around it. Scores go up. But that doesn’t always mean generalization improves. In my latest blog on To Data & Beyond, I break down: - Why does benchmark saturation happen? - What SWE-rebench changes? - Which models actually generalize best on new tasks? - And what does this mean if you’re building real coding agents? If you’re choosing models based solely on leaderboard scores, this is worth reading. You can read it from the link in the comments!

11

1 Comment

🔎 How Glean Turns Google Docs into Instant Answers (with Google Cloud AI) 🚀 Ever waste time digging through endless Google Docs to find that one piece of information? That’s exactly the problem Glean set out to solve — and here’s how they do it. 🔍 Indexing Google Docs with context Glean connects to Google Workspace (and all your company’s apps) to index content while respecting permissions — so you only see what you’re allowed to. But it doesn’t stop there. It enriches documents with context like: • Who you collaborate with • What projects you’re working on • The acronyms and language your team uses This transforms search results from generic to deeply personalized. ⚙️ Powered by Google Cloud To make this possible at scale, Glean relies on Google Cloud’s data stack: • Cloud Dataflow parses Google Docs and other sources, then adds relevance signals before storing them in the index. • BigQuery tracks user interactions and measures satisfaction, helping fine-tune search relevance. • Vertex AI trains domain-specific language models, enabling semantic understanding of Docs (not just keyword matching). 💡 The result? Instead of spending minutes searching through folders, employees get instant, context-aware answers — right inside their flow of work. This is a great example of how enterprise search is evolving: from simple indexing → to contextual, AI-driven discovery. 👉 Curious about the full story? Check out Glean + Google Cloud’s collaboration here: https://lnkd.in/g96BGf8H

2

I'm going to tell you the biggest lie in AI/ML. It’s the idea that you need a PhD to get started. For years, I've watched brilliant coders—people who can build amazing things—get stopped cold. You pick up a book or start a course and get drowned in abstract calculus before you can write a single meaningful line of code. It’s a wall. And it feels like it was designed for academics, not for us—the builders. After leading AI advocacy and teams at Google, I knew there had to be a better way. We learn best by *doing*. By writing code, seeing it work, breaking it, and fixing it. I believe the future of AI will be built by developers like you, and you shouldn't need to go back to university to be a part of it. So, I wrote the book I wish I had when I was starting out: "AI and ML for Coders in PyTorch." I designed it specifically for you, the software developer: → It’s a zero-math-first approach. We get to the code, fast. → It’s hands-on projects from the very first chapter. → You’ll learn core concepts like Computer Vision and NLP by actually building them. This isn't a textbook; it's a handbook for shipping AI. My goal is to take you from "AI seems complicated" to "I just trained a custom model with my own data." To get you from being intimidated by the jargon to truly understanding the code behind it. Stop feeling like you're on the outside looking in. It’s time to build. This is your bridge from knowing code to knowing AI. Your future self will thank you. You can see the full curriculum and grab your copy at the link below. 👇 https://amzn.to/4508E9H #AI #MachineLearning #PyTorch #Python #SoftwareDevelopment #Coding #Developer #Google #Tech

948

85 Comments

𝗔𝗜 𝗟𝗲𝗮𝗱𝗲𝗿𝘀𝗵𝗶𝗽 𝗥𝗲𝗾𝘂𝗶𝗿𝗲𝘀 𝗜𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲, 𝗡𝗼𝘁 𝗝𝘂𝘀𝘁 𝗜𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻 If India wants to compete with global AI powers like the USA and China, we must confront the infrastructure reality. Today, India operates at roughly 1 watt of compute per person, while China operates at ~20 watts per person, and the USA at ~150 watts per person. This is not just a statistic. It represents capacity, speed, research advantage, and geopolitical influence. AI is powered by compute. Compute is powered by energy, chips, fabs, and data centers. The scale required to compete is enormous: • 50 million CPUs • 7 million GPUs • 3 billion memory devices • 1 million storage systems • 3nm semiconductor fabs operating at million scale monthly • High Bandwidth Memory scaling from current capacity toward industrial-scale production — with at least 10 memory fabs required This is a national infrastructure challenge, not just a startup problem. The discussion also centered around the five strategic pillars shaping the AI economy: 🔹 Compute 🔹 Content 🔹 Communication 🔹 Cognition 🔹 Community India’s GCCs (Global Capability Centers) were the previous growth engine. The next wave could be sovereign compute, semiconductor capability, and AI-native infrastructure. The question is no longer whether India will adopt AI. The question is whether India will build the backbone that powers it ? #IndiaAI #AIInfrastructure #IndiaAiImpact #ComputeEconomy #Semiconductors #DigitalSovereignty #AIFuture #artificialintelligence

9

Running PyTorch in production? Memory is most like an issue and also a silent bottleneck. I came across this blog that shows how they slashed inference latency and costs using lesser-known tricks. Here’s what stood out: 👉 Selective Gradient Checkpointing Checkpoint only memory-heavy layers → cuts peak memory by 40%. 👉 Dynamic Kernel Caching Cache common input shapes during warmup → avoids CUDA recompilation lag. 👉 Manual Precision Casting Control which tensors stay in FP32 vs. BF16 → stability + speed. 👉 Smart empty_cache() Scheduling Call it only during idle windows → avoids perf drops. 👉 Partial Quantization Quantize only safe layers like linear → preserve accuracy, save memory. 👉 Custom CUDA Streams Overlap compute and data loads → reduces GPU idle time. 👉 Shared Memory Tensors Zero-copy multiprocessing → boosts throughput for high RPS services. These aren’t just dev tips — they’re real production survival tactics. Full blog here - https://lnkd.in/gzJSccc8

46

2 Comments

🚀 Data Shapley in One Training Run A recent paper by Jiachen (Tianhao) Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia: “Data Shapley in One Training Run” presents In-Run Data Shapley, a method for attributing the contribution of each training example to a specific model within a single training run, removing the need for expensive model retraining. This approach calculates data contribution at each gradient update and accumulates these values throughout training. The authors propose two efficient approximation techniques: First-order In-Run Data Shapley – negligible runtime overhead compared to standard training; uses gradient dot-products to approximate contributions. Second-order In-Run Data Shapley – roughly 2× slower but provides more fine-grained contribution measures using Hessian-gradient products. This efficiency enables data attribution even at the foundation model pretraining stage. Several case studies illustrate insights into the contribution of training data, including negative contributions even in well-curated datasets like the Pile. 💡 Key implications: Improved data curation – identify low-quality or harmful data points that hinder training, leading to faster convergence and better performance. Copyright considerations – training data can influence model outputs beyond verbatim memorization, suggesting a need to reconsider data ownership and compensation in generative AI. Model analysis – understand how contributions vary across training stages and different examples. 👉 Question: As attribution methods like this become practical, what are the most critical ethical and business decisions regarding data ownership and compensation in AI development? ⬇️ Research paper link in comments #AI #MachineLearning #DataScience #Ethics #LLMs

1

1 Comment

Behind every high-performing ML model lies careful hyperparameter tuning. I recently published a blog explaining the syntax and basic working of hyperparameter optimization in scikit-learn—making it easier for teams to: Improve model accuracy Streamline workflows with reproducible code Build scalable solutions with confidence 🔗 Read the full blog here: 

4

Sharing the blog I wrote on our VLM fine-tuning work. It covers the lessons that helped us reach production-grade outcomes for our use case. Trying multiple solutions or reaching for bigger models is easy. Getting the outcome you actually wanted in production is where the real work lives. Worth a read if you're working on document AI or production fine-tuning.

13

I learned about when to use LLMs vs traditional ML: LLMs shocked me with zeroshot capabilities I got 78% accuracy on sentiment analysis without training data. But my Random Forest still crushes them on tabular data. The real difference? LLMs need 10TB+ of pretraining data but then generalize to new tasks. My traditional models need 10K labeled examples per task but train in minutes. I now use this rule: If the task needs understanding context or language nuance, LLM. If it's structured data with clear patterns, traditional ML wins every time. The cost difference is brutal though inference latency matters more than you think. #MachineLearning #LLMs #MLOps #ArtificialIntelligence #DataScience

11

2 Comments