How to Train AI Models on a Budget

Most are sleeping on the power of 𝗠𝗼𝗱𝗲𝗹 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻, and every company should have a Distillation Factory to stay competitive This technique is reshaping how companies build efficient, scalable, and cost-effective AI. First, 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗠𝗼𝗱𝗲𝗹 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻? Also known as knowledge distillation, is a machine learning technique where a smaller, more efficient "student" model is trained to replicate the behavior and performance of a larger, more complex "teacher" model. Think of it as a master chef (the teacher) passing down their culinary expertise to an apprentice (the student) without sharing the exact recipe. The student learns by observing the teacher’s outputs and mimicking their decision-making process, resulting in a lightweight model that retains much of the teacher’s capabilities but requires fewer resources. Introduced by Geoffrey Hinton in his 2015 paper, “Distilling the Knowledge in a Neural Network,” the process involves: 1/ Teacher Model: A large, powerful model trained on massive datasets. 2/ Student Model: A smaller, efficient model built for faster, cheaper deployment. 3/ Knowledge Transfer: The student learns from the teacher’s outputs—distilling its intelligence into a lighter version. There are several types of distillation: 1/ Response-Based: The student mimics the teacher’s final outputs 2/ Feature-Based: The student learns from the teacher’s intermediate layer representations. 3/ Relation-Based: The student captures relationships between the teacher’s outputs or features. The result? A student model that’s faster, cheaper to run, and nearly as accurate as the teacher, making it ideal for real-world applications. 𝗪𝗵𝘆 𝗘𝘃𝗲𝗿𝘆 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗡𝗲𝗲𝗱𝘀 𝗮 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻 𝗙𝗮𝗰𝘁𝗼𝗿𝘆? In today’s AI landscape, very large LLMs are incredibly powerful but come with significant drawbacks: high computational costs, massive energy consumption, and complex deployment requirements. A Distillation Factory is a dedicated process or team focused on creating distilled models, addressing these challenges and unlocking transformative benefits. Here’s why every company should invest in one: 1/ Cost Efficiency: Distilled models cut costs, running on minimal GPUs or smartphones, not data centers. 2/ Scalability: Smaller models deploy easily. 3/ Faster Inference: Quick responses suit real-time apps. 4/ Customization: Tailor models for healthcare or finance with proprietary data, no full retraining. 5/ Sustainability: Lower compute needs reduce carbon footprints, aligning with green goals. 6/ Competitive Edge: Rapid AI deployment via distillation outpaces costly proprietary models. A Distillation Factory isn’t just a technical process; it’s a strategic move.

People debate restaurants for 45 mins but pick their AI models in 45 secs. 300+ models. Millions at stake. Yet people pick the one who has the best marketing budget. 67% of organisations globally now run LLM-powered workflows. API pricing spans from $0.15 to $60+ per million tokens, depending on what you pick. 239+ models are being actively evaluated on major benchmarks as of 2026. And the model that topped every leaderboard six months ago is already in the middle of the pack. So here's the decision framework and where today's most relevant models actually sit within it 👇 🔐 STEP 1: Local or Cloud? Sensitive data? Go local. And local no longer means weak. → Low/medium tasks (extraction, simple logic): IBM Granite 4.1 (3B, free), MiniCPM-V 4.6 (1.3B, free + multimodal), NVIDIA Nemotron-4 15B → Complex local reasoning: DeepSeek-V4-Flash (self-hostable, 13B active parameters) paired with Chain-of-Thought prompting does heavy lifting without touching a cloud API. Cloud-connected? Horsepower isn't the problem. Cost and latency are. ⚡ STEP 2: Capability vs. Speed/Cost? Maximum intelligence needed: → Claude Mythos Preview leads on GPQA reasoning. Gemini 3.1 Pro is the strongest for coding agent workflows. GPT-5.5 hits 74.9% on SWE-Bench for software engineering tasks. Speed and cost matter more: → GPT-5.5 Instant is now the default ChatGPT model — fast and affordable. → Qwen3-Max at ~$1.20/M tokens matches Claude Sonnet 4.6 on Arena-Hard. DeepSeek V3.2 sits at $0.28/M input — arguably the best cost-performance ratio available right now. 🎯 STEP 3: Does a specialised model exist for your domain? Almost always yes — and it almost always wins on narrow tasks. → Code agents: GPT-5.3 Codex (tuned for agentic IDE workflows at $1.25/M), DeepSeek-V4-Pro, Gemini 3.1 Pro → Budget-conscious general use: Kimi K2.6 at $0.95/M is the cheapest model in the current top 10 by GPQA Diamond. Gemini Flash for real-time agentic use cases. The shift nobody's pricing into their stack yet: GPT-4-level performance cost $30/M tokens in 2023. Today, you get equivalent capability for under $1/M. Intelligence is commoditising faster than teams are adapting. The open-source ecosystem is now self-hosting models that would have required expensive API access just a year ago. The right model isn't the most powerful one. It's the one that fits your task, your data privacy requirements, and your budget simultaneously. Every time. Saved the full decision tree above as your cheat sheet. Stop choosing models on vibes. Start choosing a strategy. If you liked this, you'll hate how much time you've been wasting on the wrong models. Follow me. I break down AI decisions that actually matter for builders.

When you want a large language model to get better at a specific task—like solving math problems or navigating websites—the standard approach is to finetune it: you adjust the model's internal parameters using training data and gradient descent, which is expensive, requires lots of data, and often makes the model worse at everything else. Instead of changing the model's parameters, this paper proposes to run the model on a small set of problems multiple times, compare the successful and failed attempts, and use the model itself to write down natural-language "lessons learned"—things like "when solving geometry problems, always check that your solution falls within the valid region." These lessons get iteratively refined across a few rounds and then get pasted into the prompt at inference time. The method is modeled after GRPO, a reinforcement learning algorithm where you generate a group of outputs, score them, and use the relative quality differences to improve the model—except here the "improvement" happens in the prompt text rather than in the weights. The paper shows that doing this with just 100 training examples and about $18 worth of API calls on a large frozen model (DeepSeek-V3.1-Terminus, 671 billion parameters) outperforms smaller models that were finetuned with thousands of examples at costs exceeding $10,000. The results hold across both math reasoning and web search tasks, and unlike finetuned models that degrade when moved to a different domain, swapping in a different set of learned experiences lets the same frozen model perform well in multiple domains simultaneously. Read with an AI tutor: https://lnkd.in/eA3Ud2a2 Download the PDF: https://lnkd.in/ekVxsz3B

If you’re an AI engineer, understanding how LLMs are trained and aligned is essential for building high-performance, reliable AI systems. Most large language models follow a 3-step training procedure: Step 1: Pretraining → Goal: Learn general-purpose language representations. → Method: Self-supervised learning on massive unlabeled text corpora (e.g., next-token prediction). → Output: A pretrained LLM, rich in linguistic and factual knowledge but not grounded in human preferences. → Cost: Extremely high (billions of tokens, trillions of FLOPs). → Pretraining is still centralized within a few labs due to the scale required (e.g., Meta, Google DeepMind, OpenAI), but open-weight models like LLaMA 4, DeepSeek V3, and Qwen 3 are making this more accessible. Step 2: Finetuning (Two Common Approaches) → 2a: Full-Parameter Finetuning - Updates all weights of the pretrained model. - Requires significant GPU memory and compute. - Best for scenarios where the model needs deep adaptation to a new domain or task. - Used for: Instruction-following, multilingual adaptation, industry-specific models. - Cons: Expensive, storage-heavy. → 2b: Parameter-Efficient Finetuning (PEFT) - Only a small subset of parameters is added and updated (e.g., via LoRA, Adapters, or IA³). - Base model remains frozen. - Much cheaper, ideal for rapid iteration and deployment. - Multi-LoRA architectures (e.g., used in Fireworks AI, Hugging Face PEFT) allow hosting multiple finetuned adapters on the same base model, drastically reducing cost and latency for serving. Step 3: Alignment (Usually via RLHF) Pretrained and task-tuned models can still produce unsafe or incoherent outputs. Alignment ensures they follow human intent. Alignment via RLHF (Reinforcement Learning from Human Feedback) involves: → Step 1: Supervised Fine-Tuning (SFT) - Human labelers craft ideal responses to prompts. - Model is fine-tuned on this dataset to mimic helpful behavior. - Limitation: Costly and not scalable alone. → Step 2: Reward Modeling (RM) - Humans rank multiple model outputs per prompt. - A reward model is trained to predict human preferences. - This provides a scalable, learnable signal of what “good” looks like. → Step 3: Reinforcement Learning (e.g., PPO, DPO) - The LLM is trained using the reward model’s feedback. - Algorithms like Proximal Policy Optimization (PPO) or newer Direct Preference Optimization (DPO) are used to iteratively improve model behavior. - DPO is gaining popularity over PPO for being simpler and more stable without needing sampled trajectories. Key Takeaways: → Pretraining = general knowledge (expensive) → Finetuning = domain or task adaptation (customize cheaply via PEFT) → Alignment = make it safe, helpful, and human-aligned (still labor-intensive but improving) Save the visual reference, and follow me (Aishwarya Srinivasan) for more no-fluff AI insights ❤️ PS: Visual inspiration: Sebastian Raschka, PhD

In a recent roundtable with fellow CXOs, a recurring theme emerged: the staggering costs associated with artificial intelligence (AI) implementation. While AI promises transformative benefits, many organizations find themselves grappling with unexpectedly high Total Cost of Ownership (TCO). Businesses are seeking innovative ways to optimize AI spending without compromising performance. Two pain points stood out in our discussion: module customization and production-readiness costs. AI isn't just about implementation; it's about sustainable integration. The real challenge lies in making AI cost-effective throughout its lifecycle. The real value of AI is not in the model, but in the data and infrastructure that supports it. As AI becomes increasingly essential for competitive advantage, how can businesses optimize costs to make it more accessible? Strategies for AI Cost Optimization 1.Efficient Customization - Leverage low-code/no-code platforms can reduce development time - Utilize pre-trained models and transfer learning to cut down on customization needs 2. Streamlined Production Deployment - Implement MLOps practices for faster time-to-market for AI projects - Adopt containerization and orchestration tools to improve resource utilization 3. Cloud Cost Management -Use spot instances and auto-scaling to reduce cloud costs for non-critical workloads. - Leverage reserved instances For predictable, long-term usage. These savings can reach good dollars compared to on-demand pricing. 4.Hardware Optimization - Implement edge computing to reduce data transfer costs - Invest in specialized AI chips that can offer better performance per watt compared to general-purpose processors. 5.Software Efficiency - Right LLMS for all queries rather than single big LLM is being tried by many - Apply model compression techniques such as Pruning and quantization that can reduce model size without significant accuracy loss. - Adopt efficient training algorithms Techniques like mixed precision training to speed up the process -By streamlining repetitive tasks, organizations can reallocate resources to more strategic initiatives 6.Data Optimization - Focus on data quality since it can reduce training iterations - Utilize synthetic data to supplement expensive real-world data, potentially cutting data acquisition costs. In conclusion, embracing AI-driven strategies for cost optimization is not just a trend; it is a necessity for organizations looking to thrive in today's competitive landscape. By leveraging AI, businesses can not only optimize their costs but also enhance their operational efficiency, paving the way for sustainable growth. What other AI cost optimization strategies have you found effective? Share your insights below! #MachineLearning #DataScience #CostEfficiency #Business #Technology #Innovation #ganitinc #AIOptimization #CostEfficiency #EnterpriseAI #TechInnovation #AITCO

🚀 How to manage the budget dilemma when dealing with "Model Size" vs "Data Size" One of the biggest decisions in building language models today is figuring out where to spend your limited compute budget. Should you train a bigger model or train on more data? It’s something I’ve had to decide on more than once, especially while planning model training pipelines with fixed GPU hours and tight delivery timelines. This is a real-world challenge faced by AI product and engineering leaders every day. Some recent experiments in this space have shown something interesting: 💠 If you have a small budget, it’s often better to go with a smaller model and train it on a lot of data. Bigger models don’t help much if they don’t have enough data to learn from. 💠 As your budget increases, the ideal approach shifts. You can start scaling up the model size, but the data size still plays a major role. The improvement you get from adding more parameters tends to flatten out quickly. What continues to help is feeding your model more tokens. The key takeaway for you is: ◾ For a fixed budget, a medium-sized model with the right amount of training data can outperform a large model with limited data. ◾ As budgets grow, don't just throw more parameters at the problem. Focus on the balance between model size and data, and lean toward more training data if you're unsure. Finding that balance is key. It often determines whether you’re building something usable or simply burning through compute. 🔍 While there’s no plug and play enterprise product for this yet, there are practical tools you can explore: 1️⃣ A helpful GitHub repo on scaling laws that shows how to model this trade-off (link in comments) 2️⃣ A Hitchhiker’s Guide to Scaling Law Estimation, which walks through small-scale simulations and extrapolation techniques (link in comments) I highly recommend you use these type of Open-source tools, combined with internal logs and basic plotting, to give your AI teams a strong head start on getting the most out of your training budgets. Remember, it is not just about building the POCs, it is about getting these AI products/ solutions into production I write about #artificialintelligence | #technology | #startups | #mentoring | #leadership | #financialindependence   PS: All views are personal Vignesh Kumar

Last week, a VC firm invited me to eval an AI startup. The founder pitched building a custom 70B parameter model for insurance underwriting. Tailored inference. Vertical SaaS. The works. The ask? $1M seed round. I stayed quiet during the pitch. Every founder deserves respect — they're risking everything to build something. That courage is real. But afterwards, I told the VC: "This math doesn't work. Not even close." Here's why 👇 𝗧𝗵𝗲 𝗕𝗮𝘀𝗶𝗰 𝗠𝗮𝘁𝗵 𝗡𝗼𝗯𝗼𝗱𝘆 𝗗𝗶𝗱 Training a 70B model needs ~8.4 × 10²³ FLOPs. One FLOP = one math operation (a multiply or add). A 70B model does 6 of these per parameter, per token. Trained on 2 trillion tokens. 6 × 70B × 2T = 840,000,000,000,000,000,000,000 operations. That's 8.4 × 10²³. Let that sink in. 𝗪𝗵𝗮𝘁 𝗧𝗵𝗶𝘀 𝗖𝗼𝘀𝘁𝘀 → 512 H100 GPUs × 40 days × $2.50/hr = $1.2M just for ONE training run → First run WILL fail. Budget 2-3 attempts = $2.5-3.5M → That's just training. No team. No data. No infra. His entire $1M? Gone in 3 weeks. Not even one complete training run. 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗖𝗼𝘀𝘁𝘀 (𝗣𝗼𝘀𝘁-𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴) Serving 70B model to customers: → 4x H100 GPUs minimum = $7,200/month (cloud) → 1000 concurrent users? 16x H100s = $28,800/month → That's $3.5L/month just to keep the lights on 𝗪𝗵𝗮𝘁 𝗛𝗲 𝗦𝗵𝗼𝘂𝗹𝗱 𝗗𝗼 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 Fine-tune Llama 3 70B with QLoRA on insurance data. Cost: $200-500. Time: 2 days. Same business outcome. Spend the $1M on distribution, not GPUs. 𝗧𝗵𝗲 𝗟𝗲𝘀𝘀𝗼𝗻 Bangalore has incredible founders. Brilliant people building real things. But VCs — please do the FLOP math before writing cheques. Founders — please learn compute economics before pitching "we'll build our own model." Not every AI startup needs to train from scratch. Most shouldn't. The moat isn't the model. It's the data, distribution, and domain expertise. $1M buys you a world-class fine-tuned product. $1M doesn't buy you 1% of a foundation model. Know the difference. 💡 Quick Reference: 70B Model Costs Training: $1.2-3.5M (cloud GPUs) Inference: $7,200-28,800/month Fine-tuning instead: $200-500 Engineering team (12 months): $1-3M Total realistic budget: $5-8M minimum 1 FLOP = 1 floating point operation 70B model training = 8.4 × 10²³ FLOPs Your H100 does 10¹⁵ FLOPs/sec Do the division before the pitch. #AI #Startups #Bangalore #VentureCapital #DeepTech #GPUEconomics #FounderAdvice

The most expensive annotation mistake in ML data management is labeling data that doesn't improve your model. 💶 📉 Our latest research paper from Voxel51 ML, Zero-Shot Coreset Selection via Iterative Subspace Sampling (#WACV2026 https://lnkd.in/eHqhVmwR), shows teams can achieve the same model accuracy with 10% of their training data when they curate before they annotate. That's not incremental — that's a fundamentally different way to build visual AI. This approach uses pre-trained foundation models to analyze unlabeled data and score each image based on the unique information it contributes to the dataset. Redundant samples — the ones you'd otherwise pay to annotate — get filtered out before any labeling begins. 📊 Benchmarks on ImageNet indicate that this technique achieves the same model accuracy with just 10% of the training data, eliminating annotation costs for over 1.15 million images. The key insight: most annotation budgets are spent on data that doesn't meaningfully improve your model. Random sampling misses critical edge cases. Labeling everything wastes budget on redundancy. Strategic selection based on information contribution solves both problems. This research is now part of #FiftyOne's new Annotation capabilities, alongside our new in-platform 2D and 3D annotation, ML-backed error detection, and tight integration with curation and model evaluation. Curate first. Annotate smarter. 🎨 🧠 Learn more about how you should be building your Visual AI pipeline: https://lnkd.in/ev-ntaht 

The first "L" in LLM gives away a practical challenge to working with large language models: They’re huge. Training and deploying AI models packed with hundreds of billions of parameters isn’t just a technical feat. It costs an incredible amount of money, resources, and time. But what if you need a powerful model that’s lighter on the wallet, energy efficient, and nimble enough to run on a phone or edge device? The solution is the subject of my latest Code to Care video: model distillation. In this edition, I invited my colleague, AI expert and InterSystems sales engineer Nicholai Mitchko, to explain how model distillation transfers knowledge from big models into small models, just like a student learning from a teacher. The trick is to feed both models the same data, compare their outputs, adjust the small model to mimic the big model, and repeat. Step by step, the small model learns from the big. And by iterating the process, you can create a solution that fits your needs. Chances are, you’ve already encountered these models. They’re the “turbo” versions of their bulkier cousins. The question is: Where will you deploy these small models to make a big difference in healthcare?

The biggest lie about AI in finance: how much cloud you need. Mid-frequency quant firms (<100 employees) should spend thousands monthly, not hundreds of thousands. 𝗠𝘆𝘁𝗵: You need H100 fleets. Reality: Efficiency waves cut active compute: DeepSeek-V3 is a 671B MoE with ~37B active params/token; IBM's NorthPole in/near-memory chip shows lower latency at far higher efficiency than top GPUs in LLM inference. 𝗠𝘆𝘁𝗵: You must run on AWS/GCP/Azure. Reality: Hetzner dedicated starts €37–€100/mo; comparable Big-3 VMs land $276–$580+/mo. If you don't need managed services, pick dedicated for batch/inference. 𝗠𝘆𝘁𝗵: GPUs make the bill explode. Reality: Even GCP T4 on-demand ≈ $0.35/hr (spot/reserved lower) → a few hours/day <$50/DS/month; plenty of L4/T4 spot options at similar scale, or even better see Vast.ai 𝗠𝘆𝘁𝗵: Fine-tuning is cost-prohibitive. Reality: Sky-T1-32B-Preview reported ~$450 training to o1-preview-level performance on select reasoning/coding benches, and there are many more examples of this since. 𝗠𝘆𝘁𝗵: Advantage comes from training weights. Reality: Don't train weights; fine-tune, or even better shape the inputs (context engineering), e.g. see Perplexity's success. 𝗠𝘆𝘁𝗵: Foundation inference is pricey. Reality: Google's Gemini 2.5 Flash/Flash-Lite targets ultra-low token costs (cents-per-million-token pricing via Google/Vertex). 𝗠𝘆𝘁𝗵: You need BigQuery/Snowflake. Reality: Iceberg/Delta on object storage: ACID, time travel, schema evolution—DuckDB/Trino/Spark query in place. BigQuery/Snowflake now read Iceberg in your buckets: ~$0.023/GB-mo, no copy tax. 𝗠𝘆𝘁𝗵: Warehouses are mandatory for scale. Reality: DuckDB on a €3k laptop ingests ≈2 GB/s and runs TPC-H SF3,000–10,000 locally. Arrow↔DuckDB is zero-copy. 𝗠𝘆𝘁𝗵: Parquet is optimal for all workloads. Reality: Arrow-native engines fix time-series/object-store pain: TileDB (N-D arrays), Lance (S3 + ANN), Vortex (cascaded compression; LF AI project), F3 (Wasm decoders; SIGMOD'25). 𝗠𝘆𝘁𝗵: You need a vector-DB cluster for retrieval. Reality: LanceDB runs file-based on S3 with ANN indexes (serverless-friendly), avoiding hot persistent disks. 𝗠𝘆𝘁𝗵: Embeddings demand GPUs. Reality: Static/Model2Vec encoders deliver 100–400× faster CPU retrieval with competitive quality—ideal for hybrid (lexical+sparse+dense-rerank). 𝗠𝘆𝘁𝗵: Serving stacks are the bottleneck. Reality: vLLM/TGI continuous batching yields multi-k tokens/s on A-class GPUs and big throughput gains vs naïve serving. 𝗠𝘆𝘁𝗵: Banks are "piloting." Reality: HSBC AML: ~60% false-positive reduction (with 2–4× more true hits); BofA: 2–3B+ interactions; Morgan Stanley advisor ~98% adoption; Goldman: "~95% of an S-1 in minutes." The screenshot is from LightningAI. The bubble in AI will come down, not because AI doesn't work, but because the research enabling it is working so well, it's eating its own tail.