Vinicius Mesel 🇮🇱

🚀If you're deploying LLMs in production you're gonna love this.. My good friend Pedro Andrade Azevedo put together a Python template that integrates best LLMOps practices right from the get-go. 🎯 He is really a cool dude and knows what he is talking about. What can this template do for you? ✅ Structure your Python project with best practices ✅ Seamlessly integrate LLMOps tools ✅ Enable tracing, evaluations, and monitoring ✅ Ensure robust CI/CD pipelines ✅ Scalable Serving Endpoints (Cloud Agnostic) ✅ And much more... Here’s what’s included: 🔹 CI/CD: Automated pipelines for smooth development 🔹 QA tools: Ruff, Bandit, PyTest, MyPy, and test coverage 🔹 Config management: YAML-based using OmegaConf & Pydantic 🔹 Python packaging & environments: Poetry + Docker 🔹 Experiment tracking & model registries: MLFlow 🔹 LLM-specific tools: MLflow Traces & Evaluate for traceability and performance metrics 🔹 Unit & integration tests: Built for ML pipelines 🔹 Serving Endpoint built on top of Lightning AI's Litserve LLMOps Package: 🔗 https://lnkd.in/daQGajyB That package was inspired by Médéric HURIER’s original repo, so be sure to check it out for even more insights: 🔗 MLOps Python Package (https://lnkd.in/dq2iEY6S)

713

31 Comments

I want to share bit of context on today's new releases from DeepSeek: three very small (0-500 lines of code), self-contained, yet fascinating newly open-sourced repositories. Let's dive in! 1. The first one is just data: DeepSeek/Profile-data https://lnkd.in/eWmSNwN4 While this repo doesn't contain any code files, it's still extremely interesting. This is profile data that shows in detail and with real recorded data how low-level operations are scheduled to make sure GPUs are kept busy at all times during the training and inference of DeepSeek V3/R1 (see the "profiling session" in the Ultra-Scale Playbook: https://lnkd.in/eDC9TwSj) It serves as the organizational blueprint (essentially a Gantt diagram) of the most efficient open-source pretraining to date. A great example to study and something I would love to see released more often by open-source teams: scheduling operations in the most efficient way is the core of large-scale LLM training nowadays. 2. The second one is a very small code snippet (164 LoC) on how to balance the load of experts in mixture-of-expert (MoE): https://lnkd.in/e7d73QTR It's impressive that they extracted and released such a core technique for efficiently balancing the load among experts in a self-contained format. You can read more about this in the "Expert Parallelism" section of the Ultra-Scale Playbook: https://lnkd.in/eDC9TwSj This will make the technique easy to incorporate into most distributed codebases. Congrats 3. Today's last release is larger (500 LoC) and perhaps covers the most fascinating technical part of DeepSeek V3/R1 training: the new DualPipe pipeline parallelism (PP) approach: https://lnkd.in/erpEasbX For the first time in large-scale training, the DeepSeek team was able to train using what they called a "zero-bubble regime" in PP, something never before reported in a SOTA large-scale training. If you don't know what a "bubble" or "pipeline parallelism" is, you can check the Pipeline Parallelism section in the Ultra-Scale Playbook https://lnkd.in/eDC9TwSj). This is perhaps the most impressive part of the DeepSeek technical report! Having a small, standalone codebase that was apparently able to reach this regime is fascinating. I'm so excited to try it! --- Overall, I really like their focus on open-sourcing many independent code modules, each on a specific technique and with examples. Now it remains to be seen whether other teams can use and integrate these code samples and reproduce the results claimed by DeepSeek in their paper. Looking forward to seeing more extremely efficient training available for everyone!

4,128

73 Comments

I just created a trustworthy RAG app! It provides a confidence score and reasoning for each response. Tech stack: - LlamaIndex for orchestration - Cleanlab's trustworthy LLM - Qdrant to self-host a vectorDB - LlamaParse to make complex docs LLM-ready The video below shows a quick demo of how it works! We use LlamaParse to convert complex documents into clean, LLM-readable markdown text. And Cleanlab's trustworthy LLM integrates seamlessly with LlamaIndex, offering a trustworthiness score and the reasoning behind every response. This app is ideal for deployment in areas where precision is crucial. I've shared the entire code in the comments. It's straightforward to follow. If you're stuck, feel free to create an issue, and I'll assist you. Interested in ML/AI Engineering? Check out our newsletter for in-depth lessons and get a FREE eBook with 150+ core DS/ML lessons: https://lnkd.in/gB7yHExC ____ Find me → Akshay Pachaar ✔️ For more insights and tutorials on AI and Machine Learning.

736

44 Comments

Fun facts and big numbers for Open Source Software: $4.15B invested in open source generates $8.8T of value ($1 invested in open source = $2,000 of value created). Companies would need to spend 3.5 times more on software than they currently do if OSS did not exist. Interesting research from Manuel Hoffmann, Frank Nagle, and Yanou Zhou. https://lnkd.in/eA_XiUuy h/t Clem Delangue 🤗

925

48 Comments

Most people think RAG is just “vector DB + LLM.” But as you scale real-world use cases, Naive RAG breaks fast. Here’s a breakdown of the 4 types of RAG and how they evolve: → 📚Naive RAG The entry point. You embed the query, retrieve top-k chunks, and stuff them into a prompt. Works fine for simple Q&A, but struggles with multi-hop reasoning, long context, and hallucinations. → 🛠️Advanced RAG This is where real engineering begins. You layer in pre-retrieval filtering, hybrid indexes, reranking, query rewriting, memory, and post-retrieval prediction. You move from static retrieval to modular pipelines like: Retrieve → Read → Predict or Rewrite → Retrieve → Rerank → Read Useful when accuracy, context handling, or traceability matters. → ➿Graph RAG Structured meets semantic. You extract or connect to a knowledge graph, pair it with your vector DB, and retrieve both relational and unstructured data. Prompt gets augmented with graph paths and node metadata, enabling explainable reasoning. Used in enterprise search, healthcare, finance, and anywhere structured logic plays a key role. → 🤖Agentic RAG The most powerful RAG pattern today. Now, the model doesn’t just retrieve—it plans, acts, and routes. It decides: - What to retrieve - What function or tool to call - How to persist results It combines prompt + retrieved data + tool schema to dynamically invoke APIs or external actions. Your RAG stack now includes: tool functions, graph DBs, relational memory, and agent logic. If you’re building agents, copilots, or production-grade assistants, Agentic RAG is where the industry is heading. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

2,008

139 Comments

Only 5 days since announced, already at 100 GitHub 🌟! Three months ago, I had a chat with Miguel Otero Pedrido looking for a cool project idea we could build, and transform it into a comprehensive open-source course for the AI Community. Soon after, we decided on three things: - Multimodal Processing (Image/Video/Audio/Text) - Build a Tool-Calling AI Agent from Scratch - Add separation between the Agent logic and Processing Logic using MCP. We've combined all three and have built Kubrick: A Multimodal AI Agent. The best part about it - 𝗶𝘁'𝘀 𝗮 𝗳𝗿𝗲𝗲 𝗰𝗼𝘂𝗿𝘀𝗲 𝘆𝗼𝘂 can follow along and learn from us two. We've structured it into 6 modules/lessons that we'll soon start publishing. In this course, you'll learn about: ⇢ Video Processing ⇢ Building Multimodal Embedding Indexes ⇢ LLMs, VLMs, and Speech-to-Text Models ⇢ Tool-calling Agents from scratch, in Python ⇢ Building advanced MCP Servers in Python ⇢ Advanced Python, best practices. ⇢ Advanced APIs with FastAPI, best practices. ⇢ LLMOps and tracing multimodal Agent calls. ⇢ Many more Software Engineering and AI/ML Engineering tips. We haven't found any project out there that does what Kubrick is doing. Better yet, none of them guides you through the implementation, step by step! We think you'll learn a lot, beyond the basic text-based examples, and we're excited about teaching it! 👉 𝗪𝗵𝗮𝘁 𝗰𝗮𝗻 𝘆𝗼𝘂 𝗱𝗼 𝗻𝗲𝘅𝘁? 1. Start by starring the repo 2. Follow me and Miguel Otero Pedrido as we'll be sharing updates here on LinkedIn. 3. Subscribe to our Newsletters, and stay tuned for the first lesson! 📌 I've attached all the links you need in the first comment! 📌 This course is free, thanks to Pixeltable and Comet Opik, who sponsored it! ♻️ Reshare this, let your network know about it! --- #artificialintelligence #machinelearning #deeplearning #AI ---

163

19 Comments

Every tech downturn I've seen (Dot com bubble, Great Recession, and whatever this is) follows the same pattern: 𝐂𝐨𝐦𝐩𝐚𝐧𝐢𝐞𝐬 𝐨𝐯𝐞𝐫 𝐡𝐢𝐫𝐞, 𝐭𝐡𝐞𝐧 𝐨𝐯𝐞𝐫𝐟𝐢𝐫𝐞.  Layoffs don't mean tech is doomed, they're just a self-reinforcing cycle as companies mimic each other. The demand for people will come roaring back. We just don't know when and for what skills. When opportunities present themselves, those that are looking and can move quickly will have outsized returns. Those that wait and see until it’s a sure thing are doomed to get to the party late. Be on the lookout for where the market is going and what is in demand. My thought is that big companies should be hiring unlimited AI Automation engineers to automate all of the processes that costs too much for traditional software to automate. It could be that, or it could be something else. Opportunities only present themselves to people that are open and looking. Be that person.

172

15 Comments

Most Helm Charts are garbage. I’ve used Helm for 6+ years. Written charts. Debugged broken ones. Audited 150+ for clients. But most Helm charts are bloated, fragile, and impossible to scale. They weren’t built for your infra. They were built for any infra. 𝐇𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐚𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐡𝐚𝐩𝐩𝐞𝐧𝐬 𝐢𝐧 𝐫𝐞𝐚𝐥-𝐰𝐨𝐫𝐥𝐝 𝐭𝐞𝐚𝐦𝐬: ☠️ 1 toggle breaks 3 other things ☠️ You want to disable a feature, but now the whole values.yaml explodes ☠️ PRs go stale because nobody wants to touch that spaghetti of conditionals ☠️ 20 templates in 1 chart… and no one knows what’s actually deployed 𝐒𝐨 𝐡𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐰𝐞 𝐮𝐬𝐞 𝐢𝐧𝐬𝐭𝐞𝐚𝐝 𝐨𝐟 𝐇𝐞𝐥𝐦 𝐂𝐡𝐚𝐫𝐭𝐬: ✅ 1. Kustomize + Jsonnet for opinionated builds Forget general-purpose. We build narrow, composable layers. Each env has exactly what it needs. No more, no less. ✅ 2. Internal Helm forks with pre-baked policies For unavoidable 3rd-party Helm charts, we fork and harden them. We strip out what we don’t use. We codify security defaults. No more trusting upstream chaos. ✅ 3. GitOps-native templating with reusable overlays We write infra the way devs write code. Envs are just overlays, not duplicate manifests. Auditable. Portable. Reviewable. ✅ 4. YAML lint + Policy-as-Code in CI Before anything hits a cluster, we validate against org-wide standards. This catches the “hidden footguns” Helm often lets slip. 𝐀𝐧𝐝 𝐡𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐧𝐨𝐛𝐨𝐝𝐲 𝐭𝐞𝐥𝐥𝐬 𝐲𝐨𝐮: Helm is great for demos. Disasters in production. - Most values.yaml files turn into unmaintainable state machines - Real infra needs opinionated defaults, not endless if-else toggles - GitOps doesn’t need a templating engine. It needs consistency and clarity If your team is fighting Helm more than using it, you’re not alone. We’ve rebuilt entire CI/CD stacks just to get rid of Helm-induced debt. Want a faster way to ship secure, scalable infra? You don’t need 300-line values files. You need reproducible, testable infra with clear ownership.

147

31 Comments

DeepSeek has been making waves in the AI industry, and Portuguese newspaper Observador asked me to break down what's actually happening as. Two key observations: 1.⁠ ⁠Efficiency: •⁠ ⁠DeepSeek's models are demonstrably more efficient •⁠ ⁠They achieve this with algorithmic and smarter coding, cheaper infrastructure 2.⁠ ⁠The Complex Reality: •⁠ ⁠There isn't total transparency about training data •⁠ ⁠All AI systems, East or West, have review mechanisms •⁠ ⁠There are inherent biases in how these systems are trained What's often misunderstood: While DeepSeek has a censorship component, Western systems also have "the ability to identify which topics we want to allow the system to respond to and which topics we don't want them to." The core insight: There's always a human choice in how certain answers are given. These biases exist, whether we're conscious of them or not. Full analysis in Observador (link in comments). Fellow AI minds, what are your thoughts on the topic? #ArtificialIntelligence #AIEngineering #SystemDesign #TechnicalAI

56

8 Comments

Yesterday, Google released Gemma 3, a new open-weight LLM. Finally, a new addition to the "Big 5" of open-weight models (Gemma, Llama, DeepSeek, Qwen, and Mistral). I just went through the Gemma 3 report and experimented a bit with the models, and there are plenty of interesting tidbits: 1. Vocab size. They again use a very large vocab: 262k token (in contrast, Llama 3 has ~1/2 the vocab size), which should make the model more friendly for further pre-training on non-English texts. 2. Context window. They finally support larger contexts, up to 128k like Llama 3 (up from 8k in Gemma 2). 3. Multimodal support. They added multimodal support (but the implementation is a topic for another time) 4. Training set size. The training set size is 14T tokens for their 27B model, which is in a similar ballpark as Llama 3 (15T) and DeepSeek v3 (14.8 T). What's interesting is that the smaller models were trained on fewer tokens: 12T for the 12B version, 4T for the 4B version, and 2T for the 1B version. They probably adhered to their flavor of scaling laws here (but interestingly, in contrast, Llama 3.2 1B to 405B was trained on the same number of tokens as far as I know, as the Llama team didn't observe any saturation, and "more was better.") 5. Distillation. They use (a flavor of) proper distillation where they don't just train on synthetic data generated by the teacher model but also consider (a subset of) the logits of the teacher model. 6. Post-training. For post-training, in addition to RL with human preferences, they also added the DeepSeek-R1-like verifiable rewards for code execution and math problems. 7. Memory optimization. They use both grouped-query attention and sliding window attention to reduce memory requirements to compensate for the large vocab (similar to Gemma 2) but also now the larger context sizes. But where Gemma 2 used sliding window attention in every other layer, they now have a 5:1 ratio, meaning there's only 1 full attention layer for every 5 sliding windows (local) attention layers; moreover, the sliding window size was reduced from 4096 (Gemma 2) to just 1028 (Gemma 3). According to the ablation studies, this reduction to save memory has almost no impact on the model performance. I tried out Gemma 3 1B vs Gemma 2 2B vs LLama 3.2 1B for different context lengths, and Gemma 3 (1B) seems to be the most memory-friendly model now if you stay below 40k input tokens.

1,111

34 Comments

GPT-5 is the best coding model in the world and is now the default model in Cursor! Cursor: “It’s the most intelligent coding model our team has tested.” There is an amazing story about how this model became so much better at coding, featuring some innovative core research + behind-the-scenes efforts by the startups solutions team. In building GPT-5 the research team invested heavily in coding improvements and trained a model that showed large gains on the leading coding and SWE benchmarks. TL;DR: it was SOTA in coding. However, as we tested early alpha snapshots of the model GPT-5 with several of our most sophisticated coding customers, we realized there were still some gaps in performance and personality that only showed up when you used the models in products like Cursor. Making the experience great is about more than just the raw intelligence, which was solid - it’s also about how the model actually behaves in the product that matters for creating magical end-user experiences. For example, the first model snapshot would silently make many sequential tool calls without summarizing to the user what it’s doing or why, which created a less helpful developer experience. Or, it would produce front-end designs that weren’t as beautiful and clean, even though they were functional and produced with fewer errors than prior models. Seeing these gaps, a small group of startups solutions team members and engineers sprung into action. They decided to try to tackle this problem and bring the customer voice more squarely into our research process, working backwards from the actual tasks that startups needed the model to perform in their product. The key realization is that it’s not enough to saturate the public benchmarks; we had to make sure the models work and feel right in products like Cursor. A small team sprinted to work backward from these customer use cases and create representative evals, graders, and rubrics for scoring the model’s improvements on the key dimensions that matter - some of which were subjective by nature, like front-end design. In close collaboration with the research team, these inputs were incorporated into the training of subsequent model snapshots. The cross-functional team ran rapid cycles of development/testing/iteration to steer the behavior and performance of the model. The result is an *exceptional* coding model. It not only excels at core coding tasks - producing high-quality code, fixing bugs, editing code, and completing repo-level refactors - but also has a collaborative personality and an ability to generate strikingly clean & beautiful front-end designs. It’s been an incredibly exciting time at OpenAI. We are loving GPT-5 and I’m excited for developers everywhere to experience this model! https://openai.com/gpt-5/

284

15 Comments

How to deep dive any GitHub repo in under 30 seconds 1/ Pick any public repo Example: VLIM (open-source video library) 2/ Open DeepWiki A free tool by Cognition Labs 3/ Ask a question For example, “Where is streaming implemented?” 4/ Get answers instantly With file names, code snippets, and architectural breakdowns. 5/ Understand structure DeepWiki maps the systems and high-level design. 6/ Keep going Ask follow-ups, explore dependencies, or jump to the right file. This is how engineers will explore code in 2025. Share this with the next person asking, “How do I understand this codebase?” Video: cognition_labs

89

8 Comments

Not all AI agent workflows are built the same. The framework you use directly impacts efficiency, scalability, and output quality. Here are 5 agentic frameworks you can start building today in n8n (no code!): 1️⃣ Parent-Child 2️⃣ Prompt Chaining 3️⃣ Routing 4️⃣ Parallelization 5️⃣ Evaluator Optimizer 📺 Just dropped a video breaking these down—check it out! https://lnkd.in/ghjSZfC3

147

17 Comments

On June 17, 2025, Tong Xiao and Jingbo Zhu published an outstanding paper explaining how Large Language Models work. I’ve always said: the best way to master SEO is to think like search engines — and the engineers who build them. The same applies to LLMs: the best way to understand them is to think like they do. In the near future, search engine friendliness will evolve into LLM and Agent friendliness, as more content will be discovered through new interfaces. I’m sharing both this research and my 20 fundamental concepts for understanding LLM creation — a solid starting point if you want to master the basics. Transformer architecture – Neural network design using self-attention to model relationships between tokens in parallel. Tokens & tokenization – Breaking text into units (words, subwords, characters) the model processes. Self-attention mechanism – Weighing token importance relative to each other. Positional encoding – Adding sequence order information. Parameters – Trainable weights storing learned patterns. Pre-training – Large-scale training on general text. Fine-tuning – Adapting to a specific task or domain. Instruction tuning – Training to follow natural-language instructions. RLHF – Aligning outputs with human feedback. Loss function (Cross-entropy) – Measuring prediction accuracy. Context window – Max tokens processed at once. Zero-shot learning – Performing tasks without examples. Few-shot learning – Performing tasks with a few examples. Hallucination – Plausible but false outputs. Bias & fairness – Inherited data biases. Scaling laws – Performance growth with size/data/compute. Overfitting vs. generalization – Balancing memorization and adaptability. Data quality & diversity – Breadth and accuracy of training data. Inference – Using the trained model to generate outputs. Multimodality – Handling multiple input/output types (text, images, audio). 📚 Learn more and join our course: https://lnkd.in/dp8mGy8E #SEO #AI #LLM

214

24 Comments

This is truly amazing to see! On April 15, 2025, SatYield's computer vision models detected “Safrinha” Corn across Mato Grosso, Brazil, demonstrating the capability of our Engineers developing AI to identify crop types anywhere on Earth in real time. It's not just about mapping crops, it's a step forward in revolutionizing data-driven global agriculture. Scalable, accurate, and crop-specific. #AI #ComputerVision #AgTech #CropMapping #Satyield #AgricultureInnovation #Geospatial #FoodSecurity

30

3 Comments

One of the highlights of the Llama 4 release from my (biased) point of view is that it's the first major model that got released with the Xet backend on Hugging Face Xet achieved an average of ~25% deduplication accross weights in the Llama 4 model family, which means less bandwidth/faster downloads for everyone 🔥 Looking forward to more models published with Xet in the coming months, h/t: Jared Sulzdorf

112

3 Comments

AI is just search. Love this new post by 🤖 Jacky Liang on why: - Semantic search fails where precision matters - Vector databases are doomed - Claude Code’s grep-powered search destroys Cursor’s vectors - Hybrid search (combining lexical, full-text, and semantic) is the future My favorite line: "Similarity ≠ relevance. Vector search can’t find getUserById." If you’re building an AI app, whether it’s a code assistant, chatbot, or product search, this piece is a must-read. And this is just the beginning. We’re kicking off a series showing how to: - Build multi-search systems using Postgres - Combine surgical precision of lexical search with fuzzy power of full-text - Add vector search for semantic understanding (when it actually helps) Read Jacky's post here: https://lnkd.in/g5wrUDbp

69

5 Comments

MLflow Ambassador Joana Ferreira just published an excellent article on building consistent evaluation pipelines for GenAI applications, and it tackles a problem most teams don't realize they have until it's too late. The core issue the article addresses is evaluation drift: when your development testing uses different standards than your production monitoring. You end up with a model that passes all your local tests but behaves differently in production, with no consistent way to compare quality across environments. Joana's solution centers on the "write once, use everywhere" principle: define your evaluation scorers once in MLflow, then apply those exact same scoring functions during local development AND for continuous production monitoring. No more wondering if your prod quality metrics actually match what you tested during development. What makes this walkthrough particularly valuable is the practical implementation. It shows you how to build custom scorers that check both output format and correctness, then demonstrates how the same evaluation logic works seamlessly across your entire pipeline. This kind of systematic approach to evaluation consistency is something more GenAI teams need to adopt. Check out the full article: https://lnkd.in/gC9rSehY #MLOps #GenAI #LLMOps #MLflow #AI

20

1 Comment