Sugato Ray

Ever feel like today’s AI models are getting smarter but somehow less useful? You’re not imagining it. LLM benchmarks have become the obsession. Model makers are locked in a cycle of chasing leaderboard points on MMLU, GSM8K, and other tests no one actually cares about outside of research labs. The result? Models that ace exams but fall apart in real-world use. They misinterpret vague instructions. They forget what you said three messages ago. They can’t think beyond a single step. But sure, the score went up by 0.3 percent. Benchmarks were supposed to be a guide. Instead, they’ve become a crutch. A way to show “progress” without improving anything that matters. The questions we should be asking are simple: Can the model follow messy, human instructions? Can it reason through problems, not just regurgitate answers? Can it actually help people get real work done? If you’ve worked with these models, you’ve felt the gap. You’ve seen the hype, and then you’ve seen it fall short when it counts. Are we building for people, or are we building for points? I’d love to hear what others are seeing in the wild. #AI #LLMs #Benchmarks #MachineLearning #RealWorldAI

27

3 Comments

🔬 Granite Docling 258M: Efficient Document AI for Structured Extraction IBM Research released Granite Docling 258M—a compact vision-language model optimized for high-precision document understanding with layout preservation. Key Technical Specs: → 258M parameters (lightweight, inference-efficient) → Vision encoder + autoregressive decoder architecture → Multi-stage training: synthetic + real-world document datasets → Output formats: Markdown, JSON with preserved structure → CPU-friendly inference, production-ready Performance: → Competitive accuracy vs larger models on DocLayNet & PubLayNet → Maintains layout fidelity: tables, headers, reading order intact → Fast processing for enterprise-scale document pipelines Practical Applications: → RAG systems requiring structured context → Legal/financial document parsing with layout integrity → Research paper extraction for knowledge graphs → Multi-format document processing workflows Why It Matters: Balances efficiency with accuracy—critical for production Document AI where compute cost and latency matter. Open-source availability (Hugging Face + GitHub) accelerates adoption. Ideal for teams building document-heavy AI systems without overprovisioning infrastructure. Links are in the comments.

86

1 Comment

Quantization (Q) techniques substantially reduce the size of a large language model for better inferencing. I explain the matrix operations step by step for (1) Post-training Q, (2) Quantization-aware training, (3) 4-bit Q fine-tuning, and (4) mixed precision Q. Your comments are welcome. https://lnkd.in/e4heBApp

12

In the context of Path over Graph (PoG) for LLM-based multi-hop reasoning, various graph traversal techniques are employed to discover meaningful reasoning chains across a knowledge graph. Breadth-First Search (BFS) explores all immediate neighbors of an entity before moving deeper, ensuring that short but relevant paths are not missed—ideal for shallow reasoning tasks. Depth-First Search (DFS) goes as far along each branch as possible before backtracking, which can uncover deeper, more complex chains of reasoning. Beam Search strikes a balance by maintaining only the top-k most promising paths at each hop based on a scoring function (such as semantic similarity to the question), thereby controlling combinatorial explosion. After traversal, Path Ranking evaluates the semantic relevance of each candidate path—typically by converting them into natural language statements and scoring them against the query using models like SBERT or LLMs.. Beam Search is a controlled BFS that balances exploration and efficiency. At each hop, it keeps only the top-k highest scoring paths based on a scoring function f(pi,q)f(p_i, q)f(pi​,q), where pip_ipi​ is a candidate path and qqq is the question. The score could be computed using cosine similarity between SBERT embeddings: This scoring prunes irrelevant branches and helps find semantically relevant reasoning chains efficiently. Path Ranking - Once a set of candidate paths is retrieved, each path pip_ipi​ is converted into a natural language sentence (e.g., using templates), and its semantic similarity to the question qqq is computed. The paths are then ranked using: Technique Description Breadth-First Search (BFS)Explores all neighboring nodes before going deeper — good for finding short, relevant paths. Depth-First Search (DFS)Goes deep into the graph first — might find longer reasoning chains. Beam SearchKeeps the top-k most promising paths at each hop based on a scoring heuristic (e.g., similarity to query). Path RankingAfter traversal, paths are ranked based on semantic closeness to the original question. Why Is It Important? LLMs do not inherently know how entities are related in a structured KG. Traversal over the KG's structure allows PoG to find explicit, factual reasoning chains. These chains can later be passed into the LLM to ground the answer in factual paths. #POG #LLM #GENAI #GraphTraversal #LLMReasoning_Graph

24

Large Language Models (LLMs) have immensely advanced the field of Artificial Intelligence (AI), with recent models being able to perform chain-of-thought reasoning and solve complex mathematical problems, ranging from theorem proving to ones involving advanced calculus. Thrilled to share our latest publication “One-Shot Autoregressive Generation of Combinatorial Optimization Solutions Based on the Large Language Model Architecture and Learning Algorithms” coauthred with Ph.D. Student Bishad Ghimire and Prof. Ausif Mahmood. In this research, we attempt to explore a possible solution to the fundamental NP-hard problem of combinatorial optimization, in particular, the Traveling Salesman Problem (TSP), by following the LLM approach in terms of the architecture and training algorithms.

44

2 Comments

We open-sourced our framework for validating #ExplainableAI methods! Samuel Sharpe, Sr. Lead Machine Learning Engineer, shares how his team developed guardrails for ablation studies on tabular data. Check out the framework and test your own XAI methods. https://bit.ly/4esirrV

43

2 Comments

AI: Transfer Learning The broader concept of re-using a pre-trained model to adapt it to a new related task. it's widely used for image classification, for NLP models like BERT and GPT. MIT, Chhatrapati Sambhajinagar (An Autonomous Institute)

16

Dive into the world of machine learning with Diego Lomellini. Kicking off his blog series, Diego uses Micrograd to explain core ML concepts like supervised learning, regression, classification, loss functions, & gradient descent. Check out the blog post to learn more about how models adjust weights and biases during training using backpropagation. https://ghst.ly/44n3IeJ

42

Just explored the power of "Agentic AI in LangGraph" and here are my top 5 takeaways and how LangGraph is different than other frameworks and tools: Stateful Agents: Build intelligent, multi-turn conversations with agents that remember context. Dynamic Control: Agents can make decisions on the fly, enabling complex feedback loops and collaboration. Tool Integration: Seamlessly connect agents to external tools for real-world actions. Robust Error Handling: Build resilient agents that can recover from errors. Scalable Architectures: Easily create teams of specialized agents working together. 💡 Why LangGraph is a game-changer: While frameworks like CrewAI and LlamaIndex offer agentic capabilities, LangGraph's graph-based approach provides a more robust and flexible foundation for complex, stateful workflows. Compared to CrewAI's role-based structure: LangGraph's dynamic control flow offers greater adaptability and less rigid collaboration patterns. Agents can truly decide their next steps based on the current state, rather than strictly following predefined roles. Compared to LlamaIndex's data-centric agents: While LlamaIndex excels at data retrieval, LangGraph's focus on state management and dynamic routing makes it better suited for intricate, multi-stage processes that require reasoning and memory beyond just accessing information. LangGraph empowers you to build truly autonomous and intelligent applications for customer support, research, development, and beyond. #AgenticAI #LangGraph #LLMs #AI #MachineLearning #DeepLearning #ArtificialIntelligence #Innovation #Tech

23

We’re excited to launch the Microsoft-Northwestern-WITNESS deepfake detection benchmark — a collaborative, open-source dataset of over 50,000 AI-generated and real-world manipulated media artifacts. 🧠 Expert-annotated 🌍 Real-world, high-stakes examples 🧪 Includes adversarial attacks 🚫 Evaluation-only (not for training or commercial use) Our goal: to set a high bar for detection tools and support developers, journalists, and human rights defenders worldwide. We invite the AI community to help us keep this benchmark up to date. Collaboration between Microsoft AI for Good Lab: Thomas Roca, Juan M. Lavista Ferres, Pengce Wang Kevin White Northwestern Security & AI Lab: Chongyang GAO, Isabel Gortner, Marco Postiglione, Sarit Kraus, and me WITNESS: Mahsa Alimardani, shirin anlen, Zuzanna W. and Sam Gregory To find the dataset and associated paper, visit: 🔗 https://lnkd.in/gP-_E6Aw #AI #Deepfake #ResponsibleAI #OpenSource #HumanRights #Cybersecurity #NationalSecurity #InternationalSecurity #Defense #Defence #GlobalPublicGood #AIForGood #MachineLearning #GenerativeModel

138

9 Comments

Feeling overwhelmed by the math in machine learning? 😩 Egor Howell's article shares a practical guide with recommended textbooks and online courses to build your foundation in statistics, linear algebra, and calculus.

14