Key Challenges in LLM Technical Understanding

Most people still think of LLMs as “just a model.” But if you’ve ever shipped one in production, you know it’s not that simple. Behind every performant LLM system, there’s a stack of decisions, about pretraining, fine-tuning, inference, evaluation, and application-specific tradeoffs. This diagram captures it well: LLMs aren’t one-dimensional. They’re systems. And each dimension introduces new failure points or optimization levers. Let’s break it down: 🧠 Pre-Training Start with modality. → Text-only models like LLaMA, UL2, PaLM have predictable inductive biases. → Multimodal ones like GPT-4, Gemini, and LaVIN introduce more complex token fusion, grounding challenges, and cross-modal alignment issues. Understanding the data diet matters just as much as parameter count. 🛠 Fine-Tuning This is where most teams underestimate complexity: → PEFT strategies like LoRA and Prefix Tuning help with parameter efficiency, but can behave differently under distribution shift. → Alignment techniques- RLHF, DPO, RAFT, aren’t interchangeable. They encode different human preference priors. → Quantization and pruning decisions will directly impact latency, memory usage, and downstream behavior. ⚡️ Efficiency Inference optimization is still underexplored. Techniques like dynamic prompt caching, paged attention, speculative decoding, and batch streaming make the difference between real-time and unusable. The infra layer is where GenAI products often break. 📏 Evaluation One benchmark doesn’t cut it. You need a full matrix: → NLG (summarization, completion), NLU (classification, reasoning), → alignment tests (honesty, helpfulness, safety), → dataset quality, and → cost breakdowns across training + inference + memory. Evaluation isn’t just a model task, it’s a systems-level concern. 🧾 Inference & Prompting Multi-turn prompts, CoT, ToT, ICL, all behave differently under different sampling strategies and context lengths. Prompting isn’t trivial anymore. It’s an orchestration layer in itself. Whether you’re building for legal, education, robotics, or finance, the “general-purpose” tag doesn’t hold. Every domain has its own retrieval, grounding, and reasoning constraints. ------- Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

AI models like ChatGPT and Claude are powerful, but they aren’t perfect. They can sometimes produce inaccurate, biased, or misleading answers due to issues related to data quality, training methods, prompt handling, context management, and system deployment. These problems arise from the complex interaction between model design, user input, and infrastructure. Here are the main factors that explain why incorrect outputs occur: 1. Model Training Limitations AI relies on the data it is trained on. Gaps, outdated information, or insufficient coverage of niche topics lead to shallow reasoning, overfitting to common patterns, and poor handling of rare scenarios. 2. Bias & Hallucination Issues Models can reflect social biases or create “hallucinations,” which are confident but false details. This leads to made-up facts, skewed statistics, or misleading narratives. 3. External Integration & Tooling Issues When AI connects to APIs, tools, or data pipelines, miscommunication, outdated integrations, or parsing errors can result in incorrect outputs or failed workflows. 4. Prompt Engineering Mistakes Ambiguous, vague, or overloaded prompts confuse the model. Without clear, refined instructions, outputs may drift off-task or omit key details. 5. Context Window Constraints AI has a limited memory span. Long inputs can cause it to forget earlier details, compress context poorly, or misinterpret references, resulting in incomplete responses. 6. Lack of Domain Adaptation General-purpose models struggle in specialized fields. Without fine-tuning, they provide generic insights, misuse terminology, or overlook expert-level knowledge. 7. Infrastructure & Deployment Challenges Performance relies on reliable infrastructure. Problems with GPU allocation, latency, scaling, or compliance can lower accuracy and system stability. Wrong outputs don’t mean AI is "broken." They show the challenge of balancing data quality, engineering, context management, and infrastructure. Tackling these issues makes AI systems stronger, more dependable, and ready for businesses. #LLM

Some challenges in building LLM-powered applications (including RAG systems) for large companies: 1. Hallucinations are very damaging to the brand. It only takes one for people to lose faith in the tool completely. Contrary to popular belief, RAG doesn't fix hallucinations. 2. Chunking a knowledge base is not straightforward. This leads to poor context retrieval, which leads to bad answers from a model powering a RAG system. 3. As information changes, you also need to change your chunks and embeddings. Depending on the complexity of the information, this can become a nightmare. 4. Models are black boxes. We only have access to modify their inputs (prompts), but it's hard to determine cause-effect when troubleshooting (e.g., Why is "Produce concise answers" working better than "Reply in short sentences"?) 5. Prompts are too brittle. Every new version of a model can cause your previous prompts to stop working. Unfortunately, you don't know why or how to fix them (see #4 above.) 6. It is not yet clear how to reliably evaluate production systems. 7. Costs and latency are still significant issues. The best models out there cost a lot of money and are very slow. Cheap and fast models have very limited applicability. 8. There are not enough qualified people to deal with these issues. I cannot highlight this problem enough. You may encounter one or more of these problems in a project at once. Depending on your requirements, some of these issues may be showstoppers (hallucinating direction instructions for a robot) or simple nuances (support agent hallucinating an incorrect product description.) There's still a lot of work to do until these systems mature to a point where they are viable for most use cases.

Tech IQ: 1-LLMs Aren't Magic Wands (And That's Okay!) Today let's talk about something that's been buzzing in boardrooms: Large Language Models (LLMs) like ChatGPT, LLAMA, DeepSeek etc. They are incredible tools, but here is the catch, they are not plug-and-play magic. Let me explain with a story. Say a colleague asks, Why can't we just connect an LLM to our database and let it answer questions in plain English? Isn't that what AI does? ...Great vision. But here's what's missing in that mental model... 🔍 What LLMs Don't Know (Unless You Teach Them) LLMs aren't mind readers. Imagine handing someone a 1000 page book written in a language they don't speak and asking them to summarize it. That's an LLM without context. To talk to your data, it needs - Schema & Metadata: What do your table names mean? How are they connected? - Data Dictionaries: Is "revenue" called "Rev," "Sales," or "$$" in your system? - Data Profiles: What's normal vs. an outlier? Is "Q4" always the biggest quarter? Without this, the LLM is guessing. 🧩 The Invisible Workflow   Turning a casual question like "Show me last year's top-selling products by region" into an answer involves micro-steps:   1. Decoding what top-selling means (revenue? units sold?) 2. Joining 5+ tables (sales + inventory + customer data) 3. Filtering 10k rows without hitting token limits (yes, LLMs have text "budgets") 4. Explaining results in human language without misinterpreting numbers This isn't magic it's engineering. ⚙️ How Do We Actually Make It Work? Two paths: 1. RAG (Retrieval-Augmented Generation): Teach the LLM to "look up" answers in your data like a librarian. But first, you need organized shelves (clean data + clear metadata) 2. Fine-Tuning: Custom-train the model on your business's language. Think of it like teaching company jargons to a new hire Both need time, testing, and iteration. 💡 Key Takeaways for Leaders   1. LLMs need context they don't "learn" your business by osmosis 2. Token limits are real. Think of them as text message character limits… but stricter 3. Data quality matters. Garbage in = confusion out 4. Start small. Pilot a single use case (e.g. FAQs) before overhauling workflows 🚀 The Bigger Picture   LLMs are powerful, but they're like Formula 1 cars they need a skilled pit crew (your engineers) and a well-built track (your data infrastructure). The ROI? Huge. But it's a partnership, not a solo act. Next time someone says, "Let's just plug in the AI," smile and ask: "What's step one?" Tech IQ Mission: Simplify tech concepts for leaders. No jargon, no eye rolls just clarity. Refer my git repo for detailed process flow on using LLM for querying your data base. Got a topic you'd like me to discuss? Let me know! 👇

The hard part isn’t the stack. It’s the semantics. Clients often ask me something like this: “We already have SharePoint and we want to use AI to better mine knowledge from it — and also to create documents that follow our templates, standards, and business context. What is the tech stack we need?" My answer is usually the same: 'The hard part isn’t the stack. It’s the semantics. Because if it were just the stack, installing Microsoft Copilot would solve it for everyone." Frameworks and plumbing are table stakes. What actually makes or breaks an enterprise Copilot is what happens between the tools — the meaning, the intent, and the interpretation. What’s actually hard ➜ Ontology & vocabulary: Agreeing on what things mean. Defining entities, relationships, and synonyms that everyone shares. ➜ Context engineering: Deciding what evidence to pull, at what depth, under which permissions, and with what freshness. ➜ Prompt contracts: Stable, testable prompt structures for each task or persona. ➜ Meaningful relevance: Ensuring what’s retrieved actually answers the question, not just matches a keyword. ➜ Evaluations: Building repeatable ways to test grounding, citation accuracy, and output quality. ➜ Persona fit: Adjusting tone, context depth, and risk to the person asking. ➜ Governance: Tracking drift, versioning prompts and models, and managing escalation paths. How to make it work ➜ Map your domain: With SMEs, codify entities, synonyms, and disambiguation rules. ➜ Codify retrieval policy: Define evidence types, chunking rules, recency, and security trimming. ➜ Standardize prompt templates: One per task, with typed variables and unit tests. ➜ Add quality gates: Automatic checks for grounding, citation, contradiction, and leakage. ➜ Build an evaluation harness: Gold tasks, rubric scoring for correctness and grounding. The real challenge isn’t integrating APIs or spinning up an LLM. It’s making sure the knowledge retrieved carries meaning, the generation reflects context, and the output fits the user asking for it. That’s where AI stops being a toy — and starts delivering business value.

Over the past year, I’ve been involved in 10+ generative AI projects. Surprisingly (to me at least), the technical complexity of these projects often resembles data engineering optimization problems more than traditional "AI." Here are some of the key challenges I’ve observed, many of which seem more likely to serve as viable moats than any "fine-tuned" model: Indexing and Organizing Large Data Sets When processing or summarizing massive amounts of unstructured data, it’s impossible to fit everything into the context window of an LLM API request. The challenge is organizing and indexing this data accurately before reaching the “LLM step” in your pipeline to maximize its utility. This involves not just architectural decisions but also a cost-versus-accuracy trade-off when choosing models. For example, if GPT-4 tokens are 10x more expensive than GPT-4-mini but offer only 7% better accuracy for your use case, is the higher cost justified? Is it sustainable within your business model? Add to this the time-consuming process of benchmarking and testing other model families, and it becomes a significant effort. Selecting Models Across the Pipeline In large data pipelines, LLMs may be utilized at various stages, requiring decisions about which model to use where. These choices depend on cost, execution speed, and accuracy, and finding the optimal balance is a complex and non-trivial task. Execution Speed for Large-Scale Use Cases Some of the most compelling LLM use cases involve processing tens of thousands—or even millions—of pages of unstructured data with associated search and query functionality. For many such applications, execution speed is critical. Users expect results in seconds, not hours. Slow execution makes it difficult to iterate on ideas or hypotheses. Achieving fast results while maintaining accuracy when dealing with vast unstructured data sets is a significant (and expensive) challenge. Prompt Quality and Edge Cases Crafting high-quality prompts, handling edge cases, and benchmarking results are tedious but essential tasks. While most people are aware of this at a high level, its dealing with all the edge cases that takes a lot of iteration and work. While the power of LLMs is undeniable, the most differentiated aspects of many generative AI systems today lie in the steps that precede the involvement of an LLM. These challenges—data organization, indexing, and pipeline optimization—are where the real complexity and opportunities for innovation currently reside. Maybe this will change in the future, but for now, this domain feels more akin to big data engineering than traditional AI. My first company LYFE Mobile was programmatic ad platform that started in 2011 and faced some of the exact same challenges. Integrating, normalizing, indexing & cost optimizing massive amounts of data. Its interesting that as our technology evolves, some of the main problems of data engineering seem to be timeless. TIA Ventures ScrumLaunch

This paper systematically reviews the current applications and challenges of LLMs in patient care across 89 studies from 2022 to 2023, covering 29 medical specialties. 1️⃣ Over 94% of studies analyzed LLMs as medical chatbots, but only 20% explored their use in generating structured patient information, such as discharge instructions and informed consent documents. 2️⃣ Most studies evaluated GPT-3.5 (53.2%) and GPT-4 (26.6%), with GPT models outperforming older architectures (e.g., BERT, LLaMA) in patient care tasks, though open-source models remain underexplored despite transparency advantages. 3️⃣ Non-comprehensiveness (87.6%) and incorrectness (87.6%) were major concerns, with frequent hallucinations (42.7%), delirium (38.2%), and confabulation (20.2%), highlighting risks in medical contexts. 4️⃣ Non-reproducibility (42.7%) was a critical issue, as LLMs produced inconsistent answers to identical medical questions, raising reliability concerns for clinical use. 5️⃣ Many LLMs lack medical-specific optimization, with issues such as limited clinical reasoning, restricted internet access, and implicit knowledge gaps affecting their accuracy and applicability in healthcare. 6️⃣ The review categorizes LLM limitations into design and output issues, identifying 6 second-order and 12 third-order design limitations (e.g., lack of medical optimization, data transparency issues) and 9 second-order and 32 third-order output limitations (e.g., non-reproducibility, incorrectness, bias). 7️⃣ Ethical and safety risks were widely reported, including privacy concerns (9%), misleading information (38.2%), and potentially harmful content (29.2%), emphasizing the need for stricter safeguards. ✍🏻 Felix Busch, Lena Hoffmann, Christopher Rueger, Elon van Dijk, Rawen Kader, Esteban Ortiz-Prado, Marcus R. Makowski, Luca Saba, Martin Hadamitzky, Jakob Nikolas Kather, Daniel Truhn, Renato Cuocolo, Lisa Adams, Keno Bressem. Current applications and challenges in large language models for patient care: a systematic review. Communications Medicine. 2025. DOI: 10.1038/s43856-024-00717-2

Key Takeaways: 1. Abstraction Overload in LangChain • LangChain’s modular abstraction appears helpful but often masks hidden LLM calls, chaining them unpredictably. • This leads to uncontrolled compute costs, inefficiencies, and difficulty debugging complex behaviors . 2. Production Mayhem • Developers aiming for production-ready code get entangled in “abstraction complexity”: misalignment between advertised simplicity and real-world deployment challenges. • The frameworks can do more harm than good once scaling or operational stability becomes critical . 3. Opaque Cost Structures • Hidden or duplicated API calls result in inflated usage and unpredictably high costs across large-scale applications . 4. Debugging & Transparency Issues • When frameworks internally manage the agent logic, traceability suffers. Engineers lose control and visibility—making debugging a nightmare . 5. Call for Better Tools: Atomic Agents • The author critiques overhyped frameworks and proposes Atomic Agents, a minimalist alternative that: • Offers clear, explicit control over LLM interactions • Reduces hidden calls and complexity • Enhances predictability and ease of debugging .