Agentic AI isn't the cheap digital labor many believe it to be. It's probabilistic software demanding budgets, monitoring, and human oversight. The true measure of success will be completed outcomes, not tokens used. Agentic AI is significantly more costly than anticipated, both to build and operate. As we scale, the costs associated with back-end LLMs will only increase, impacting the dynamic behavior and autonomy of these agents. It's crucial to understand the system's scaling potential, token consumption, and overall business justification before committing. These fundamental questions are often overlooked, leading to unexpected expenses and an inability to decouple from costly LLM dependencies. Evaluate if agentic AI is truly needed and cost-justifiable for your enterprise. #AgenticAI #AIStrategy #LLM #TechCosts #BusinessValue #FutureOfAI
Great analysis as always David Linthicum - and another illustration of how driving real ROI from AI is orders of magnitude more complex than 'traditional' tech-enabled business change, and not being well considered by many organisations. More more complexity to be considered in each and every aspect (people, process, operating model, cost management, value discovery, governance, data, etc), driving 1) the 'demo/deployment gap', 2) growing services segment.
Good point, David. Engineers building AI systems must also design proper monitoring, observability, and governance around these solutions. This enables them to track outputs, monitor token usage, measure ROI, and take actions to optimize the systems and not just for effectiveness, but also for cost efficiency. Not every action within an AI solution needs to invoke an LLM. Certain tasks can be handled through deterministic logic, which can significantly reduce costs. LLMs should be used primarily for decision-making or reasoning tasks where they add real value