AI Agents vs Workflows: Control and Autonomy

Most teams build an AI feature, it works in isolation, and then it breaks the moment real operators touch it. The model was right. The integration was wrong. This happens because AI gets bolted onto existing workflows as a black box. The operator doesn't understand what the model is doing, so they can't debug it. The engineer who built it doesn't understand the operator's actual process, so they optimized for the wrong signal. Neither side trusts the system enough to rely on it. We've stopped treating AI integration as a model problem. We treat it as a *feedback loop problem*. Before we write any code, we map three things: (1) What decision is the operator actually making today? (2) What does the operator need to see to trust the AI output? (3) How do we measure whether the AI is helping or just adding latency? Then we build the AI as a *tool in their workflow*, not a replacement for their judgment. Transparency first, accuracy second. The difference between a pilot that scales and one that dies is whether the operator understands why the AI said yes.

In our recent working paper, my colleague Oguz A. Acar and I shed more light on governing machines in the context of AI-to-AI agent interaction in lead-generation. We let the agents talk, take actions, and listened. See Oguz's post below for key insights. Will we soon witness the end of the old way of chasing leads? We hope this paper triggers a wider research conversation on AI-to-AI interaction. There is a lot left to understand. Where do you see the next implementation for AI-to-AI agent interaction? #AI #agents #ai_to_ai #lead_generation #governance

AI agents increasingly make decisions for organisations. We are moving towards a world with almost fully AI-intermediated platforms, with agents representing sellers, agents representing buyers, and very little direct human interaction in between. How should platforms govern markets where AI talks to AI? In a new working paper (w. Dr Jafar Sabbah) we built an AI-to-AI simulation and tested 16 governance configurations across 160 runs and 2,560 seller–buyer dyads. We examined four common platform levers: information disclosure, agent autonomy, structured interaction protocols, and visible reputation. The broad takeaway is that governance built for humans didn't translate cleanly to AI agents. But the effects were nuanced. High disclosure improved progression across the funnel. Autonomy didn't matter much early on, but mattered for qualification and especially conversion. Reputation helped at the start, but faded by the end. Structured protocols, somewhat surprisingly, suppressed progression up to qualification. Moving forward, organisations will need to study agent behaviour the way they study human behaviour. But unlike human behaviour, it will shift quickly as models, contexts, and tools change. A moving target. The best they can do is start building the testing infrastructure now. Link in the comments.

The biggest lie in enterprise AI right now: "We just need to find the right use case." No. You have hundreds of use cases. The problem is never finding one. The problem is that nobody in your organization has truly decomposed a single workflow into steps that are actually executable by an AI agent. There's a massive difference between: "we could use AI for claims processing" and "steps 2, 4, and 5 of our claims workflow are structured, rule-based, and touch 3 systems via API - an agent can handle those while a human handles step 6." The first is a slide deck. The second is a deployment plan. This is why we built the Workflow Deconstruction Model at Nebelus. Every workflow is a chain of atoms. - Some you automate. - Some you agentify. - Some you keep human. Until you've done that decomposition, you don't have a use case. You have a wish. What's stopping your team from moving past the "use case identification" stage?

Most organisations can describe what their AI systems are intended to do. Very few can evidence what their AI systems actually do under real-world conditions. That gap — between intended behaviour and observed behaviour — is where AI deployment risk is quietly accumulating. This is not a documentation problem. It is a testing problem. Traditional software testing assumes stability: the same input produces the same output. But modern AI systems do not behave that way. In generative AI systems, variability is not an exception. It is the default operating condition. That single shift changes everything. It means: 1. Behaviour must be tested, not assumed. Policies and guardrails describe design intent. They do not guarantee real-world behaviour. 2. Design and execution are different dimensions. A system can be well-designed and still behave inconsistently in production. 3. Testing must account for uncertainty, not eliminate it. The goal is not deterministic correctness — it is measurable behavioural reliability under defined conditions. I’ve been developing a structured approach to AI behavioural testing that explicitly treats AI systems as probabilistic systems rather than deterministic software. It forces a set of uncomfortable but necessary questions: How do you test something that can behave differently on every run? What does “sufficient evidence” look like under uncertainty? How do you form confidence in system behaviour without assuming consistency? Most AI governance today is focused on intent. The next stage is focused on evidence of behaviour. And that shift is already underway. The organisations that learn how to test AI systems properly — not just document them — will have a very different level of understanding of their deployed systems. That gap will matter more as AI becomes embedded in core business decisions. I’ll break down how structured AI behavioural testing actually works in the next post. #airisk #aigovernance #aitesting

A lot of AI teams are still measuring the wrong thing. They are optimizing for model performance on tasks that look impressive in evaluation, while the business is paying for reliability on tasks that are annoying in production. That gap is where a lot of disappointment comes from. The workflow does not care that the model scored well on a benchmark. It cares whether the system can repeatedly handle work that is: - ambiguous but not interesting - repetitive but high-volume - low-drama but operationally expensive - full of ugly edge conditions nobody puts in a demo This is why I’m increasingly skeptical of AI claims built on isolated task quality. In real operations, the more useful questions are: - how often does it need rescue? - how often does it create cleanup work? - how often does it shift burden downstream? - how stable is it on the 200th run, not the second? That is a very different standard from “it worked in our test set.” I think the next maturity shift in AI buying will be this: teams stop asking only whether a system can perform, and start asking whether it can stay boring. Boring is underrated. If a workflow runs quietly, consistently, and without creating managerial debris, that is usually more valuable than something that looks brilliant in a product demo. A lot of AI value will go to teams that learn to prize operational dullness over theatrical intelligence.

Last evening, ShareIT went full AI mode once again. But not the “AI will replace humanity by Friday” kind of AI. Our latest online edition, cleverly titled “Not another AI Agents presentation - What are AI Agents?”, was intentionally named after the exact thought most people probably have these days after seeing their 48th AI webinar invitation this month. And honestly? Fair enough. That’s precisely why Eduard Bosnea and Andrei Mutescu, both C# Specialists and passionate AI enthusiasts at CGM Software Romania, decided to take a different approach: less buzzwords, more substance; less hype, more architecture; less “look what ChatGPT did,” more “here’s how AI systems actually work in production environments.” During the session, CGMers explored: • the difference between AI Agents and AI Workflows; • when autonomous systems actually make sense (and when a simple workflow is the smarter choice); • orchestration patterns for multi-agent systems; • implementation strategies, scalability and security; • prompt injection risks, hallucinations and “SlopSquatting” (yes, apparently even AI can install fake dependencies with too much confidence); • and some very real anti-patterns where things fail spectacularly while everyone politely calls it “part of the learning journey.” One of the strongest ideas from the session? Agent = Model + Harness. Meaning the real challenge isn’t just the model itself, but everything around it: context engineering, tool orchestration, monitoring, security, failover mechanisms, architecture decisions, and making systems actually usable in real-world environments. We love these kinds of sessions because they reflect who CGMers are: curious people who don’t just jump on trends because they’re trending. People who want to understand technology deeply, question it critically, and explore how it can responsibly help us build better software and better digital healthcare solutions worldwide. Huge thanks to Edi and Andrei for a technical, honest, practical, and engaging ShareIT edition. Turns out that another AI presentation can still be worth attending after all. 

The best AI agent on a benchmark of real professional work completed 33.5% of tasks. That's one in three. The same week that number was published, a private-equity consortium paid up -- not for a model company, but for the firm that makes AI work inside actual organizations. So here's the question I'm sitting with: who on your team actually owns making AI deliver? Not "we're exploring it." Not "the platform handles that." A named human. An accountable role. Someone whose job it is to catch the one-in-three that fails and close the loop. Because the companies winning right now aren't the ones with the most impressive model access. They're the ones who solved the accountability layer. Four things that move the needle: Name the human who owns the outcome for every AI initiative. No owner, no project. Find and reward the person translating platform capability into shipped work. That role, not model access, is the real bottleneck. Treat one-in-three reliability as the design assumption, not the failure case. Ship a governed, usable tool faster than you write the policy banning the unsafe one. The model is a commodity. The operator is the asset. Thursday question: if the best AI agent finishes one in three tasks, and the smartest money is buying the people who deploy AI rather than the models themselves -- who on your team owns that role, and how would you prove it worked?

One thing that breaks AI workflows faster than many teams expect: the user changes direction mid-conversation. Not dramatically. Just slightly. They start with account recovery, then mention a failed payment, then add that the account was already verified yesterday. For a human operator, this is normal. The problem changed. For an AI workflow, this can be much harder. The system may keep treating every new message as a detail inside the original path, instead of noticing that the user has moved to a different problem. So the workflow continues. But it continues in the wrong direction. Interestingly, recent research showed roughly a 30% performance drop across multiple LLMs when users gradually revealed or shifted intent over several turns. Not because the models forgot how to solve the problem. Because they locked onto the earlier assumption. This is where many happy-path tests miss the issue. We usually test scenarios in isolation: - account recovery, - payment failure, - verification. But real users don’t stay inside neat flows. They interrupt. Correct themselves. Add missing details. Change goals halfway through. This is what rarely gets tested before release: not individual scenarios, but the transitions between them. Especially after turn 3, 4, or 5 — when the system has already formed assumptions about what the user wants. Curious how founders and teams are handling this today: How are you testing intent changes once users stop following the expected flow?

As someone who uses AI heavily every day, it already feels like the core capabilities are here. The models can reason, analyze, build, write, and execute at a pretty impressive level already. What feels missing now isn’t necessarily more raw intelligence, it’s systems that can improve themselves operationally over time without constant human intervention. That’s why some of the work coming out of Anthropic around continuous improvement loops is really interesting to me. A few ideas are starting to converge: 1. Reflective “Dreaming” systems Not memory in the chatbot sense, but systems that can review previous sessions, identify patterns, and refine workflows over time without retraining the core model. 2. Outcomes-based evaluation Separating execution from evaluation feels massively important. Dedicated reviewer/grader agents operating in separate contexts can reduce drift and improve consistency across longer workflows. 3. Multi-Agent Orchestration Instead of one model trying to do everything, different specialist agents can coordinate around specific responsibilities and tasks. When you combine those ideas together, you start getting systems that can: - coordinate work, - evaluate outcomes, - refine processes, - and continuously improve performance over time. Feels like the next phase of AI is less about “making the model smarter” and more about building systems that can adapt and optimize themselves.