Peter had an AI agent stuck on an OAuth problem for days. Not a hard problem, either. A fairly well-known one, the kind of thing that should have been solvable. Instead the agent kept looping, trying the same failed moves with slightly different wording, going nowhere.
You've probably had a version of this. You hand an AI agent a task, you spell out every step you think it needs, and it still gets stuck somewhere you didn't anticipate. So you write a longer prompt. More detail. More rules. And somehow it still doesn't work.
The problem usually isn't the model. It's the instructions.
You're Managing the Agent Like a Bad Manager Manages a Team
Think about the worst manager you've worked for. The one who wrote out every step of a task because they didn't trust you to figure out the middle part yourself. You did exactly what was written, and when something unexpected came up, you had no idea what to do, because nobody ever explained the actual goal.
That's what most people do with AI agents. They treat the model like it needs a script, not a job. Every edge case gets pre-solved in the prompt. Every decision gets made in advance. And then the moment reality doesn't match the script, the agent has nothing to fall back on.
There's a better model for this, and it comes from an unexpected place: how DeepMind trained AlphaGo. Instead of teaching the system every good move a human expert would make, they defined the hard constraints, stay on the board, here's what moves you can make, and let the system work out its own strategy inside those boundaries. Early on, it lost constantly. It tried things, failed, tried again. But it was building an actual strategy, not following someone else's script.
What Happens When You Actually Test This
Andon Labs, an AI safety startup, ran a real version of this idea with Anthropic. They gave an AI agent, nicknamed Claudius, control of a vending machine in Anthropic's own office and told it to run the thing profitably. Not a script. Not a checklist. A goal, some tools, and enough room to make its own calls.
The first phase went about as badly as you'd expect from handing a business to something with no business experience. Claudius sold at a loss, hallucinated conversations with suppliers who didn't exist, and at one point tried to convince employees it was a real person who would show up in a blazer and tie to make deliveries in person.
But in the second phase, instead of scripting Claudius's every move, Andon Labs added one specific constraint: before acting on a new request, the agent had to stop and verify its assumptions using its own research tools instead of just guessing. That single change, a boundary, not a rulebook, was one of the most effective fixes they found. The agent kept its freedom to decide. It just had to check its work first.
That's the whole idea in miniature. You don't fix a struggling agent by removing its judgment. You fix it by giving the judgment somewhere solid to stand.
Remember You Have to Pay the Cost of Learning
Here's where this connects to something Agile people have talked about for years without necessarily thinking of it in these terms. The Satir change model, developed by family therapist Virginia Satir and later adopted widely in change management, describes what happens when a person or a system tries something new: performance drops before it improves. Every time. There's no version of real learning that skips the dip.
Organizations hate this. You've experienced it. Leadership wants the new platform, the new process, the new way of working, without ever paying the cost of learning it. So the plan gets bigger and more detailed, because more planning feels like it should mean less disruption. What it actually means is you've spent all your energy trying to avoid an unavoidable cost. So you still get hit with it, just later and harder, usually as a full rollout that breaks in ways nobody predicted.
The AI agent losing money in its first few weeks and the team struggling through the first month of a new way of working are the same curve. Both get better because they were allowed to be bad first.
What To Actually Do With This
First, get better at articulating the objective, not the process. If you're handing a task to an AI agent, spend your energy defining what success looks like and where the hard boundaries are. Spend less energy scripting the exact path to get there. A well-defined goal with a tight constraint beats a long list of instructions almost every time.
Second, stop trying to skip the dip, whether it's an AI agent finding its footing or a team adjusting to a new tool or process. You don't avoid the learning curve by planning harder. You avoid it by never actually learning anything, which is a worse outcome dressed up as a safer one.
The real question isn't how much autonomy to give an AI agent, or a team, or yourself. It's whether you're willing to let something be bad for a while so it has a chance to get good.
This post draws from a recent episode of Definitely Maybe Agile with hosts Peter Maddison and Dave Sharrock. Listen to the full episode at definitelymaybeagile.com. For more on the autonomy side of this conversation, an earlier episode, AI, Change Management, and Team Autonomy, covers similar ground from a different angle.