Welcome And The AI Hype

Peter [0:04]: Welcome to Definitely Maybe Agile, the podcast where Peter Maddison and Dave Sharrock discuss the complexities of adopting new ways of working at scale.

Dave [0:13]: Hello, Peter, how are you? Great to catch up again, virtually if not in person. I was just thinking about what today's topic might be, and honestly it's just a replay of conversations I keep seeing on LinkedIn. There's this whole narrative around AI making the "wrong bit" quicker. Coding is getting faster and faster, and I keep wondering whether that's actually the part of the conversation that matters.

Dave [0:45]: What impact does getting features out the door faster actually have?

Peter [0:48]: Well, I think part of the problem is that coding was never really the bottleneck to begin with. That's the thing you see on LinkedIn, and I believe it's true. The real problems always lived in two other places, and what AI has done is amplify that and make it obvious. There was an earlier take that said, "no wait, pull requests will be the bottleneck." But pull requests weren't really it either. The two big problems that have always been there, and never really been solved, are: do we actually know if the thing we produced is valuable? And what should we be producing in the first place that will be valuable? Those two things are still messy. Making the little bit in the middle go faster hasn't solved for either of them.

Dave [1:44]: I really like that. It reminds me of conversations around sprint length. We've been working with iterative delivery for decades now, and if you go back to the early days, the Scrum Guide says four weeks or less, with a preference for shorter. The general rule of thumb until recently has been about two weeks. And that length was never really determined by how hard the coding was. It was determined by exactly what you're describing: are we building the right thing, do we know how to build it. Sprint planning was never "the plan," it was how you arrived at the plan. How do you decide what to build, what it touches, how you manage quality? All of that included time for coding, but a lot more time was spent figuring out whether you were solving the right problem at all.

Peter [2:53]: Yeah, and that hasn't changed, in a sense. We still have a process for getting to something of value that we can look at and judge: is this something we want to put in front of a customer? That's still true. Being able to get to that point faster doesn't change the fact that we still have to put it in front of a customer, get feedback, and figure out if it was actually valuable.

Dave [3:27]: I'm smiling right now because this reminds me of kids and homework. You know that moment when they rush through it, throw it at you, and say "done, I'm heading out"? Just because they did it fast doesn't mean it's going to get the marks you're expecting. You still have to review it. There's always that "hold on, sit down, let's look at the hard part." Is this valuable? Are you solving the right problem? Is this something we can actually be proud of handing over? It was never about how fast the work got created. It's about whether there's real value there.

Peter [4:12]: And there's still friction in that piece. That's not to say there isn't a lot of value in being able to do the other pieces faster now. There's plenty we can do today that we couldn't before. It's just that a lot of that saved time ends up going toward making the thing we think is valuable more robust, higher quality, more finished than it would have been before we showed it to anyone. It's less scaffolded, more complete, because now we have that option. We might also be able to look at it sooner. But that raises the question of whether we can actually make decisions faster. And if that decision depends on customer feedback, we can't speed that up unless the circumstances allow it.

Feedback Loops, A/B Tests, And Their Limits

Peter [5:12]: If you're in a purely digital space, you can run forty experiments, or a hundred, push them out across a large user base, and measure a specific metric. That's one way to do it. But not every piece of software delivery has the option of gathering that kind of data.

Dave [5:33]: So what you're describing is post-deployment validation. Once something's out in the real world and users, or proxies for users, are interacting with it, you can effectively A/B test the old feature against the new one. I think that's a critical piece. But from a sprint-length perspective, I'm more interested in the internal stakeholder side. You and I still need details, context, probably a conversation, maybe someone persuading us that a new take on a feature or a new approach to a common problem is the right one. I don't think that's really an "AI speed" conversation. That still needs time.

Peter [6:28]: Everything about the organization, actually. I took all of that product management knowledge, wrote it into a markdown file, fed it into an LLM, and said "behave like my product owner, you make the judgment calls."

Dave [6:41]: Go on then, answer your own question.

Peter [6:45]: Oh, no, no, no.

Dave [6:46]: I'm leaving that entirely up to you. It's interesting, because we've talked about this a lot: this obsession with optimization and efficiency. If something takes five days and we can do it in five hours, think of all the value we've created. But what's interesting is the data coming out of surveys and analyses shows a lot more code being written, a lot more deployments, and the value is still hard to find. These things are a lot more complicated than just counting lines of code.

Peter [7:31]: Yeah, and shipping more code might just mean shipping more bugs, more edge cases, more complexity, more chances for things to break because dependencies between systems start to conflict. All of which, at scale, with enough tokens thrown at it, is potentially solvable.

Dave [7:48]: What I find interesting is that this first-order focus on optimization and efficiency isn't where the real value is going to come from. The real value comes from the things we can do in addition, separately. There's been a lot of talk about test automation and general product hygiene, and I'm not sure that's where things are headed either. But when you talk about validating ideas and building different solutions to test them, that's scenario planning. It reminds me of the lean practice of the last responsible moment: deferring commitment. The whole idea was that you defer commitment because you can manage two or three potential solutions and only decide once you have enough new information. That's scenario planning.

Scenario Planning Without Overload

Dave [8:48]: And of course, most of the teams I've worked with never really had time to do scenario planning. Maybe now they do.

Peter [8:57]: Yeah, I agree. It's one of those things you can do with enough discipline. But I have had the same experience: teams try it, they do it successfully once or twice, and then say "I'm not doing this on every single change." The complexity of designing, running, and managing multiple scenarios, then comparing the feedback across all of them, is just too high. Yes, you can do it, but is it necessary for every change, or is it better to just build toward a single solution, put it in front of a customer, and learn from that? I think it depends a lot on what exactly you're doing, the size and scale of the change. There are a lot of factors that go into deciding which way to go.

Dave [9:59]: And as you're describing that, we started talking about stakeholders and internal decision-makers understanding what's being built. Well, we've just made that decision-making harder, because instead of choosing between option A or B, or "we ruled out everything except A," those same stakeholders now have options A, B, C, D, and E. The feedback might say "choose option C," but as a leader you probably need to understand why. You need to be able to explain that decision to stakeholders who weren't in the room. You need to feel confident it isn't hallucinating, that it isn't a misread of where things are actually going. That's a deeper conversation, and it takes more time.

Peter [10:48]: That's part of what we're seeing in the numbers too. The recommended solution isn't always the right one. Sometimes that's because the AI didn't have enough context when it presented the option. Sometimes it's because the context it did have made a certain path look like the simplest solution, so it recommends that, even though the simplest solution is also the least secure or lowest quality. There are a lot of failure modes like that, and being able to think critically about what you're being handed back is becoming a really important skill.

Sprint Length Equals Decision Latency

Dave [11:36]: So does this conversation lead us toward shorter sprints?

Peter [11:42]: I think where we started, and we touched on this before we hit record, is this idea that the length of a sprint, the period over which you're delivering a concrete piece of value, should be based on the organization's decision and learning latency. Not on anything else. It comes down to: how quickly can the organization actually make decisions and gather information? That decision and learning capacity is what should dictate the length of time you have to deliver a piece of value.

Dave [12:31]: The conversation I've had with teams in the past has usually gone the other way. Until recently, the pushback on sprint length was always to push for longer sprints so teams could get more done. In two weeks, teams would push back and say "we've got a complex system, we can't make a meaningful change in two weeks, we need three." My response has always been that sprint length has nothing to do with how much you can produce, and everything to do with learning latency, exactly what you're describing. How long can we go without finding out we're headed in the wrong direction? If we spend two weeks chasing the wrong objective, we throw away two weeks of work. That's the cost of learning, at minimum. So we need two things: the ability to learn something, meaning people in the room with context who can look at what's been built and validate it, and the ability to actually make a decision, whether to keep going or change direction. AI generating ideas faster is powerful, and I can imagine sprints getting shorter. But there's also something valuable about stakeholders showing up every couple of weeks to talk about what's being done. In a startup this moves faster, but in a larger organization, how do you hold your stakeholders' attention every week, or every day, if you're running really short sprints?

Peter [14:06]: That takes me back to something I keep coming to: product owners exist to reduce decision latency.

Dave [14:11]: Right, and that's another piece worth pulling on. Product owners reduce decision latency. There are other roles that don't, let's put it that way.

Peter [14:24]: I think there are some genuinely fundamental pieces here. It's easy to say sprint length should relate to an organization's decision and learning latency. But how does an organization even know what its own latency is?

Dave [14:45]: Here's one signal that you're moving too fast, or at least not talking to the right people. A common anti-pattern is a sprint review playing out to an empty room. If the stakeholders who actually own the decisions, who are accountable for direction, aren't showing up to your sprint review, that's a sign you're not getting the feedback you need.

Peter [15:20]: Yeah, if you don't have that feedback when you need it, it's the age-old problem: why did you wait until this meeting to tell me, you could have just sent me a message.

Dave [15:36]: Well, that's a slightly different point. Maybe you don't need a sprint review with a full audience because you're getting rapid feedback some other way. Great. But often that rapid feedback isn't tied to the longer-term objectives and impact. If those quick decisions aren't backed up in front of a broader audience, and the team is left hanging, waiting for someone to step in and say "actually, here's the decision we reached," then you're not really getting the learning either, if nobody owns those decisions.

AI Agents And Delegated Decisions

Peter [16:14]: Back to my product owner example, since you never quite answered the question. If you had an agent with all the organizational knowledge, what if your actual product owner, who's been stuck in meetings all day, just got a report at the end of the day saying "here are the decisions I made on your behalf today"?

Dave [16:33]: I think that's going to happen. And it comes down to how you validate those decisions. All of us, you, me, our listeners, are already taking recommendations from our favorite language models and nodding along, "that'll do, next."

Peter [16:49]: Yeah, good enough, because there are just so many decisions to make.

Dave [16:52]: I'd like to think we're using judgment to separate high-risk from low-risk decisions, letting AI agents handle the low-risk ones while we stay informed on the high-risk ones. But honestly, I think that's wishful thinking at this stage, because we're still figuring out what that even looks like.

Peter [17:19]: I think it's also just the sheer rate of decision-making. It becomes really hard to keep that judgment sharp the whole time. Because you're making so many decisions so often, it's much harder to think critically about everything crossing your desk.

Dave [17:43]: There are certain roles built for this. I'm thinking of ER physicians, who make life-or-death decisions rapidly with very limited context. But roles like that come with massive investment in the people doing them, both the training and the repeated exposure that builds the skill. Most leaders in most organizations aren't trained for that. They don't have that experience of making rapid decisions constantly. I wonder if we need to start training leaders and executives to operate that way more often, because my honest first reaction when someone says "make a quick decision" is "hold on, slow down, I need context, I need to see if something's changed." Even with a dashboard pulling it all together, I'm still not going to feel comfortable making a big investment decision without a bit more time, or without building consensus with other stakeholders who have a stake in it.

Peter [19:12]: Yeah, and I wonder too whether the consequences of those decisions are part of what sharpens the mind in those roles. It'll be interesting to see where this goes.

Dave [19:31]: Yeah.

Peter [19:32]: I think we're getting close to our usual wrap-up point.

Dave [19:37]: Yeah.

Takeaways And Next Steps

Peter [19:38]: It's been a good conversation, as always. Let's land on three takeaways for our listeners. Where would you want them to focus?

Dave [19:51]: We started with coding and how fast we can move now with the tools available, and how that might be a distraction. There's more to the process than that, especially the design piece, whether we're solving the right problem. So I'd say the first takeaway is: be wary of speeding up or shrinking one step in your process without recognizing how many other steps feed into it, some of which are informal, implicit, and hard to measure.

Peter [20:27]: Building on that, thinking systemically about what's happening becomes critical. We could get away with caring less about it when the hand-off from one step to the next moved at human speed. Once it's moving at AI speed, you have to think more about the end-to-end delivery system, so you can actually spot where the constraints are and decide what to do about them, rather than hoping it all works out, or assuming productivity comes from cutting developers and letting AI write all the code.

Dave [21:08]: That leads naturally into the third point: decision latency, learning latency, and the pressure of making high-risk, high-value decisions in a short window. That's something worth exploring further, because it's not as simple as "just make more decisions faster." We all have a mental process, a comfort level, that we go through before making decisions like that. Speeding it up isn't as simple as it sounds.

Peter [21:43]: Totally agree. With that, let's wrap up. To everyone listening, don't forget to subscribe, and tell your friends to subscribe too. You can reach us at feedback@definitelymaybeagile.com. Until next time.

Dave [21:57]: Peter, until next time. Thanks again.

Peter [21:59]: You've been listening to Definitely Maybe Agile, the podcast where your hosts, Peter Maddison and Dave Sharrock, focus on the art and science of digital, agile, and DevOps at scale.