AI Meets The Software Delivery Bottleneck

Welcome and Catching Up [0:04]

Peter [0:04]: Welcome to Definitely Maybe Agile, the podcast where Peter Maddison and Dave Sharrock discuss the complexities of adopting new ways of working at scale. Hello, Dave. How are you today?

Dave [0:13]: Excellent to see you again, Peter. It was great to spend a bit of time with you in person last week. It was good to hear what you’re up to and catch up properly.

Peter [0:24]: Indeed, indeed. It was a lot of fun. A very long, tiring week, though. There was a lot going on.

Dave [0:30]: That’s the downside of being in person. The conversations can go long into the night.

Peter [0:36]: Indeed. So what are we talking about today?

AI and the Future SDLC [0:40]

Dave [0:40]: One of the things we’ve been throwing around back and forth a lot is the software delivery lifecycle and the impact AI brings to it. Through our conversations, we started exploring the idea of looking at the SDLC in terms of clock speed. What is the end game if I really bring in agentic AI and AI-enabled delivery?

Machine Speed Versus Social Speed [1:10]

Dave [1:10]: There’s an end game where development is basically done at machine speed. But there are other parts of the software delivery lifecycle. If I think of customer behavior, launching a new product or new features on a product and waiting to see how customers interact with it, that can never go to machine speed. There’s always going to be a gap of days, weeks, even months where we have to wait and observe customer behavior before we can conclude whether a particular feature was a success or not.

Peter [1:42]: Which raises a question. When we have hundreds or potentially thousands of changes going into a large system in a very short period of time, how do we tell which ones people like?

Dave [1:58]: Yeah. And part of the challenge is that the analysis side is the easier part now. We have significant compute to run multivariate testing, change lots of features in parallel, and segment our customer base so we can test lots of ideas. So the constraint is less about doing the analysis and tracking what’s going on. It’s more that we’re fundamentally limited by the number of customers we can segment, and by the fact that customers work at their own speed. They’re not going to have an agent behaving on their behalf. They’re going to walk through an e-commerce workflow themselves, or go through some business flow, whether that’s a B2B system, a system for employees, or one for brokers or agents. They’re going to go through it in their own right, at their own speed. So you don’t get to shortcut that feedback.

Peter [3:05]: Well, we might have to at some point. For example, imagine a system where your identity lives in some form of certificate with all the information needed to fill these things in. The agent could pull that down, populate the different fields, and send it off. But at some point the human still has to make a judgment call about whether this is what they want to do. Or, if I’ve given all the instructions to my agent, I can let it go make those decisions. So the agent goes and buys you a pizza versus buying you a flight versus buying you a hat.

Dave [3:46]: There’s some point in any of these customer behaviors where the customer has to behave, right? By all means we can send data back and forth, and we don’t have to fill in those tedious forms. But we’re still going to have to decide that we want to go down this route and complete the data gathering, or decide that we need to buy or use whatever services we’re looking at. That’s going to slow down the evaluation of those thousands of minor tweaks we’re making to a system. Which means we’re outstripping our ability to validate that the changes are moving us in the right direction, in terms of a positive return, whatever that return might be.

Peter [4:38]: Yeah. And I think you described this well. One was social speed and the other was machine speed. As systems do the execution faster and faster, we can hand off to these non-deterministic agents that can potentially make judgment calls on our behalf. There’s a whole ethics and governance side of this that we’re leaving out of the conversation for now, around when and how that should happen. This is more about what could potentially happen. There are parts of this that, at some point, need to move at social speed, the speed at which a human interacts with the system. Part of the solution at a large scale is that you have lots of customers, so you can test many different things. But even then, you’re not going to know right away. You have to leave all those experiments in place long enough to learn whether behavior changed the way you were expecting.

Approvals and People Systems Stall Flow [5:45]

Dave [5:45]: And I think we see a very similar headache at the beginning of the software delivery lifecycle, in product discovery and ideation. We can potentially hand off a lot of that. But if we’re going to differentiate based on the identity of the company we work with, its values and how it makes certain decisions, we need a point where we take a moment to inject that direction into requirements definition, discovery, and ideation. So as excited as we are about shortening and speeding up elements of the software delivery lifecycle, we’re still going to be held back by the slowest parts. It’s like any conversation about being constrained by the slowest-moving piece of the system.

Peter [6:44]: Yeah. Traditionally, that’s theory of constraints. We still have that part, and understanding how we optimize or change the system so we’re working with that part in the best possible way is part of the work.

Dave [7:08]: I think there are a few things that bubble out of it. Number one, there are lots of conversations right now about how AI applied to the software delivery lifecycle is going to magically change a lot of what happens. We need to be careful to recognize that it’s going to change parts of that process, but the process is still limited by its slowest piece. And at the moment, there isn’t much talk about what that slowest piece looks like. If you really want to move the needle, start with the slowest pieces. Think about how you make good decisions about which features are landing well and which aren’t, and how you make better product discovery decisions. Those are the places where you’ll have a significant impact. If you can shorten those, the proportional shift in what you’re able to do is probably the greatest benefit.

Peter [8:03]: Yeah. And we’ve talked about this before. It was never the generation of the code that was slowing things down. It always took a long time to figure out what we were going to do and how, and to make sure we captured all the pieces. Or to get it into customers’ hands and learn fast enough to create those feedback loops. We’ve gotten a lot better at that back end. Organizations have adopted practices that make it much more feasible, and there are many places where we can show evidence of that. But the front part of the system still requires an awful lot of decision-making. It depends heavily on the systems inside the organization for approval, for making sure there’s sufficient authority in the right places, that risk and compliance have looked at it, that security has taken a look, and that everyone has done all these steps before it ever gets to the code. So it’s almost like we’ve taken one piece and shortened it, but there’s still a lot of effort needed to improve the rest. Organizations that already start from a product operating model, where they’re very focused on the products they deliver, obviously have an advantage here.

Dave [9:32]: Yeah. And one thing I wanted to mention as you describe that. These aren’t technical systems. They’re people systems, social systems. The politics of getting decisions approved, supported, rolled out, and backed by teams or individuals in the organization who have a pretty strong influence on what can be built.

Peter [9:59]: If you have to go up another level to get approval, or go to a different department, the actual technical task often takes very little time these days. If it’s been built out, templated, and managed properly, it takes milliseconds. Getting everybody to sign off on everything they need to sign off on, so it can actually happen, can take an awful lot longer.

Dave [10:24]: Yeah. And as we explore this space, something interesting comes out in addition to that. The whole software delivery lifecycle moves at the pace of its slowest section, not its fastest. We understand that, but we’re forgetting it in this rush to speed up big parts of the software delivery lifecycle.

Experiments at Scale and Token Economics [10:45]

Dave [10:45]: There are two things that bump out of this that we wouldn’t necessarily have expected at the outset. The first is that if we can build things really quickly, we can build many versions. The conversation we’ve had around complex problems, setting up probes, experiments, or bets to go and test, has always been more of a theoretical exercise for most companies. Very few would actually go to market with a dozen interpretations of how customers might behave and see which ones work. With today’s build speed, a dozen or even a hundred variations becomes feasible. It’s still constrained by the fact that evaluating which ones work remains awkward and difficult.

Peter [11:36]: Yeah. The validation of it. If you have a sufficient harness, you can build hundreds of versions of something, run them through a validation harness based on whatever criteria you’ve set, and throw away the ones that don’t meet them. So you can build in a very different way. Although every company that sells its product by the token is very happy to hear people say things like this, because every one of those thrown-away copies costs money. So you need to start understanding the economics of it.

Dave [12:21]: Well, that’s the cost of each experiment, right? The time taken to build was certainly a real cost. That’s being replaced by token costs. But also by validation costs, which perhaps haven’t come down as much as we’d like to think.

Peter [12:39]: And this comes back to having a strong enough harness wrapped around all of this, one that keeps evolving and building out the validation parts of the end-to-end process, so you’re properly managing and governing the cost. You’re not taking the biggest, most expensive model out there, pressing the “I’m feeling lucky” button, and expecting something valuable at the other end. That’s basically putting a hundred monkeys in a room and hoping one of them writes Shakespeare. It’s a way to do it, but an incredibly expensive one. If you do it that way, you’ll probably reach the point where hiring a few very good engineers serves you better. It’ll be much more cost-effective, because otherwise you’ll be burning a lot of dollars.

Dave [13:35]: And part of that cost is customer behavior validation, not just technical validation. That one is going to be constrained by social speed as much as by your ability to segment your audience appropriately.

Moats, Stickiness, and Faster Competitors [13:55]

Dave [13:55]: The other thing I wanted to point out, and I think we’re both seeing this in some of the companies we work with, many of them startups or in the early stages of developing products, is the moat and the need for protection. At the beginning of the software delivery lifecycle, around product ideation, you can skip a lot of work by pointing at an existing product that does well in its field and saying, “We want a copy of that, with a bit of our own interpretation.” That greatly reduces ideation cost. Your development cost is heading toward zero, or at least a much shorter time frame. So the period when I can operate in a market without viable competitors shrinks rapidly too. Any viability conversation has to take into account that we may not be the dominant player in that field for as long as we’d hoped.

Peter [15:00]: Yeah. And this brings us to how you protect that. Part of it is that you want to be there first and win the customers. How sticky you are becomes a key question. Stickiness can come from data, but it can also come from how good the product is. Do I feel inclined to go look elsewhere? Someone still has to come in with something attractive enough for me to switch from what I have.

Dave [15:30]: Well, there are switching costs, and a number of other factors. What I’m seeing in a lot of these conversations about moats, when you look at business ideas, is that it’s harder and harder to find one. That doesn’t mean there are no moats. But deciding how you’ll protect your advantage is becoming more and more important. It used to be that even if you couldn’t really protect it, you could still have the market to yourself for a number of years. Think of Netflix. Yes, other streaming services came along, but it took many years. Everybody waited to see whether Netflix would succeed as a streaming entertainment provider. Once it showed it could, it still stayed dominant for years, simply because it took time for competitors to pull things together and compete.

Peter [16:26]: Yeah. And I think this ties back to the social speed part. That’s part of the moat. The human interaction, the speed at which your customers can respond to changes, and the speed at which you can get that learning back into the system all matter a lot as you think about how to build, protect, and grow products as you bring them to market.

Dave [16:56]: Absolutely. So how do we wrap things up? What are the three takeaways?

Takeaways and Theory of Constraints [17:00]

Peter [17:00]: Well, if two of them are social speed and machine speed, we’re two-thirds done. I like the concept of social speed, and I think it’s worth exploring further. We need human interaction to get certain types of information. Where can we accelerate those pieces, and where shouldn’t we? If you accelerate in the wrong places, by replacing social interactions with machine interactions where they don’t belong, you may find you lose customers because of it.

Dave [17:41]: And there are a number of cautionary tales about exactly that. Those decisions were made, it shook the confidence customers had in the company, and it changed the relationship.

Peter [17:55]: Yeah. So I think that’s a key point. A lot of the acceleration in the system is focused on the development practice. Generating code becomes very cheap. But you still need to understand your system and make sure decisions are made in the right places and judgment is applied correctly. If your system wasn’t set up well for that in the first place, using AI to generate more code faster isn’t going to fix it. You still have a lot of work to do on the system itself.

Dave [18:33]: And that leads naturally to the last one. We’re always building on the shoulders of giants, and one of those is Eli Goldratt’s idea about optimizing a process one section at a time, the theory of constraints. This is just a reminder that the theory of constraints has never told us that coding is the constraint. As we accelerate certain steps of the software delivery lifecycle to machine speed, we have to recognize that those steps aren’t the constraints. The constraints are elsewhere, in the things moving at social speed. That’s where we need to put our attention, at some point even more than on the machine-speed sections.

Peter [19:20]: I’ve done value stream mapping across hundreds of organizations at this point, and I don’t think I’ve ever run into one where coding was the constraint in the system. And that’s well before AI. It’s just not typically where the constraint is. So, with that, I’d like to thank you, Dave, for an interesting conversation. We’ll wrap it up there and look forward to next time.

Dave [19:44]: Fantastic. Thanks again.

Peter [19:46]: You’ve been listening to Definitely Maybe Agile, the podcast where your hosts, Peter Maddison and Dave Sharrock, focus on the art and science of digital, agile, and DevOps at scale.