Estimating

Welcome to Definitely Maybe Agile, the podcast where Peter Madison and David Sharuk discuss the complexities of adopting new ways of working at scale.

Hello, Dave. How are you today? Peter, I cannot believe this is the seventh episode. This has actually it's been a lot of fun, and uh hopefully there's there's more to follow. But let's see what we're up to today. What are we uh discussing?

We're gonna talk about estimation.

You're kidding, right? So this is the we should have probably started everything with uh with estimation, shouldn't we?

Yeah, I estimate we're gonna have a lot of fun today.

So um maybe let's start with that. Um what are the what's the description of estimation or the definition? We're talking here about backlog refinement, estimating work to be done in a team in terms of estimating size.

I I think I think we can even go further beyond that. I mean, those are very I mean when this is an agile podcast as well, and but we're it's any time that we're looking at estimating like how long is this piece of work gonna take? Uh, or or even it doesn't even necessarily need to be how long, but how much is it gonna be, how or any kind of estimation about what the size of something is, and uh just how bad human beings are at actually doing that versus how good we think we are at doing that.

This is very true. Whenever I'm having conversations around estimation, I always use the example of uh um how bad we are at absolute versus relative estimation just by using an example of, and this is pre-COVID obviously, but being invited to a party and having a party game where you estimate everybody's height in inches or centimeters, and how how kind of incredibly out of touch our absolute estimates are with the actual response to that. Now, of course, I'll joke there that there is another version of that game, but if you start using things like weight or age, all I can say is you don't get invited back.

Yes, yeah. And it's um that that comes to uh interview questions too and things.

So exactly, yeah. But I mean, this is the you know, the starting point, if you like, is kind of joking aside, is absolute estimation, we're we're bad at it. Relative estimation, we stand a chance at going into that. And I'm again, if I think of any time a project manager has asked me to estimate how long it will take me to finish a blog post or how long it will take me to get some work done, what goes on in my mind is something along the lines of I think of a number which is possibly what I actually need, and I'll double it, or I'll add a buffer, or I'm going to manipulate that number even before I tell anyone, in order to come up with something that might be more reasonable. And I think this is part of the headache with absolute estimation. We're just we don't know how that is close to the reality of what's going to be required to get something done.

That's right. And unless unless it's something that we've done time and time and time again, so that we know that, well, every time I've done this before, it's taken me this amount of time. Uh and so I can I can, with a fair amount of confidence, say that as long as nothing goes wrong, it's gonna take me that amount of time to do it again. Uh but whenever we're working in a creative knowledge-based space, uh, it's next to impossible to do that because we're being creative. We're creating something. We have never done this before. So our ability to be say, okay, it's gonna take that long, uh goes right out the window. And so we're now really in this stage of I'm just gonna give you a wild-assed guess. Yet we rely on this, we run our businesses on this. We we, in fact, it's it's quite incredible uh how much uh there is this anticipation and expectation that somebody is gonna give you an accurate number and how disappointed they are when we don't meet them.

Well, but I think the the point kind of in that in that comment that you're making there, and we'll we'll come on to how to use the numbers and why people need numbers in a second. But before we look at that, the the fact is we're not estimating one thing. We're estimating a lot of different things coming together. And whenever I talk to teams about this and to help try and understand, I always describe the three dimensions. They're estimating that the actual effort involved, uh, whatever that might be. If I'm carrying boxes up and down the stairs, then I'm gonna I can kind of do time emotion studies and come up with what that number is. But then there's there's two other aspects to it. One is the complexity. If what I'm carrying up and down the stairs is in bags and they're paper bags, and things keep dropping out of it, so the complexity of what I'm carrying up there is difficult, or several things are very, very heavy and really difficult to move around. And there's the complexity side, this this uh uncertainty around complexity, there's a complexity piece that is really difficult to estimate. As you said, that's a an unusual, it's it's a uh um a critical part of the creative process and really difficult to get our hands around. And then the third thing is uncertainty, right? So we're estimating like effort, complexity, and then just generic uncertainty. I don't know if somebody's going to phone me while I'm trying to take things up and down the stairs and tell me that you know I need to go pick my dog up because the dogs run off and they're causing problems with whatever. I've got to go run outside and do something. So there's this uncertainty of something that might happen, whether to do with the work or to do with interruptions, that will impact my ability to get something done.

Exactly. And yet uh it I think one of the worst examples of this is of course, um, and now I'll go into that. I just play sort of like the sprint planning exercise where we come in and uh the whole uh planning poker, and we're gonna put numbers on things and we're gonna use that to say how much we can get done because we're gonna assign points, and now that we have points, this means that the people outside of the team can say, okay, you got this much done, and we're gonna measure that, and we'll call it velocity, and now we've got a an estimate of uh your success, and we'll use that now to track the team because clearly that's a good way of doing that, and that brings up a whole other set of problems, all of which seem to stem from this fact that, well, we started estimating it in the first place and tried to put a number on it, um, versus, well, some of the alternatives, which I think we'll get to in a bit.

Well, uh, and I wanted to pick up there, Peter, because that exercise of whether it's planning poker or however we have that, I'm actually I think it's one of my I mean, I enjoy it a lot. I think it's a really great conversation. But at the same time, what I'll often do to start that off is it the numbers don't matter. Ignore the numbers for the moment, because it's the discussion that comes out of that. And if I just use planning poker as an example, and in this shift to the virtual world, we've been using t-shirt sizing and relative estimation more and more because it's just so much easier to do uh if we've got a whole bunch of work to estimate and we've got a virtual collaboration tool like Miro or Mural, it's just even easier moving things around to do them around t-shirt sizing. But I miss the structured discussion and conversation that happens in a planning poker exercise because it's a really good way of getting the loud voices to kind of quiet down and the knowledgeable but potentially quieter voices to give them a platform to speak up. And it's a great conversation about what exactly does it mean to do this work? Where are the risks? Where are the things that we maybe don't know what the problems are going to be, and where are the things that we know pretty well? So I I find that discussion to be really, really good. I find the planning poker structure to drive it out, especially for a new team that's not got um a kind of a smooth, fluid way of discussing things in place yet, to be really valuable. I just don't care about the numbers that come out of it. If that makes sense.

That makes perfect sense. And I'm the same way. It's uh it's not about the numbers, it's about the discussion. It's about the planning, not the plan, as uh Eisenhower, I believe it was said. Uh it's there's a lot of that, and there's um there's and I think it's called the no BS uh planning poker. This is a variant on uh your your typical planning poker where there's just three different options. Um, and there's a not safe for work and a safe for work piece. I'll put the link in the bottom of the uh of the uh podcast for people. Uh but that that basic idea of uh like I don't know, there's no way we can do this. Uh and uh I can't quite remember the third one, but they're this this idea that this is really what we want to know, and this is the conversation that we want to have. Is uh uh if if I look at this piece of work that you're putting in front of me based on everything that I know, is this something that we're gonna be able to get done within the time box that we've given ourselves to get it done? And that then gives us an understanding or an estimation around we're comparing this to something, um maybe the time box, and saying, okay, uh based on that, my estimate is that yeah, this is doable or it isn't doable, or I really just don't know. Let's talk about it.

Yeah, and I I mean, even as you're describing that, Peter, you can hear it's a technical conversation about solutioning. About I always think of it as the technical due diligence. Before we get going, we're talking a little bit about you know what needs what do we need to do in the system, what do we need to kind of work on, what are the sort of different pieces that are coming together in order to deliver the functionality we're being asked to deliver. And that um that discussion is great. But the s the there's a whole other problem there, which is I have a product owner and stakeholders that needs to be able to plan. So maybe if we agree to this point, Peter, that the discussion in an estimation conversation within the team is valuable and we definitely want to see that discussion happening, but we also agree that the numbers that come out of that aren't necessarily um, maybe we're using them in a different way. If I go back out to the product owner and the stakeholders, what is it that they need from estimation? What are they expecting?

Yeah, and I think that's a very, very good point. It's uh the and this comes back to something, of course, that we always like to talk about, inspect and adapt, come back and look at it. What is the purpose of the action that we're taking? Is this serving us? Are we looking at things in the right way? Uh, why are we putting ourselves through this? Is it actually going to help us deliver better, um, more valuable uh outcomes? And uh this is all a part of that. It's the so what is it that the people outside the team need to be able to see to be comfortable that we're on track, that we're delivering to things we need to deliver to, that uh other concerns that uh are external to us maybe uh are being taken care of, and and this is often what the driver for needing these estimations is.

Well, I mean, ultimately they need to know what commitments they can make, right? Is this feature going to be in a release on a given date? Is you know, is this going to be delivered? If I go to a stakeholder and say the feature that you talked about is is just about to be picked up, can we give commitments on when that's going to be delivered?

Yes, exactly. And uh and the people need that to be able to say, okay, are we on track? Are we going in the right direction? Um and so this is important. So, like, so how do we suggest to the listeners how they go about handling that?

Well, uh, so in in the the situations where the there's a there's an interesting mix between accuracy and if you like, there's this feeling that if I give somebody a number, the number is eight, then there's an immediate pull of you know, that is eight days, or it's uh eight weeks, or whatever the number is, we're going to convert it to something, calibrate it in terms of time. So if it's eight story points and the delivery team does 25 story points in a sprint, then that going in there, I get some confidence that it's going to be done. But we start getting caught up in the accuracy. So I think one of the first conversations that I'll often have is a single number isn't accurate. The accuracy comes because I have a basket of lots of different numbers, and I can look at how well the team performs in delivering a lot of different items over time. So this is basically boils down to if I have 15, 20, or more uh different samples and I sample those, I'm going to get a normal distribution from it. And therefore, I can start making uh predictions as to how comfortable I am that the team is going to be able to deliver items similar in size to what's in this basket of goods that I'm using as a model. So the one of the first conversations I'm trying to figure out is stop looking at the single number and thinking that's an accurate measure in time. It isn't. It's a data point that I can I can use to make a prediction and I can look at an average and look at how things are going, but the individual sample isn't the accurate item that we think it is.

That's a very good point. And that takes us into that realm of statistics, and uh, another thing that uh people are uh people know the words but often don't really understand some of the statistical logic underneath that and that this is what you're describing is that one number doesn't actually give you enough information to possibly be able to gauge what the potential outcome is. You need to be able to monitor the system over time and look at its behavior to be able to start to draw any kind of conclusions as to what might be capable of. And there's also this understanding that variance will occur in that, in that if uh because the team itself, if we're looking, say, at a team's delivery capability, it's going to get better, um, hopefully. Uh and uh as it starts to take on a task, then they'll start to work together better, they'll get more efficient, they'll uh get more effective at delivering uh pieces, and if they're doing similar things again and again, they're going to presumably get better at delivering those and they'll be faster. So it's it's difficult to be able to say, hey, we're gonna take one point in time, take one number and say, okay, we know what's gonna happen next. Uh yet we still seem to have this desire to do so. Um I hear it all the time, in fact.

The desire, I mean, if we're planning in, you know, we've got the mindset, and this is comes from us for very good reasons from a project management sort of mindset, which is I have a work breakdown structure, I can estimate the time of each of the tasks, I can add them all up, and that tells me roughly how long I'm going to have to wait. It's in a sense, there's not too much difference to that in taking a whole bunch of work items with some numbers on them, adding it up, calibrating it, not using, you know, looking at how many of these, you know, how much velocity was done in the last sprint and assuming they can do something similar going forward. Where we struggle is then saying, and you'd better meet that, because of course it's there's a normal distribution over it. So there are, you know, 50% of the time, if it's just a straight normal distribution, 50% of the time they're going to be absolutely on the um uh on the the the they'll hit the nail on the head or they'll finish earlier, but 50% of the time they'll they'll overshoot and they'll take a little bit longer. And of course, for an agile team, we don't mind the overshooting. We know it's going to air backwards and forwards. Sometimes they'll overshoot and sometimes they'll finish early. But if we uh if we kind of get obsessed about focusing on whether they hit the number, we're losing the fact that there's a lot of learning and uncertainty and things which are moving around. And one of the the if we need to worry about that, we can look at things like um, you know, standard deviations basically and decide how much risk, how much uncertainty we are comfortable with, or whether or not we kind of build in a buffer and just say, you know, we we need the team to aim for two-thirds of their total capacity because that gives us the confidence that 90% of the time they're going to be able to complete everything within a spring.

Right. And that's the this the longer we drag those estimations out, the uh the further into the future that we're looking, the less and less likely it is that we're gonna possibly be accurate, the more risk we're introducing into the system. So the if we have cascading events that we've we've got estimates on each of those, which is your your typical project plan, then that's your the further you go, the less and less likely those things are gonna happen. So just being able to look at what do we have right in front of us now, uh, how can I comparatively estimate those and understand where I'm gonna go next, and then start to build on that is a is a better strategy when you're dealing with something that we can't necessarily know um what the impact of these things on each other is gonna be or where or how any of these individual parts are gonna occur.

Yeah, a metaphor, Peter, that I've been using more and more is drive. Canada is a big country, right? So if I look at driving from Toronto to Vancouver or Vancouver to Toronto, um when we talk about something like that, there is there are several days worth of driving going on there. And what's interesting, if we tried to do that, maybe it's our once-in-a-lifetime Trans-Canada trip, we we can plan the first days worth of driving or the first few days worth of driving pretty well. We can look just slightly ahead. We might even book a hotel or we might meet some friends at a restaurant or decide where we're going to stop for coffee mid-morning and mid-afternoon or whatever it might be. We can make those predictions pretty well because we have a fair idea of what's going on. Now, the the interesting thing is if we try to look at the end of that three or four-day road trip, there's very little certainty about where we're going to be because we know many things might happen. You know, I might have a puncture, there may be construction or weather may delay us. We might see a friend that we've not seen for a while and decide to hang out for a couple of days before picking up the road trip again. So there's a lot of more uncertainty the further in the future we look. First of all, and I find that analogy is relatable. And if we can look at it from a view of backlog management and talk to stakeholders about that, they tend to be a lot more comfortable about vague, vaguer promises in the future versus more accurate, more, more comfortable commitments nearer, you know, in the near term. For example, is there any way you might take that metaphor for other aspects of estimational planning?

I think the I I like the metaphor, I think it works very well. It's there's the the not knowing the future and the we plan out the bits of it that we can possibly see. We want to provide enough uh enough certainty for uh, hey, what what can we do now to uh reduce the risk, to mitigate the risk, so we might like pack a spare tire, or we might uh ensure that we've um got enough food, or we might look up what uh hotels are available in certain places so that hey, if we do decide to take this uh side trip, that we know where we might possibly stop, or we might make sure we have the right capabilities. So we there are things we can do ahead of time that will give us options, um, which brings me to the whole that that takes me down that real options path and looking at uh like at what point you need to actually make the commitment to the decision you make, and then our desire to have uh enough information. And do we we often overcommit and want too much information before making the decision, um, which is another thing I think which leads to uh the estimation problems because because we we want more information and so we spend more time estimating, and uh whereas if we actually just took the the action so we could learn, we'd probably be able to uh get further faster.

Yeah, I mean everything that you've just said there, Peter, that that feels two things are hitting me my ear as I'm listening to that. Number one, it's about the conversation, the planning, the discussion. And you're just describing how much planning do we need? Do we need to pack a spare tire? Do we need you know a flask of coffee or whatever it might be? How much how many days are we packing clothes for? Whatever those conversations are. But the other thing, and that's one of the reasons I like this metaphor, because as at some point somebody's gonna stand up and say, hey, let's stop talking about it. Let's jump in the car and get going. Right? And and that's exact that's that piece that we want to get to. We want to do enough conversation at the outset to make sure we've kind of thought through the things that we may need to worry about. But then let's actually learn by hitting the road and moving forward and start seeing what progress we can make and how that's working.

Uh yeah, I really like that. Now so uh I I think uh and we I knew we'd talk about this topic for uh for quite a while. So we'd like and we're now at like uh like 22 minutes. That's uh we're uh we're deep into this uh conversation. So are there any other points you'd like to add?

So first of all, 50% of the time we were going to be under 20 odd minutes. We're on now we're on the 50% when we're over. I'm good with that. That's part of the fun of the uncertainty of it. But to be fair to our listeners, it probably makes sense to uh kind of wrap things up. Um what strikes me when we Talk about estimation. I guess there's a few things that come to mind, and we talked a bit about accuracy and pushing that to one side, and talked a bit about how much planning is too much planning. But the two things that that resonate with me to start the conversation around estimation is number one, that that uh planning is not the plan piece, I think is the way you put it, but the discussion that happens is important. That discussion has to happen somewhere, and uh that technical alignment discussion, I'm a huge fan of that. And if it's happening through estimation, great. And if it's not, it needs to happen somewhere else. And then the second thing is the number doesn't matter, right? It's it's a tool, but the number really doesn't matter. Anything that you'd add, Peter.

I I I love those points. I think uh yeah, that comparative estimation where we're saying, okay, uh this is small, medium, large, or is the uh the piece that uh I I uh was gonna put into the uh the comments of the the podcast there around uh is this something we can achieve? Is it something that there's uh and the the safe ones were too frighteningly big and no faintest clue? Um so and that but that that point of like comparatively, is this something we're gonna be able to do? And let's have a discussion, because if we say there's no way, what why do you think there's no way? What is it we could do to change it so that we can actually achieve this? So um and I think that that kind of wraps things up.

Excellent. Another fun conversation, Peter. Always appreciate it. Until next time.

Until next time. Thanks. You've been listening to Definitely Maybe Agile, the podcast where your hosts, Pete Madison and David Sharp, focus on the art and science of digital, agile, and DevOps at scale.