ChatGPT for Agile Transformation

Welcome to Definitely Maybe Agile, the podcast where Peter Maddison and Dave Sharrock discuss the complexities of adopting new ways of working at scale. Hello, Dave. So here we are again for another exciting episode.

I'm looking forward. I was just thinking as we're coming into this, the topic today is about AI and Agile. And I'm as we're coming in, I'm thinking, I think we can be replaced. Maybe we just cut this off and go to find one of the many tools out there to put a conversation together in the in the tone of uh Dave and Peter chatting on a Friday afternoon or a Thursday morning or whenever it is that we chat.

Oh, exactly. That's it. It's like, as Dave and Peter, write and record a podcast about this topic. Actually, no, not even about this topic. Just go pick a topic and then decide what you want to write about.

So I mean, I I just came from a conference this week, and and there is so much conversation in the air about what AI is going to do and disrupt and and so on. And what's really interesting, so the the it was the scrum gathering. So the scrum gathering uh out in Portland, Oregon, and it's filled with agile coaches and scrum masters and people going through that sort of a transformation. And of course, there's sort of a lot of activity about whether or not AI can replace your agile coach. So maybe if we just started with that sort of lingering question in the air, what's your thoughts on that?

It's interesting because uh there's there's definitely examples in the marketplace where people have uh started to configure a large language model in the form to act as a coach. Uh the the general feeling is that it helps and it can actually do quite a reasonable job up to a point, but it isn't a replacement for sort of full-on therapy or anything like that. It's not a that so this idea that but it can can actually do quite a reasonable job of a lot of the kind of the simpler pieces. And there are there is, of course, a lot of nuance here. It's uh it's the making sure it has all the right context, uh being able to interpret what it brings back as um is this the right thing or the wrong thing? So I I think there's still an element from my perspective that it's a an augmentation to the capabilities. It can is an accelerator, it's something that can allow us to go much, much faster if used well.

I just I I've got to draw you back right to the beginning of that statement, which is where you you suggested uh AI is not really a uh set up for therapy. And I just want to clarify, I am not sure agile coaches. So I think that's a maybe another conversation. But uh let's, I mean, sort of joking aside, I think we need to be careful about where where the line is drawn and what the sort of help is that you can get uh from your uh friendly neighborhood enterprise agile coach, whether AI or human.

Yes, and I I certainly didn't mean to imply that uh agile coaches provide therapy. It's more is more the the what we see in the industry and then how this technology is being applied at an incredibly rapid rate, um which is also the kind of the interesting piece too.

So I don't think the pace of change is accelerating, and that's that's exponential change. And so so one question would be should we sit back and wait for the AI coach to fine-tune itself and become better and better and then bang, take on the the one that wins the rat race towards becoming that tool? And I would argue that the answer should be no, absolutely not. I think we need to try out those tools and try out the uh this, as you said, as an augmentation rather than I mean the idea that I'm going to let go of a scrum master and replace that with an interface and say this is uh chat GPT slash whatever tool that you want to use, Scrum Master or coach, obviously at this stage I'd say probably not a great move. I do like the idea. If you say the word, like view it as augmentation, I think it's really quite interesting. And quite frankly, I mean, we all know a lot of the role of Scrum Master or team coach or of an enterprise coach is reflecting back to the people that we're working with, what they're saying, so that they can view it from a different lens or consider what's being discussed. For for example, there's also a lot of very sort of standard situations that you'll find yourself in, like a like a sprint planning, number of other conversations, say estimation conversations from refinement, where I think there's plenty that uh an AI coach can take care of.

Yeah, there's there's a lot of there's a lot of that sort of work which uh can easily be taken on and advice and guidance, and a lot of those pieces to uh there where the the AI can definitely do this and it in a lot of cases uh do it a lot more consistently than uh than humans can. Uh so there's definitely those sort of areas. And uh I mean when you go and talk to people in the industry, I mean there's there's a this scale between uh and the fewer of these I'm finding this day where say, well, it's all a hoax and we can forget about it all the way through to they're gonna replace all of us. Um and I I somewhat fall in the in the camp of uh, as I was saying, uh people and their robot friends, where it's the the the positive outlook of uh the the AI helps and augments us and allows us to accelerate and get better and learn faster and remove barriers faster. Um I I know from uh talking to a friend of mine as soon as ChatGPT 4 came out, which was a big step forward, even though it didn't come out that long after uh 3.5, which allowed it to base both take in images and to be able to go out and read websites and then improve its results based on what it went out and read, which caused a whole bunch there's a whole other area of like hey, there's some other connotations of that. But but if we if we take that in the positive light, uh this idea that, hey, write me 10 stories to build a banking app. Uh now uh go read this website over here and improve those stories. Now split those stories into smaller pieces. Now go and write the code that satisfies those. Uh go write the test, or even the other way around, go write the test and then the code. So and it will go away and happily do all of these things for you.

I I mean I'm reminded of uh the sort of driving aids, the augmentations that we have in many of us in our cars nowadays, whether it's parking, self-parking, whether it's uh you know lane control and all of these other tools and various uh generations of them. And I think uh in all of the cases, we have we we're involved. We're there ready to grab the steering wheel, or we're there to monitor what's going on and so on. And the the whole point is that we should be there today with the state of what's what that technology can do now. And I think when we look at AI as a coach, the idea that that can disappear off and coach a team without oversight, without being an augmentation of an existing coach, I think is really something that we that we want to think of it as that augmentation. I do wonder whether there's a lot of conversations that if you think of it as a sort of, and I'm thinking of the coaching conversations which become deep or emotional or sensitive. Those conversations as they start, they start in in a similar way. And I guess the part of it is that being there to step in and and overtake. That's where I it I see it as an augmentation of maybe just getting things up and running and so on. But there has to be that that there are points at which, and this is this touches on the therapy comment that we started with, right? There are points at which where we've all been in, where we've got to sort of slow things down or really tightly focus in and separate out some of the key issues that are going on. And I'm not sure that I it's not that I don't think an AI tool couldn't do some of that. I think it would be it would have to be something that which is either very much um kind of like uh a human interaction in the middle just to make sure it's going in the right direction, or some other knowledgeable expert. Because I think the the the room for harm there is much higher in those particular charged conversations that I'm maybe thinking of.

I I I would I would agree there there is a there is a difference, right, in terms of some of the conversations uh that you're listening to. Listening to tone and not and understanding and being able to see how somebody is responding to what's being said, seeing how the interactions are going, be able to speed up or slow down and measure whether the the understanding is truly there. That that aspect of coaching, a large language model, which is essentially what these are, which is just a big database little word technique, isn't gonna be able to do that. There are other aspects of AI out there which are working in that space. There's very there's other companies doing that, but if we're just talking about kind of the chat GPT all over the world, it isn't going to be able to do that aspect of coaching because that just isn't a function of uh the capabilities that it has.

But but what I um and I this is certainly what we're seeing, I think, is as a tool to accelerate. For example, you were talking about the user stories and getting the coach. I mean, that's that's a fantastic accelerator that can leave us, you know, with what would have been weeks and weeks of work to get the foundation of that laid out in probably hours, if that. And now we're just way further down the line. And that from a and what's interesting, I mean, everybody's kind of aware in the background that the productivity growth that we've been seeing has been stagnating for some time. This is part of that headache with things like remote work and whether people need to come back in the office. And the challenges that a lot of the leadership are looking at are less about individual conversations around particular offices, but more about how do we jumpstart that productivity growth again. And I think this is a huge, huge. I mean, you just describe a situation where many weeks of work gets cut down into, let's say, a day or a few hours, and we still have loads of work to do on top of that, in terms of perhaps validation, in terms of alignment, with with a broader picture that just won't be visible to the AI accelerator tools that we're thinking of. They're the augmentation, but there's a lot there around productivity improvement, which would be quite powerful if we get our head around it, if we if we understand how to use it in that way.

Yes, yeah. And then I mean, for from some schools of thought, productivity hasn't uh moved forward in decades. That uh we've uh but there's a whole other more macro conversation around that as the as the we've we've created things, but we haven't actually necessarily moved our ability to produce more as a as a society forward in a very long time. And so tools like this start to indicate where all of this might be leading that might actually start to uh create some of those productivity gains if they're targeted at the right sort of problems and actually trying to help us move forward faster and solve interesting problems. So there's definitely value from that perspective without a shadow of a doubt. So I I think there is a lot of value in these tools. Uh, and there there is, like, as uh anybody who goes on the internet, you'll find hundreds and hundreds of them doing all sorts of different nuances of different tasks, right?

I I just wanted to add in on this one because the other thing is, and we've we've talked about this when we've had our conversations in the past, which is you step in and you see people who are less experienced than they perhaps think they are, or to the position or the role they're playing in. And so you end up with scrutmasters and coaches perhaps who aren't performing at the right level. And I think one of the things around those that AI augmentation type of thing is it raises the bar for everyone. And I think ignoring the top end where you want innovation or or you want those nuanced conversations, which are very, very personal and specific and contextually relevant, and looking to the other side of things about how do you get, for example, the role of the scrum master really just raised up so that there's a consistency across an organization. I think there's a lot of opportunity or s or or situations where that might well be a great place to go and use that sort of AI coach augmentation.

And I I I think it goes beyond coaching in that respect. I think it's uh every role really in a lot of organizations.

User story, I've we've got a thing, user story writing, training, documentation, technical documentation. I mean, there's loads of the these things that honestly we we've all we all know when you get to the end of a day and you go, Oh, really? I've got to do X. And that there there are things which are, how would I put it, mundane but necessary. I mean, think of many of the items in a definition of done for a team, for example. They're essential, they're important, but they're also somewhat repetitive or mundane, or not really what we feel that we're there. You know, what's not not really the the kind of gets the creative flow in our mind. It's not what we want to be doing.

Yeah, or even just not having to start with a blank sheet of paper and uh like get getting you from from zero to somewhere, and and that initi and I mean there's lots and books around that, but this idea of going from zero to one, like it's the that's the hard button.

Yeah, for sure. Yeah. Now what about okay, let's spin the coin the other way around. What what can go wrong?

So so here, uh and we we touched on some of this before, is that if I mean if you're getting if the advice isn't nuanced to what the actual situation is and it isn't interpreted correctly, then it can cause uh harm in that respect. You could argue that same could have come from a human coach, though, but but I I I yes, it depends, right, on your point of view whether you think it's more or less likely. Uh but but another one that uh and I I've seen this repeatedly is that when I if I go and ask Chat GPT a particular set of information, um especially if it's got numbers in it, it will quite happily tell me a bunch of bullshit. I mean, like lies, downlies, and statistics, but it will lie through its teeth and tell me all sorts of stuff that I know blatantly isn't correct. Uh so there's that side of it too, having enough understanding and situational awareness to be able to look at what it's giving back to you and being able to say, Well, I I happen to know one of the fun things I like to do is like ask it like, uh, when did these two companies come together? Well, I was asking it about a standards the other day, and I said, Well, what's the most current version of this standard? And uh it said it's on version eight, and blah blah blah told me all about it. And I said, Well, it's not, it's on version 11. So I I just went, try again, uh, go read this website, try again, and it came back. Oh, it's on version 10. I said, Nope, just try again. It did apologize each time though, which I thought was quite nice. It said, uh, no, sorry, I got that wrong. Now let me try again. Canadian AI that you're dealing with, obviously, there. Yeah. Clearly, clearly. Uh, but but that that type, that's where I see some of the mistake. It there's no confirmation piece in that.

Well, and that kind of touches on another element that I was thinking, because it it needs to give a response, right? And I think if I think of the number of times as a coach where actually the right response is no response, and that really raises that whole thing of is is uh an AI tool that is geared up to give a response wherever it gets that response from or however it finds it, maybe there's already and it this is somewhat the the age-old debate of coaching versus consulting, potentially, but it I I really like the examples that you're sharing because it's it's not just about needing to give a response, but it's almost the uh uh I don't know if it's an inability, but it's not able to say, I don't, I'm not sure I can tell you what you need to what you're asking about, or I'm not sure there's a response here that would be suitable. I'm sure that's going to change if they can apologize already. It's only a matter of time before I can say I don't know.

So well with with all of these things, I mean the counter argument is that well, if I went and asked a person, they might tell me that it was version eight, right? Because they might be not know that there is a more recent version. And so am I being too critical of it that it didn't know either? Yeah, I mean, yeah.

That feels to me like one of those questions where the correct response is no response, right? Yes, yeah. I think uh uh one of the other things that always uh, and I'm not sure if this is a valid concern, but is the sort of smoothing out of the unique pieces. If you think of any sort of response where I'm gonna go back in and look at how others have responded and pick, you know, some version of the most popular slash you know, common response. The most common response, or the the one that probabilistically is the chosen path through a particular conversation, misses out the opportunity for unique experiences in those conversations that might um and and I'm is this is another question of do coaches do this either? I'm not sure. There's a lot of confirmation biases. We have um we have wonderful stories that we want to share, or we have wonderful experiences we want to guide people through that perhaps are equally not unique. I don't know, but I'm not sure how those confirmation biases differ from the confirmation biases that you'll get in an AI tool.

Yeah, well, and there's some interesting pieces there because with the AI tool, I can say, and this is one of my favorite ones to play with, actually, is uh be more such and such. So be more Dave.

Okay. And is this where we come clean and say that my side of the conversation has been AI all the way through this conversation?

Yeah, it feels like it, you know.

Uh yeah, it's I should probably have let you on let you finish that sentence that you were just saying. So, yes, but you can say be more Dave, and then it comes forward with all my cognitive biases straight up, right?

Yeah, well, exactly. And uh and and for for people who have more presence. So, like I think uh things like Elon, for example, be more Elon. But but this this idea that you can tell it to take what it had and rephrase it and reframe it based on a particular uh nuanced set of uh sort of biases. And it and that when you start to actually look at how that's working under the covers, there's this interesting interrelationship between language and the world model that it's built for itself, and how because theoretically, some of the things that it responds with, it's surprising that it responds in the way that it does. Um and it's actually what it is doing is it's deriving a version, a its world bias, its world model from how we use language and how we string words together, because that's essentially what it's using from a response perspective. So um it's quite fascinating from that perspective when you think about it.

Um I when I do a lot of leadership work, and when one of the things that I feel are is really, really powerful in that leadership work is understanding how language, the words we use, matter. And I do wonder, given that I mean, part of the reason we focus on language and the words is because it's rare, for one thing, and it's also very, very important in how we communicate and how certain messages get across. So uh, and I do wonder whether this is helped or or um squ pushed into the background with the AI augmentation sort of things that we're talking about right now. Uh whether it's you know brought up to the surface as being language is important in these particular circumstances, this is how to use it, or whether it gets pushed into the background because that's rare, and therefore most of the time a different response is being given.

Yeah, it's it's that's an interesting way of looking at it. It's uh if you're getting the it's almost your thing, the tragic tragedy of the commons type thing.

It's like Tragedy of the Internet as we find it, which is how about uh we kind of uh pull a few thoughts together. What are the takeaways from that conversation?

Uh so I I think uh the top takeaway for me is that uh I would much rather look at this as a way of uh people and their robot friends augmenting our ability and accelerating our ability to get better faster and to solve problems. And so that that's the piece I think that's critical to focus on. Um I think that that's probably one of the main takeaways. I think uh there's a second one for me would be around there's still a need to understand uh what it is you're being given so that you can uh have a sensible conversation around is this the right thing? Because not everything it's giving back to you is necessarily correct. Uh, there's definitely um even from a coding perspective. I mean, people there's plenty of examples, people who said that it gives them back code, but the code isn't right. However, most of the time it is. So I mean, but it it's probably more it's probably accurate more of the time than your average developer.

But yeah, that's uh Well, I think it's if I just um I wanted to reinforce the the sort of augmentation side. I think that is is something to take away. And and and I'd add to that to say probably not probably, don't wait for a product that is going to somehow meet an imagined need. Uh go try it now, but do it in the same way that most of us in our cars use the car driving AIDS, which is with our hands ready to take back control when we need to. Um, and I think that would be the key takeaway. I don't think we should be sitting on our laurels waiting for the perfect product to come out. We want to be augmenting what we do in as many creative ways as we can, but ready to take back control should we see the need.

I think that's a good way of summing it all up. Um so with that in mind, should we say uh hit subscribe? Don't forget that. And and send us some feedback at maybe agile. Oh yeah, for sure. Because uh hopefully this sparks some uh some ideas and interest. And if people want to chat about this type of stuff, we're always happy to talk.

And um just maybe one thing I'm gonna add, Peter. Uh we're we're beginning to have guests join us. We've got a few lined up in in the near future. Uh so if anybody listening has a perspective for something that they'd be prepared to kind of have a chat with us and uh get join into the conversation, then feel free to reach out. Fantastic. So until next time. And good to talk to you, and looking forward to the next time.

You've been listening to Definitely Maybe Agile, the podcast where your hosts are Peter Maddison and David Sharrock, a focus on the art and science of digital, agile, and devil scale.