Working with legacy systems

Welcome to Definitely Maybe Agile, the podcast where Peter Maddison and Dave Sharrock discuss the complexities of adopting new ways of working at scale. Hello and welcome to another exciting episode of Definitely Maybe Agile with your hosts, Peter Maddison and Dave Sharrock. I always feel like we're going to prepare a lot more for really exciting episodes of my radio has a voice on you know, let's go. Maybe we're doing press-ups or something before we get going.

Ryan, what what are we here to talk about this week?

Uh well, this this week we're gonna talk about um an interesting topic actually that uh you brought up that had come up in a uh in one of your trainings, but uh is a topic that I come across a lot uh in my field as well, which is like when we've got a big, monolithic, horrible, hairy, complex system uh of uh legacy spaghetti architecture, and along comes this uh planned legacy spaghetti architecture. Well planned, yeah. I mean it it's it's naturally grown this way over time. Um and we come and we talk about hey, we're gonna do all of this uh sprinkle some agile pixie dust, sprinkle some DevOps Pixie Dust, and we're magically suddenly gonna be able to incrementally deliver stuff into production, we'll kill CAB, we'll fire all the architects, we'll uh get rid of all the project managers, and now we're all agile, and it's fantastic. And now what happens? And how come it didn't work?

I was gonna say anybody listening, don't do what you just said, is because there needs to be a warning label on it. But I think what you're you're talking about is big change, you know, replatforming or or some new system that comes in. One of the quick questions, I mean, on the you know, simple conversation is we're going to deliver incrementally. So you come along with a big platform, big change, we go, well, we still want to deliver incrementally, but you want to add something before we even have that conversation.

Because the the immediate piece you run into is, well, we can't do that here because we're dealing with this big monolith, and it's got 150 million lines of code in it. And um, the people who started writing this have all long left the company. Um, we are continually dealing with this. It takes us six weeks to test this. Um, we've got that there is no way we can incrementally do this. We've got to do these massive integration tests and validate everything because we've got 10 teams all modifying different parts of it, and all the business logic is tangled up in all of the code, and it's a mess. So, how do we deal with that? And just pull the cables out of the back of the data center and away you go. Yeah, that's actually probably one of the best ways to deal with it.

I think actually. Well, I and I I think one of the first things to recognize, and this uh is something is I sometimes feel like uh people want simple solutions to complex problems. And a simple solution is we deliver incrementally. Complex problem, the one you're describing, is just we we've got to to reach balance. We've got to understand the principles. What's the intent of delivering incrementally? And how do we transfer that intent, those principles, those guiding, you know, the the because in in many cases the intent that we're trying to achieve is solve complex problems through making small steps, small changes, because you maintain stability, you maintain control of those changes. Now, if you're in a situation where it's actually already making a small change is the equivalent of making a big change because it takes six weeks to get any change out, then there's a there's a some sort of mediation route that you've got to get to, remediation route, to get to a point where you can get the intent. So the long way of saying the first thing being understand the intent of your simple answer rather than totally ignore the simple answer. Yeah.

And and this, I think, is an underlying what you're saying there is that it takes time to do that remediation and it takes effort and investment and investment and learning and a lot of thought. And that's the part I think where a lot of people start to come and stuck with the it will never work here because they're also all running at 120%. They don't have the capacity or the the bandwidth to think about what it might take to do this, and so that there needs to be space given to take a pause, take a step back, and consider okay, yes, true, it is a big, complex problem. Uh, now let's think about what do we want to do about that? Uh and what could we get out of this if we do? And how would that benefit us and the overall system as a consequence of us being to do that, to be of doing this work? So, where is the biggest problem that we have right now today? It is the biggest problem, those six weeks of testing, because if it if it's six weeks of testing, but you have a beautifully designed release plan, and you can you constantly are able to execute this, and that six weeks is always that six weeks, and it's all predictable and it's well laid out, and that, but it takes you 12 months to decide what it is you're gonna build, then tackle the 12 months. Right. So there's that that's one that's one possible piece of of Sonja, but that kind of feels like we're still dodging the okay, how do you tackle that big monolith and start to break it down into pieces?

Well, but but I I'm not even sure that we're talking about breaking it down into pieces, because in many organizations, what you actually find is that that monolith is going to be replaced by a new platform. That is so the replatforming is is literally you know a lift and shift from the old system to the new system, whatever that means. Of course, running the two in parallel becomes, you know, maybe that's off the table. I always like to say, is that off the table? Because actually, if you're moving piece by piece, and I've seen great success where you're slowly carving off. And to your point earlier on, which is what you're hinting at, I think, which is we want to use business value, that intent again of de-risking a problem, but also of maximizing the return that we get from it by looking at which part of this huge challenge is the biggest headache. Let's solve that one, let's break this big thing down. I'm actually reminded of all of those uh books about cleaning up and sorting everything out. And one of the ones that always stuck in my mind, which I've used to great effect, is you take all the stuff that you're trying to sort out, whether it's your desk full of filing and so on, and you split it into two. And then you take that next split it into two, and you carry carry on doing that until you're sitting in front of a manageable pile of stuff that you think is the most important of all the piles you've now separated out. Not suggesting we start doing that with lines of code or anything, but there is an element of looking at algorithms, but yeah, but there's an element of just looking at it and going, okay, which of these two problems is the biggest problem to solve? And given this is the biggest problem to solve, most vital, urgent, whatever it might be, continue doing that until you know what you're carving out. I like that sort of microservices and the whole decoupling that you often see where you're carving out the functionality you care about or you're using the most, or that's front of mind and needs to be done next.

The stuff that you want to change the most, yeah, the stuff that's going to be most frequently changed, and that understanding of how these different pieces connect together. So there's the answer comes is that it depends, essentially, is what the answer to this question comes. Depending on what is needed to move you forward in the system. Small incremental changes are easier to do on smaller pieces of work where it and it what it comes down to is what is the cognitive load that I need in order to be able to understand this in its completeness to be able to make this change. And if my if a single team cannot understand this in its entirety, then it's too big, too complex for that team to take on to make that change independently, which means that it's it's going to be too big from that perspective. Now you also, though, run into this other side of this, where you it may be okay to have that monolithic piece because you there's only because if it doesn't change very much, then it may be easier to maintain and manage that. Because you've also got to consider, because it's a system, that the when you start to have many tiny little pieces to manage, that there's a lot of operational complexity. And you've got to be very cognizant of the fact that by breaking it up into smaller pieces, I haven't just created another headache for myself further down the line. Because now I've got to look at the complexity of managing the interconnection between all these little pieces I had. Whereas I only had one thing to manage before, and I've got a hundred.

And let me pause you just for a second, I too, because I just want us to take a moment to put a pin in a couple of things that you're touching on. And this is obviously a topic that's close to your heart. But um, but let me just pull out two things that I heard you talking about there. The first one is do you have to change the system? It may be monolithic, it may be couple and spaghetti, whatever, however, we want to describe that system. But if we're not changing that system very frequently, if it's pretty stable when we do change it and we can we can incrementally kind of add bits and pieces to it. And I'm thinking a lot of green screen uh tech, a lot of you know the mainframe sitting in at the core, right? In the somewhere deep in an organization which has been running pretty solidly for decades. How much of that right now has to be changed? I get it, if you're trying to innovate.

You can definitely implement DevOps type practices to improve the delivery on platforms like that without a shadow of doubt. I've done it many times.

But and I'd say agile practices as well. I don't want to limit it to DevOps, but I think what's interesting there is that um, again, not getting too um deep into that one, is that there's a lot of benefit you can bring to it, but there's also there's a lot of, you know, many times it's really like let's not rock the boat too much on here, in some, you know, in in many cases. I also wanted to bring up the other point that you were talking about. What was really interesting when you were describing that is as I was thinking, I've been listening to, you know, looking into sort of the big tech companies that everybody looks to right now, Fang and and these other companies, Facebook and Amazon and so on, Netflix and all the rest. And what's interesting there is they're they're sort of from the ground up, they've built so many of their products in this very decoupled way. So you get your two pizza teams at wherever it is, managing, you know, a single thread throughout the whole system that is, if you cable, you know, bring them all together, there is a big cable. And what you're describing is going back a couple of generations in some ways, but looking at those big systems and breaking it down cable by cable, piece by piece, to be able to tease those functions out. Aligning, I mean, there's there's this sort of alignment that's happening there between yeah.

Well, well, we can we can start to talk about heterogeneous and homogeneous architectures and when you need the uh the actual pieces to be aligned. Because this is one of the other big advantages that some of those newer hyperscale companies have is they have generally a fairly homogeneous way of architecting things. And and because those commonality that has enabled them to really accelerate the way they do the drums. But by the way, I'm working with a bank at the moment who actually has some of those sort of key engineering principles underlying some of the systems they've built at scale, which has indeed helped them accelerate because they have this these common architecture standards that have, well, actually, it's not quite true, they don't have the common architecture standards to support it, but they have the the comp these massive component platforms which are fairly consistent that have enabled them to build on top of that. So if you have a common platform that is solid enough that you can start to um experiment and innovate on top of, that can again start to accelerate.

So again, what I'm gonna do is just because I think you know, for people listening in, we're you know, coming up from maybe a couple of minutes before we wrap up. But what I'm looking at here, that pin that I want to draw out, is we started this conversation about intent behind this simple thing of deliver incrementally. And you come in with a big problem and you say, there's no way we can deliver this incrementally. And what's interesting is we've gone through our conversation and we've looked at various things coming into that, is we're just ending now where you're talking about architectural principles from which you can then make all of these decisions going forward. So I find it quite interesting that starting with intent and ending with those intent, the principles. I think that just speaks in in many ways. This is something that I find pervasive in the agile and DevOps space, which is the first thing that people start talking about. Or what are the principles that are guiding the decisions that we're making and how we're going forward? So maybe, I mean, anything that you would add to that?

No, I think that I think principles underlie or we have to agree on the principles we're operating under. Um the I well, and this is not exactly a principle, but it's one of the first questions I always like to ask when I'm looking at this from a strategy perspective, which is what prevents us from delivering this today? That's uh something that uh I know um other value stream architects and friends like uh Brian Finster like to talk about things like that as well. It's uh because it gets you thinking in the right way.

Well, and I think I mean this falls back to what we always talk about, which is change is rapid, it's happening everywhere. So there's no point doing something in two years when you can do something today. It's that that whole piece of it. And if we then bring in that whole bit about the principles, I think we're really beginning to see how you know it's a it it it comes full circle in some ways that some of these conversations really come back to understanding intent and principles and then building what is actually quite common sense decisions and practices and ways forward on top of that, whether it's what can we get out of the door today, which is another principle, you know, deliver fast from the lean world, or whether it's the you know, some of the other decisions you're making about how you break down this monolith, do you need to?

And there's there's another piece around the particular question you're asked here, which was around how do small incremental changes uh occur when you're doing complete systems replatforming. So there's another piece here. Uh the, I mean, one of the common ways is you you put a um uh a shin in front of it and you say, okay, uh if it's this, it goes to over there, and if it's the new stuff, it goes over here. But from the interface at the start top, it looks the same to both. And you start to build the new stuff in parallel to the old stuff, and that's how you start to say, I've got two systems, I'm building out in parallel, and I'm moving pieces of the logic across to the new world as I go along, and this becomes my new system. So that type of um replatforming in that manner, which takes longer, but it's less disruptive. Um, whether you're gonna do that or you're gonna try to do a massive big bang, which I would highly recommend not doing, although I I know of some that are about to happen, but uh you're you're that's a that's a good way to incrementally start to move from one system to another.

What I find interesting, because what you're describing, we we definitely need to pull this together, but what you're describing there, just to touch on it, is um that costs more, costs more time, it costs more, you know, more effort, and so on. And I think what's interesting is we really want to be thinking about this, is there is what in a in a complex, rapidly changing environment, even though that costs more, it's still probably the smartest, best route forward to limit your risk and to be able to take advantage of some of the changes that are coming through. And again, you and I can both tell stories. Organizations we've worked with where that's worked, where they've followed that through and done it, and organizations which chose not to, and the cost savings that they were trying to prevent having to spend on were actually dwarfed by some of the impact that came on later on.

And yeah, for any listeners, look up the uh lift and shift shot clock, where it's the the benefits you thought you were getting by taking the shortcut uh are rapidly eroded to the point that uh yeah, you lose them all. We've all had that experience for sure.

So yeah, yeah. Uh okay, shall we wrap it up there then? I think so. Yeah. How would you like one or two things just to say, remember this, remember that? What would you say?

Uh I I think in the in this instance uh of replatforming, think thinking about what the end-to-end system needs and are you focusing in the right area to me is possibly one of the most key pieces. Um so that you're you're treating the problem space in the right way. Uh, the other, the other part of this, I mean, I do being able to do small incremental changes to the systems you are working with is it is going to be life-changing for everybody involved. It's going to be very empowering, it's going to enable your teams. Your teams are going to love what they're now able to do. And so it is, it is definitely the goal you should be targeting. Uh, how you how to get there will depend on the systems you're working with and how you're going to how they need to change as you evolve. There you go. That's my perfect. That's my two-sentence answer to this.

Excellent. Well, thanks again. As always, Peter, it's been fun. And uh look forward to our next chat.

Remember to hit subscribe.

Yes, uh, and comments below. Always uh add any questions you may have, maybe experiences that you've had in either things that worked or things that perhaps you learnt a lot from.

Yes, uh, you know, those big bang wee platforms that blew up in your face, those side things. Until next time, thanks a lot. You've been listening to Definitely Maybe Agile, the podcast where your hosts Pete Maddison and Dave Sharrock focus on the art and science of digital, agile, and DevOps at scale.