I’d like to let you in on a secret: I don’t believe in projects! I believe it’s a bad idea to use a project to deliver or maintain software.
We talk a lot about products in agile. For example, we call our backlog a product backlog, not a project backlog. We talk about our teams as product teams or feature teams (or possibly component teams), rather than project teams.
Still, projects still seem to be the norm for how we deliver our products and any sizeable additions or changes to them. And let’s be honest: a lot of agile teams successfully deliver awesome products using projects. However, even in an agile world, a big number of projects fail. 38% of all Scrum are not classed as successful. This is by the way nowhere near as bad as the 71% average across all software projects!
If so many software projects are not successful, should we use them as our go-to solution for how we deliver our work? In the last few years, a movement called #NoProjects has formed, saying we shouldn’t. I am willing to sign up!
Why are projects bad?
Prince 2 (“the world’s most practiced project management methodology”) defines a project in the following way:
A Project is a temporary organization that is created for the purpose of delivering one or more business products according to an agreed Business Case.
The thing that stands out the most with that statement is probably the bit about “temporary organisation”. A group of people come together, deliver the project and disband.
Thankfully, most of us in the agile world don’t work like that. We tend to have reasonably stable teams, working on the same product for a long time. This saves us from wasting the effort it took to get the team up and running (the forming, storming and norming stages of their journey as a team) and we get to keep the knowledge they have built up over time.
If a criterion for a project is a temporary organisation, we’re not really doing projects (maybe we should call it “project-but”?).
There is more
However, while we’re happy to drop the bonkers idea of the temporary project team, we cling on to the concept of projects. Unfortunately, this can bring with it a certain mind-set which is inherently bad to Software development:
- Projects have an end, products don’t
- Success for a project is often defined as delivering the features on time and on budget, not getting the most value
- Projects contain big batches of work
Let’s look into each of these in a bit more detail!
Projects have an end, products don’t
The biggest problem with a project approach is that a project, by definition, has an end. We work on any project for a finite period of time. This can be either a fixed date or a flexible date but after this date, we consider the project completed (or terminated) and we move on to the next one.
A product, on the other hand, will live on. This means that the deliverables from a project will live on well beyond its end date.
Support and maintenance
Support and maintenance will be required to keep our product running. Changes will be needed to keep it relevant. These activities will continue up until the point when the product is decommissioned.
As a product team, we will be the ones dealing with these things after go-live. Bugs will come in and tweaks and changes will be requested. So much for defining the scope and an end date!
Ironically, the project model might even increase the risk of these issues from previous projects popping up, requiring our attention. When we work in projects, everyone involved do their best to deliver the project. This risks shifting the focus from a sustainable code base to getting the project out.
Compromising good technical practice
Good technical practice may be compromised. This will result in known and unknown bugs, technical debt and a codebase in need of refactoring. We fool ourselves into thinking that we will deal with it after the end of the project, when things are quieter. But guess what, there will be another project then!
Beyond the very short term, this reduced code quality will unavoidably lead to a slowdown. The code will get increasingly hard to work with and once the decay has started, why put in the effort to deliver the new features with a high code standard, when the code base is so painful to work with anyway?
Success for a project is often defined as delivering the features on time and on budget, not getting the most value
A project is focused on delivering the agreed deliverables, on time and on budget. If we do that, we tend to say we have successfully delivered the project. The underlying assumption is that by delivering the deliverables, we will deliver the benefits.
There are several problems with this:
- Only outcomes are valuable, output isn’t. Even if we do exactly what we say we would, exactly when we say we would, we can still deliver something with little or negative real impact. Maybe we delivered a new web shop exactly as specified, but it ends up taking very few orders. Imagine if we had used all that effort better!
- Ending at a particular date is in itself not valuable. We will almost always have dates to meet. For example, if we deliver a website for a yearly music festival, doing so two weeks after the festival isn’t very useful. We need to make sure that whatever we deliver by that date has the biggest impact of anything we could do. On the other hand, what’s to say that just because the festival has finished, we should stop our development? Maybe there are some features we can add with a good ROI that would keep the festival goers come back and sign up for next year’s festival?
- Ignoring feedback and learnings sets us up for failure. During the development of any software development, we will learn things along the way. In agile, we even try to maximise this learning through building iteratively and collaborating with customers and users throughout delivery. However, if our main aim is to deliver a certain scope, we might end up focusing on getting all the features completed. The new ideas or changes will be added to the bottom of the backlog, in safe knowledge that we will never get to them before the project finishes.
Let’s not be unfair, though. Project plans often include an activity for benefits analysis, some time after the completion of the project. That’s when we look at the KPIs to verify whether we achieved the intended benefits. The obvious problem, though, is that after the project has finished, it is too late. We will already have spent all the time, effort and money!
Projects are big batches of work
A project is by definition a big batch of work. We wouldn’t go through the overhead of starting a project for a tiny little change.
Admittedly, in an agile project, we do allow ourselves a great deal of flexibility when it comes to the scope. We can add or remove items from our backlog as we go along. We aim to build small and frequent increments. Nevertheless, we’re human and it is easy for to fall into the trap of a big batch mentality when there is a list of stuff after which the project is finished. We “know” what we need to do, so let’s optimise to deliver that as efficiently as possible!
Big, big, big
As we all know, with big batches comes… well, yes, big responsibility!
- Big decisions. Should we start this 6 month project? Or should we throw away the 2 months’ effort we spent defining it? Should we make a change which will have a big impact on our planned work? change? Should we pull the plug after the back-end team have spent months building their part? Huge decisions!
- Big risk. Compared to a small batch, a big batch will be more complex and more things can go wrong. When something does go wrong, the implications are also likely to be bigger.
- Long lead time. Delivering a big batch of work takes longer than a small batch. This means it will take longer to validate that what we are doing is the right thing. Even more risk!
- Complex coordination. As we’re dealing with big risk and big decisions, we need strong control and monitoring. To help with that, it may be easier if the back-end team deliver a batch of functionality to the front-end team, who then deliver their batch. More batches and hand-over!
What’s the alternative?
So, if projects are that bad, what do we do instead? Well, I’m afraid I’m not going to reveal something ground-breaking here. We should use continuous delivery. And if we’re already doing that, we should stop trying to wrap a project around it.
More specifically, we should:
- not work to a big predefined list of features. Instead, we should focus on outcomes (actual end-user impact. We should prove at every opportunity, through small frequent releases and A/B testing, whether the solution we’ve chosen has the intended impact. If not, we need to make changes!
- always focus on the thing that will give the best ROI, not a fixed end-date. That might well mean we continue to deliver something beyond when we thought we would move on.
- make small decisions, all the time, not big decisions. We do need a big picture but rather than a big definition and planning phase, we should start out with a rough idea and create more specific requirements and plans for each item just in time, as we learn more.
Common agile practices will help us deliver in this way:
- Cross-functional feature teams, able to create a whole solution to deliver the impact we’re aiming for, without being depending on other teams or people. The goal isn’t to simplify the coordination between teams, it is to reduce the need for it!
- Collaboration with users (or representatives) throughout the delivery, so that we understand their needs and get feedback on the work we deliver. Feedback that will have real impact and, just end up at the bottom of the backlog!
- Continuous integration, automated testing, refactoring and other technical practices that help us make sure that we’re able to keep making small and frequent changes to our code at a high but sustainable pace.
We normally define project success as delivering a set of features on time and budget. However, by doing so, we don’t necessarily get the benefits we were after. In fact, the risk is that we reduce our chances of getting them.
When so many software projects fail, should we keep doing them? As agile teams, we’re already breaking the project model in so many ways. Isn’t it time we take the full step to shift our focus from time, scope and resource to always delivering the most valuable thing we can, as effectively as possible?
- I first heard of the idea of #NoProjects in the article If you need to start a project, you have already failed (and a subsequent talk with the same title) by Evan Leybourne. Allan Kelly is another big advocate and I particularly enjoyed his article No Projects – Beyond Projects.
- Scrum project success rate from the State of Scrum 2015 report from the Scrum Alliance.
- Software project success rate from the 2015 CHAOS report, from the Standish Group, covering 50,000 software project.
What do you think? Are software projects a relic of the past that should die? Or are they awesome (or just a necessary evil) and we should focus on how to make them better? Please share your thoughts in the comments below.