Most software engineering estimates are garbage.
That’s not because companies are using the wrong methods or tools. Work-breakdown structure or analogy-based? Mechanical or judgmental combination? Function, use case, or story points? SEER-SEM, WMFP, or Wideband Delphi? Fine.
The tools aren’t the problem. Rather, most estimates are garbage because they’re based on a fundamentally flawed understanding of how quality software is built.
The impact goes far beyond cost overruns and missed deadlines. The typical approach to estimates ends up forcing bad behavior while privileging vanity metrics over delivering actual business value.
Noise and non-determinism are inherent to software engineering
In Agile environments, estimates are often based on story points and velocity. How “complex” will it be to create a discrete piece of the solution? And how long does it typically take us to complete a story of that complexity? (I’ve written previously about how this approach to Agile corrupts Scrum with Waterfall methods of control.)
When estimating this way, we understand that not everything will go according to plan. But underlying most estimates is a dangerous assumption that even this uncertainty can be quantified and factored into our estimates. If optimistic engineers tend to underestimate how long a given task will take by 15%, we just feed that correction into the formula for a better prediction.
This obsession with specifying and measuring the full process in advance wraps a plus or minus variance around a system that views engineers as machines pushing predictable work products through a pipeline at a steady stream. Then this metaphor of software development is treated as real and translated into mathematical calculations that clad fantasy functions with a veneer of quantitative validity.
Yet to state what should be obvious, human beings are not machines. (Thank goodness for that.) And maybe less obviously, the complexity of any non-trivial software engineering task is almost impossible to accurately estimate in advance.
Our field is so new and rapidly changing. This makes last week’s performance a very poor predictor of next week’s velocity. So many of the interesting challenges we face each day are novel and unknown, and even the known ones won’t stay still.
Consider a trivial example: implementing a login page. Any experienced software engineer has done this dozens or hundreds of times. We know the pattern of the solution well, and we can make some predictions about how long the next one will take. But then along comes a new, more secure way of handling authentication and authorization, and suddenly we have to rethink and reimplement how a basic login page will work.
Most software engineering challenges are far more complicated than a login page. This is as it should be when we’re tackling big problems and creating substantial value. We’re doing things that haven’t been done before, or maybe haven’t been done as well as we think is now possible. We’re in uncharted territory, with a compass but no map.
This uncertainty, in other words, is good. It’s a sign that our ambition is sufficiently visionary, that we’re taking on work that is meaningful and worthwhile. Poor predictability is not the problem. Arbitrary estimates are.
As a statistician might put it, there is too much noise in the system, more variance than we could possibly correct for in our estimates. And the work we’re trying to estimate, if it’s work worth doing, is fundamentally non-deterministic.
When estimates are based on the myth of metronomic coding machines tackling deterministic work, they’re a complete waste of time.
But wasting time is only the beginning. It’s the least serious cost.
Bad estimates force bad behaviors that are bad for business too
Garbage estimates don’t account for the humanity of the people doing the work. Worse, they imply that only the system and its processes matter.
This ends up forcing bad behaviors that lead to inferior engineering, loss of talent, and ultimately less valuable solutions. Such estimates are the measuring stick of a dysfunctional culture that assumes engineers will only produce if they’re compelled to do so—that they don’t care about their work or the people they serve.
Falling behind the estimate’s promises? Forget about your family, friends, happiness, or health. It’s time to hustle and grind.
Can’t craft a quality solution in the time you’ve been allotted? Hack a quick fix so you can close out the ticket. Solving the downstream issues you’ll create is someone else’s problem. Who needs automated tests anyway?
Inspired with a new idea of how this software could be built better than originally specified? Keep it to yourself so you don’t mess up the timeline.
Bludgeon people with the estimate enough, and they’ll soon learn to game the system. They’ll overestimate complexity to buy themselves more time. They’ll slow down when they’re progressing too quickly so they don’t set future expectations too high. Smart people would be foolish to do any less.
Individuals and interactions create more value than processes and tools
“Individuals and interactions over processes and tools.” That’s one of the key values of the Agile Manifesto. It’s a statement of what we should value as compassionate and ethical human beings. It’s also an assertion that focusing more on people than processes leads to better quality results.
A generative software engineering culture is built on a foundation of trust and driven by human relationships. It’s a social network of adults with a shared commitment to crafting high-quality, high-value solutions that solve significant problems or seize meaningful opportunities.
There’s no gaming the system in such a culture. Common cause inspires people to do their best work.
Progress is measured by value created, not tickets closed. And if (when) people discover there’s a better approach than what’s specified in the estimate, they readily share these ideas, knowing that a quality solution, not an arbitrary estimate, is the ultimate measure of success.
When we focus on individuals and interactions rather than processes and tools, we empower the full value of what each individual has to offer, and we multiply the value that teams create in collaboration.
It may be less predictable than what we get when we control people with a detailed estimate for a fully specified product, but giving up that control and predictability ultimately unlocks much greater value.
Roadmaps, ranges, and relationships are the way
It’s tempting to suggest we could do away with estimates altogether.
I do think there are some compelling scenarios in which we could do just that: Agree on our shared mission, take ownership of our shared vision, then work together to create quality software without any prior prediction of how long this will take or how much it will cost. Just imagine the big, meaningful problems we could solve, the elegant solutions we could craft.
However, such an approach is rarely practical in a business environment, where we usually must make pragmatic compromises with budgets and schedules.
The answer, then, is not to eliminate estimates altogether but rather to approach them as a conversation in a culture of mutual trust.
Product and engineering teams should have open and honest conversations at the beginning and throughout the software development life cycle. These conversations start with the assumption that everyone does care and will do their best to solve the important problems, on time and on budget.
What do we think we can accomplish with the resources available? What can we deliver and when? What are our backup plans if time or resources run short?
These conversations lead to provisional roadmaps and ranges: With humility, here’s how we think the project will unfold. And here are the upper and lower limits of how long we think it will take to complete.
As development progresses, the conversations continue. If some aspects of the project turn out to be more difficult than expected to solve, do we delay a feature? Select a simpler solution? Agree to modify the roadmap to accommodate the extra time?
If (when) we come up with a more valuable idea in the midst of development, do we alter the roadmap or save that idea for the next round?
When relationships between and within teams are healthy, these conversations happen all the time, and they lead to higher-value solutions.
When garbage estimates rule through processes and tools, all these changes along the way are perceived as a failure to stick with the estimate. But the failure is actually in the estimate itself. It’s a failure to recognize the greater value created when we trust good people and teams to do their best work.
Instead of deadlines and tickets, we can lead with mission and vision. We can recognize and accept that every collaboration is a conversation, and every project is a journey of exploration that cannot, that should not, be fully planned out in advance.
Because in engineering, as in life, the good stuff is often not what we plan before we start. It’s what we find along the way.