Tuesday, June 10, 2008

Estimation

I had an interesting discussion some time back about the relative merits of attempting to give a SWAG (silly wild ass guess) on level of effort to build some features for a product. It was the very early stages of product definition and a rough level of effort was required from some participants. 

My first reaction was the suggest that we put more meaning around the two or three word features that were listed out because I did not even know what the words meant, let alone how long it would take a team to build them. If I do not know what these things actually mean, how can I provide any form of estimate - there was not enough information. Yet, despite my stance, there was still a general insistence that the information be provided.

This got me thinking more deeply about estimates and estimation in general. There are many estimation techniques out there such as cocomo, wideband delphi, function point analysis, some of which I have tried, some I have not. But I ask myself now, is there any value in pursuing any of these? Are they any more accurate than a 'gut feeling' (I guess that is synonymous with a SWAG)?

So, can estimation techniques provide any kind of reasonable output? I would say the answer to that is a guarded - it depends. There are many factors that govern predictability.

- PEOPLE - team size, mix, skills, talent, effectiveness bonding, business domain knowledge
- PROCESSES - how the team works, rigidity, willingness to change, working environment
- TECHNOLOGY - equipment and tools, choice of libraries, languages, frameworks

Sure, there are many more than I have listed here, point is - the more that is known, or understood, the more likely that those involved will be able to provide meaningful estimates. 

So, for example, if a team is asked to provide estimates to build something in the business domain that they understand very well, with technology stack they have prior knowledge of, with a good team mix and the right input, environment, tools etc. they can provide something meaningful. However, I would still consider their output with a healthy dose of skepticism, because users/customers/product owners are prone to changing their minds. Even developers change their minds during the course of a project.

Where to now? The problem with the above estimation ideas is that they are based on the assumption that things remain static during a project. But projects are a creative activity, not a production line. Its like asking an artist to provide an estimate on how long it will take him to paint a picture - he may get half way, and clear his canvas and start again from scratch if he doesn't like the way its taking shape. He has that prerogative - its an artistic process. 

Now, can teams at least commit to a specific capacity? To a degree I think they can - a team who have worked together before may know that all things being equal, they can deliver 20 story points per iteration. Does this mean anything in terms of estimates - I would say a qualified yes - if everything remains unchanged. If circumstances change (and they will) then the team should be able to use that information to advise customers/users/product owners that project parameters have changed and that scope or time to market adjustments should be made. The more work a team delivers on a project, the more they learn about themselves, the tools and the domain and the more accurate their estimates become over time. In my opinion, this places an even higher emphasis on open/honest communication channels with executives and stakeholders to allow them to make sensible funding decisions.

1 comment:

Paul said...

Andy,

I can tell you've given this a lot of thought. I agree with everything you say. The problem I think stems from people wanting to turn a SWAG into a prediction. Tim Lister once mentioned a case where teams kept "estimates" to two decimal places. When historically their estimates were always 50-100% off the mark.

He called this confusing accuracy with precision. I can accurately estimate to a precision of +/-50% :) given a certain mix of knowns and unknowns :)

I think another issue here is yet again developers not being willing to say "I don't know". Instead we are seduced into turning our SWAGs into precise predictions.

Agile tries to do away with all this nonsense by using "past weather". So in answer to how long it will take, you can say: "I don't know, but I can build a little and see". With concrete empirical data you can make more confident guesses about the future, but it is still a guess. The future weather could prove more stormy then the past. So we are back to a lack of precision and the need for contingency.

So instead of SWAG, why not "Suck-it-and-see"? And instead of pretending that our plans are precise, why not do ourselves a favor and build in some slack for those rainy days ahead?

Sounds like common sense doesn't it?

Paul.