Monday, June 23, 2008

Silo Driven Development (SDD - an antipattern)

In Johanna Rothman's latest post, handoffs don't work, requirements hand off to development teams is discussed and I think she makes a great point. I believe that this behavior is part of a larger issue of silos in development organizations. Unless organized along the lines of product development, silos are anti-patterns whether BA's working on requirements, DBA's working as part of a database team, a tester working for QA team or an architect working in an architecture/strategy group.

So, lets take a minute to look at the requirements example because this one is probably the most common problem. Now, I tend to think that business analysis is part of a developer's everyday job, so the role itself is important; but why do we use a business analyst?

- Do they have better domain knowledge? Often not, they are usually skilled in business analysis, so like a developer, their knowledge of the domain is secondary. Additionally, developers tend to ask many more questions than their BA counterparts.

- Are they better communicators? I don't think there is any difference (in my experience)

- Do they save developers time? Well undoubtedly if the developer starts work without talking to the customer at all - but is this what we want. This, I think is a serious problem and leads to the developer missing a key opportunity to pick up on the domain, the more they can talk directly to the customer the better, having a proxy in between is just a recipe for miscommunication and misunderstanding.

- Don't they get it right first time? No, unfortunately not. Johanna's post describes this much better than me. Often subconsciously, customers use empirical feedback to steer their requirements along the journey. Expecting them to sign their name in blood on a requirements document and it will never change has long since been dismissed as naive at best.

So, why do I seem to be committed to the development perspective and not feel for the poor BA? Quite simply its a matter of responsibility. A business analyst has to deliver a paper document, proving how good or otherwise that is proves to be quite subjective. A developer on the other hand takes responsibility for delivering working code. Did you ever hear of a BA getting in trouble for not delivering a document on time? Do documents fail testing or cause null pointer exceptions during use? Many customers read requirements documents and think to themselves 'I have no idea what this all means - but I am confident everything will come out exactly as I expect it to.'. 

Even if you work with a BA, cutting out the silo and having the BA work with the development team - and the team all working with the customer, (all other things being equal) should result in a better product.


David WRight said...

I have been a Business Analyst/Architect for 25 years, and I really don't need you to feel for me. See my comment on Johanna Rothman's post to see my main point; the problem is the quality of deliverables, not their existence. See to learn about how it is done.

Andrew said...

David, thanks for your response, I have commented on Johanna's blog.

The main point I am trying to get across, is that software creation is a team activity, and creating any form of artificial boundary is counterproductive.

Unfortunately, requirements cannot all be known in advance of development.

I have been at the sharp end of projects where requirements have been signed off in documents but customers still want to change things. It is unrealistic to expect otherwise, there are too many variables.

Once you accept that things/people change, the process gets much easier, and results in happier clients.

Its a difficult thing to accept though, so I understand, I was there myself not so long ago.


David Wright said...

Sorry, got to disagree. If you drive it out of step-level definitions of a business process, you can define all the requirements up front; we at IAG do it all the time, in 1 to 2 week discovery sessions. Quality artifacts like Requirements Statements are not boundaries, they are a means of communication and agreement.

A "team activity" does not mean a group of people in a room somehow generating output. Teams have positions, and people with different skills fill positions. shortstop is different than first base, but you need both and everyone else.

And yes, "change happens". If the business immediately wants to change your requirements right after sign-off, obviously a poor job was done. The key is to deliver quickly, whatever methodology you use, and when real change happens, like a business process has changed, then you deal with that in the next delivery.

Andrew said...


Step level definitions of a business process are all well and good assuming you have your business processes mapped out correctly. What if they are not?

What if during the process of building something real and putting it in front of the customer to get feedback they realize its incorrect or is cumbersome and unworkable in practice? What if, after a 1 to 2 week discovery session, and one minute after you have left the building, the customer remembers something critical or changes their mind? Maybe he decides he wants the screen laid out differently because he has seen what the real technology can actually do for him? There are too many potential what-if’s in life to be able to predict with any certainty that something is definitively correct.

What does ‘correct’ mean anyway? In software development, it means that your customer is happy with the software product (not the requirements artifacts) and it will presumably help them make more money, compete better in the marketplace etc.

Change is all about human nature. Business leaders often change their minds quickly to gain competative advantage in the market. People have the right to change their minds, especially your customer – who is supposedly always right. Well, from a funding perspective, they are. If they want to change things up completely after a 1 to 2 week session, throwing it all away they can – its their choice and their money. Does that infer to you that a poor job was done? Not at all, its not the dilligent analyst’s fault that the customer changed his mind. It’s not about how well you do the job – its about accomodating what the customer wants – if he wants to change his mind its up to him, not you or any other team member. Fully engaged team members are there purely to make the customer happy and to serve them. So in this context, what does ‘sign-off’ mean to you? It probably means you think you’re done, but I have never worked on a project where that stage means you’re actually done. Done means your software is delivered and being used by your customer, it has no relevance to the delivery of a document.

I agree with your statement about requirements being a means of communication. Its all about the customer communicating to the development team what they want. Often, its necessary to capture these requirements because there are a number of people involved and they need to refer back to captured information, to reduce the chance of miscommunication by explicit statement. Just enough communication for everyone to share this common understanding is whats required. If there is just one developer, then he doesn’t necessarily have to write anything down, unless his memory is not that great – which probably applies to most of us.

Yes, teams do need a mix of skills, but whether these skills are held by different people or the same person playing different roles is the question, I assume. Either way can work, but the truth is, whatever the team make up, developers have to understand what the customer wants. If analysts are employed and leave developers out in a typical ‘requirements phase’ the results are often disastrous. Developers are on the hook – they are responsible to produce something that works. If they don’t have the relevant information to do this, they cannot do their jobs.

Not sure what you meant by ‘A "team activity" does not mean a group of people in a room somehow generating output.’

Finally, couldn’t agree more with your point about delivering value quickly, and the best way to do that is to do a little and get the software in front of the customer as quickly as possible, so you can use his feedback to determine the direction you need to take next.

David Wright said...

In short, I can only refer to the dozens of satisfied Fortune 500 customers of IAG. We need to do it right, or we do not get repeat business or good references.

Other thoughts...screen design is not requirements, changing a layout is not a requirement change.

....A cumbersome design/implementation does not invalidate requirements, it is just bad design.

...valid requirements can often tell you that a project should not be proceeded with, such as when the cost of meeting the real requirements will be more than the value produced...

...and most projects don't develop new software, what with enhancements, BPM systems, packages, SaaS... but they all need Requirements.

We will just have to agree to disagree, I guess...

Andrew said...

David, we’re closer to agreement that you think. However, my experiences are very different to yours and we have both formed different opinions based on those experiences over the years.

I feel that requirements are – to a degree - whatever your customer wants them to be – so if your customer feels strongly that they want screen design to look a particular way – its their money – who am I to tell them they’re wrong. If I have misinterpreted this, then who does own screen design? Developer, analyst? We might discuss options or their feelings about it might lead me to more interesting questions. A while ago I was working on a project and the customer told me about a very specific screen layout which made no sense based on our previous conversations. It actually turned out that our thinking of the domain was flawed and only revealed by this exercise. Customers are not expert at producing requirements, they just know roughly what they want. Its our job to get it out of their brain and into working software, but they need to be guided by whats possible. The more they see of what can be achieved, the more it steers their thoughts and ideas towards something real. Its very hard to imagine something without seeing a glimpse of where you want your thoughts to go. How many times have you been inspired seeing something amazing – could you have dreamt it up yourself?

Agree with your points about design/implementation do not necessarily invalidate requirements – although it depends what you mean by cumbersome.

On your point about proceeding with a project. This is interesting, and a big subject in its own right, so I am summarizing in the interests of brevity. I have seen so many poor project initiations and there is no simple answer. Everyone in the team needs a common understanding of the product vision. However, estimation is notoriously bad, so ROI calculations at very early stages are hard. Why is this? There are so many different variables and you cannot account for the inevitable ahead of time. Sometimes, you can get closer than others – it depends on team make up, technical stack, domain knowledge and many other factors. The sad reality is though, that estimates are always out.

Maintenance and new features on existing products however, tend to be a lot more accurate – mainly because archiectures are already understood, domain issues have often been ironed out, the team knows how to do most things, it tends to be ‘more of the same’. So, to borrow from some of these traits to improve project estimates, the best way to work with new projects is to do a little bit, thus exposing the main architectural mechanisms. This yields real, empirical results which provide us with a much more reliable basis for estimation. Now, developers feel more comfortable providing and committing to estimates, because they are dealing with known entities, technically at least. The more pieces are revealed, the further along the cone of uncertainty the estimates become. But the real benefit of working this way, is that you have pieces of a real product early. That is priceless for business people, not only does it provide reality based ROI, but the customers also immediately see the benefit of their investment dollars.

Anonymous said...

ITSolusenz departments manage all components web application, software development including, Application Development Company, software development company india, Software Development Services.

Software Development India said...

I tend to think that business analysis is part of a developer's everyday job, so the role itself is important; but why do we use a business analyst.