The Analysts Dilemma

What's the hardest part of software development?

Too vague?  Lets make it multiple choice:
A. Architecture
B. Code design
C. Algorithms
D. Business Analysis
E. Data Structures

If you answered anything other than D then you're an idiot.  Seriously, look at the title of the post!  How could you NOT know that D was the right answer.  This isn't some open debate, this is more like high school, and I'm the teacher on this blog, and whatever I say is the right answer is the right answer.  It doesn't matter what you think!  Much like the relationship an analyst has with the customer.

This is what makes analysis the hardest part of software development.  You really really want everything to be lined up in nice neat logical rows so that you can build the software in nice neat modules.  But those damn users just refuse to do things logically and neatly!  And despite how much you try to have it your way, you just keep getting Cs and Ds.  Ultimately you have to give in and just give the users what they want.  Embrace the wrinkles, the complexity, and the real world.

This is the picture of the world usually painted by Agile and DDD, and it's almost correct.  Because it is true:

1. The real world is complicated
2. You can't dramatically simplify how your users work
3. You have to make your users happy

Don Norman, author of the great book The Design of Everyday Things, talks about this in his Business of Software talk.  As he says there, the real world is NOT logical.  But then he goes on to talk about what makes Analysis really hard: you can't trust what your users tell you.

That may sound harsh, but no matter how you cut it, it's true.  Don Norman doesn't come out and say that, but he tells a story which I've seen happen first hand many times.  If you ask people how they do their office work, and you write down everything they say, and then you read it back to them, they will completely agree with its accuracy.  But when you go and watch them actually doing the work, you'll see that what they told you isn't what they're doing.  If you ask why, the usual answer is because they are dealing with a special case.  "We usually do it that way, but in this case I have to..."

So not only are we stuck analyzing something seemingly illogical that we can't force into a logical mold, we also can't rely on being given fully accurate information from the only people we can get information from!  We.  Are. Screwed.

And believe it or not, I can make it even more difficult for us!  Because frequently the introduction of software doesn't just automate the manual process people have always performed, it actually changes the process.  Meaning that as you go, you're making things that were once true, false.  It's got some quantum mechanics flavor there.

So what are we supposed to do.  The first thought is to try to get more accurate information up front, but this will never succeed.  There will always be an edge case someone didn't think of.  And trying to drill into nitty gritty details without anything solid to build on leads you to become focused on things that don't matter, and over design.  Ultimately wasting time, and making it harder for you to respond to change when things inevitably do change.

Instead you have to do one, or both, of the following:

1. Teach your [users, customers, product owners, domain experts, etc] about the software side of things and get them intimately involved in the design and development of every aspect of the software.  From architecture to process to UIs.
2. Aggressively shorten the feedback loop in any way possible.  Get your designs, prototypes, early implementations, betas, and releases in the hands of users and make them work with them as quickly as you possibly can.

This is why it is so so so important to write agile code!  "Agile" code is code which is easy to change, to some degree of easy.  Once we embrace the fact that the hardest part of software development is analysis, and that truth be told analysis is basically impossible, we realize the most important thing for our code is to be able to respond to change.  This has some dramatic implications on how we approach code: understanding becomes more important than execution or writing speed.  This is why DDD, BDD, and SOLID are so important!

In the end, we have to stop thinking of analysis as something that happens once at the beginning of a project.  Instead we have to minimize how much time we spend up front, and actually use our code as a tool to help figure out what the customer actually needs the software to do.  We have to get to the point where learning something new from a [customer, user, product owner, domain expert, etc] doesn't cause us to grumble and complain about how no one ever tells us the right stuff.  Its time we owned up to the fact that this is how the real world works, and stop moaning about it, and start expecting it, and finding ways to turn it to our advantage.