Learning to Focus

You can't program well, efficiently, and successfully unless you can focus.

The enemy of focus is distractions: your boss walking in, your phone ringing, your co-workers talking to you, emails, IMs, tweets, text messages.  These are distractions that actively steal your focus.  There are also distractions that you create yourself: reading Google Reader, Facebook, Twitter, and Reddit, talking to your neighbors, working on too many things at once, and so on.

These kinds of distractions need to be managed:

• Turn off email notifications and check them less frequently.
• If you are on IM, mark yourself busy when you're working.
• If someone walks in or calls, tell them you need 20 minutes to wrap up.
• Limit the number of things you are actively working on at one time.
• Schedule breaks.
• Expect to be interrupted.

The last two are especially important.  Scheduling regular breaks as in the Pomodoro Technique helps you to stay focused during your work periods.  When you know you have a break coming up its easier to put off answering messages, and reading crap on the internet.

Expecting to be interrupted by phone calls keeps you from getting frustrated when it happens.  It also means you have to work in small increments and keep note of where you're at so when you do get interrupted, you can get back into it more easily.

Hard Work

This is all fine and good, but at the end of the day the hardest part of focusing is that it is hard work.  When I first attempted the Pomodoro Technique I couldn't believe how hard it was to work for 20 minutes straight.  I had no idea how often I was allowing myself to be interrupted, by active interruptions like phone calls and people talking to me, but mostly by interruptions I created myself like reading crap online and talking to other people.

Actionable

Another important element of staying focused is you need to know what you're working on with enough detail to actually DO SOMETHING.  This shows up in Getting Things Done, when it talks about managing your tasks it recommends you write down both the task and the first actionable step to completing the task.  There is a somewhat subtle but important distinction there.

Focus vs. Interruption Roles

Focus gets very difficult when you have different job responsibilities too.  For example, if you are expected to program and manage a project.  Programming is a role that requires you to be "in the zone", "in flow", "plugged in".  In other words, focused.  But managing a project is an interruption driven role.  Answering peoples questions, meetings, reviews.  Interruption driven roles don't work well with focus driven roles.  To make this work, you have to find ways to set aside time to focus without skimping on your interruption requirements.

Value Your Focus

But no matter what your role is, its absolutely crucial that you understand the importance of focus and that you take your own ability to focus seriously.

More Engagement

Are you engaged at work?  I don't mean like, engaged to be married.  I mean are you ENGAGED?

If not, why not?  Are you not paid well?  Do you not like the work?  Do you not like your coworkers?  Do you not like the management?  Do you not like your boss?  Are you bored?  Is the organization keeping you from doing your best work?  Do you feel unproductive?  Do you feel unable to contribute?  Do you feel micromanaged?  Why?

Maybe this guy can help explain it:


Maybe your organization's attempts to motivate you are actually, though accidentally, de-motivating you!  Isn't it ironic?  So what should they be doing?  What do you need to be engaged at work?  To be at your best, to be your most productive, to be happy?

According to Dan Pink, its as simple as 3 words:  Autonomy, Mastery, and Purpose.  This observation lines up so nicely with the observations in First, Break All The Rules too!  But the problem with these observations is that they are only observations.  They provide really useful information, but don't help your boss figure out how he should act day to day.

That's actually one of the main reasons why I think First, Break All The Rules is such an excellent book, and so much better than any other book on "management" I've ever read.  There simply is no cookie cutter, one size fits all, set of steps you can follow to create an environment in which everyone is engaged.

Now, if you're the boss you can start putting your understanding of Autonomy, Mastery, and Purpose to work right now.  But if you are not the boss, what good is this to you?  If you have some "direct reports," you can apply these ideas with them within your own projects.  At least that's a start.  But if you don't have "direct reports," then it looks like knowing this stuff isn't going to help you at all!

I remember the first time I read Peopleware...  On the one hand I absolutely loved it.  But on the other, it was just depressing.  And I believe that my company would actually rank very highly compared to other companies on these factors.  But it didn't matter, it was depressing!  That's because this kind of stuff seems too far out of the control or influence of a lowly programmer.  What impact can one person have on things like culture?  Or the autonomy granted to employees?  Or the purpose behind the work?  Or big-M vs. little-m methodologies?

I think the answer to these questions is, quite frankly, that on your own you can have very little impact.  BUT!  I believe a group of like minded people, with common goals, patience, and a dash of determination can get together within any organization and make a huge difference.  Those people can become engaged in the struggle to be engaged at work!  And, as cheesy as it may be, any group of people starts with just one person.  To be successful at this you have to have a broad idea of where you're heading. If all you want to do is be like 37 Signals, you're out of luck.  But if you can embrace the real kernel of truth in all the observations found in so many places these days (including what the guys from 37s are saying), and tailor that to your organization's unique goals, strategy, and personality...  Then you will be able to make your company, or your division, or even just your team one that all your friends will be jealous of.  And everyone will end up doing better work because of it.

So if you're not engaged at work, stop looking up!  Stop waiting for someone else to change!  Get out there and get started making a difference today.  Even if just a small one.  Be prepared to lose a lot of battles, but don't let one set back prevent you from continuing to work at it.  Because it is worth working at.  And you don't have to get 100% of the way there.  You don't have to end up with a ROWE to be engaged at work.  That's because lots of little improvements will add up.  And could maybe even start a steam rolling effect.

The key is to recognize that it's worth striving for, that you don't have to keep looking up waiting for someone else to make the changes, and that its ultimately a community effort.

Passion

I think there are two qualities that set really great developers apart:

1. Technical competence
2. Passion

Pretty much in that order.  If Bob has strong tech skills, it means he can solve complicated problems to a reasonable level of quality independently.  But if he is lacking passion, it means he wont be looking for opportunities to improve, or to push the envelope on issues like code quality, cleanliness, productivity, etc.

If Bill has a lot of passion, it means he'll be on the lookout for ways to improve.  Both in his own work, and his team's work.  But if Bill doesn't have the technical competence to back it up it means he's simply unreliable.  The impression will be that he pays lots of lip service to quality and improvement but never manages to actually deliver any.

Bennie, on the other hand, might have both of these qualities.  This makes him reliable and constantly improving. And not just improving himself, but improving those around him.  Bennie is the guy who's likely to not only complain about some "policy" his office has that he feels is hurting more than helping, but to actually work on getting that policy changed.  Effectively, Bennie is a leader.

I think people with passion naturally end up leading, regardless of whether they are in "leadership" roles.  You don't have to be the boss to influence how your company works.  And you don't have to be the team lead to influence what technologies get used and how.

However it is this fact that makes passion a double edged sword.

Passionate people are more likely to challenge the status quo.  Which is good.  Unless they are challenging it in ways that actually hurt their team or hurt their company.  This runs along the same lines as some issues I discussed in an earlier post called Engaged Employees.  In a nut shell, if the priorities of the passionate people don't line up with the priorities of the business, you've got trouble ("With a capital T.  And that rhymes with P and that stands for..." Passion?).

When the passionate people become dis-engaged, there are two likely outcomes.  They might start "farting around" with "improvements" that don't actually help the team or the business accomplish any of its goals.  It is probably still true that these "improvements" are "good" in their own context.  But in the context of the business, they might actually be "bad," or possibly just unimportant (and therefore a waste of time).  On the other hand, the passionate people might decide to simply checkout.  They might decide to put in the minimal amount of effort possible, with an attitude of "screw those guys."  If just one person was acting in either of these ways, it wouldn't be that big of a deal.  But when it's your passionate people, it can lead to much more trouble.  These are your leaders after all, and their attitude and behavior rubs off on everyone else.

A company plagued with dis-engaged passionate individuals (like Bennie) would probably like to trade them all for simply technically compete people (like Bob).  The Bobs would stop causing so many problems and stop challenging everything and stop being in such a bad mood all the time and just get their work done.

But we have to ask, what has led the passionate people to be so removed from the goals of the business?  I think there is always a simple one word answer to that question: Management.  The book First Break All The Rules makes the case that a managers job is simply to bring out the best in his people.  And not the best out of context.  If you're an amazing yo-yo player but you program for a living it's not your manager's job to bring our your best yo-yoing.  Obviously.  It's his job to bring out your best in the context of the goals of the business (What a Programmer Wants in a Manager).  If your passionate people don't know what the priorities of the business are, or if they can't figure out what they could do today to have the biggest impact on the company, then management has messed up somehow.

But just because management has messed up doesn't mean we get to blame them and call it a day.  It doesn't mean we can just give up, stop trying, decide not to care, or adopt a bad attitude.  That is the path to the dark side.  As Bobbies, we have to do what we've always done: Fix it!  This is going to be harder for us than, say, designing a better data layer.  This is now a people problem.  And as technical nerds, we are probably not the best suited individuals to address people problems.  But sometimes we have to embrace the circumstances life throws at us, however uncomfortable they may be, and we have to grow up, step up, and fix it!

People Problems

I'm sorry to have to tell you this, but your job is to solve problems for people.

I don't even have to know what your job is, and I can still say that with a pretty high level of confidence.  Are you an engineer building bridges?  You're solving a problem for people who need to get across that divide.  Are you a cashier at the grocery store?  You're solving a problem for people who need to pay for their groceries (and for people who need to take money from people for groceries, double whammy).  And of course, are you a computer programmer?  You're solving SOME, maybe not very well specified, problem for your users.

It really doesn't matter how far removed from people you are, you're still solving problems for people.  You could be completely devoted to the inner most workings of the Windows kernel.  You're actually worse off!  You're solving problems for users who want to check their email, and programmers who want to write MS Word, and designers who want to make Windows look pretty, and marketers who want to sell Windows as the most stable ever, and finance people who want to show market growth, and the list just goes on and on and on...

Sometimes you are aware that you are solving problems for people, but maybe you don't really know what problem you're solving.  Like the grocery store clerk who is really nice and friendly and makes all the customers happy going through the line.  Maybe they think their job is to make you happy, but it's not.  The problem you want them to solve is to figure out how much money you owe, and take it from you.  And you want them to do it quickly.  If they can do that AND make you happy, then they are a very good grocery clerk.  If they only make you happy, then they just aren't very good.

A similar thing happens with programmers.  I frequently get confused and think that my job is to make high quality software and write high quality code and do good high quality design work.  But its not.  My job is to solve some problem my client has.  As long as I fix their problem, I've done my job.  Maybe I could have fixed their problem better with beautiful code I'd be proud to hang on my wall at home.  That would be a bonus.  But all I'm supposed to do is solve their problem.

That is, unless I'm actually supposed to be developing a long life product, which will require all kinds of future enhancement and maintenance and re-configuring.  In that case my job is to both solve the client's problem AND develop a strong software product.  These are two competing goals.

And that's the thing about People Problems:

• They are never clear cut
• They tend to overlap
• They always involve trade-offs

They're not clear cut because frequently you don't know what problem your supposed to be solving.  Or there are a bunch of problems you're supposed to be solving and you don't know which is most important.  Or different people have different problems and you need to simultaneously solve them all.  Create an app for a user!  Create it in the time your managers wants!  Make the app a platform to build a product on for your boss!

And this is where the trade-offs start.  You simply can't HAVE your cake AND eat it.  See?  Once you EAT it, you don't HAVE it anymore.  Or if you HAVE it, then you haven't EATEN it.  See?  I just thought that up. Pretty good huh?

The "real world" is all about People Problems.  And People Problems inevitably lead to trade offs. And trade offs inevitably lead to disappointment.  For example, if you choose to make the code super high quality, you wont be disappointed, but your boss will be because of how much it cost.

The trick is to understand that above all you are solving people's problems.  Then understand that necessarily involves trade offs.  Then try to take a big picture view when deciding on how to make those trade offs.  Hopefully that will at least help you deal with the inevitable disappointment.  At least you'll know you made the best decision you could for the specific People Problem you were faced with.

Of course, you'll probably be disappointed when you find out later that you didn't have all the facts straight and your decision was based on faulty information or just information that has since changed.  But that's a whole different dimension of dealing with People Problems.

What a Programmer Wants in a Manager

Management is an oddly fascinating subject. It's kind of dirty word these days, but when you distill out all the nonsense and get down to what it's really about, it's interesting. Managing programmers, or "knowledge workers," is, in many ways, a special case and requires special consideration. This is because you can't break a programmer's job into a series of reproducible steps. Programming is an inherently creative job, which is why it's often compared to craftsmanship.

In thinking about how to manage programmers, I tend to empathize with the programmers more than the managers. The way I see it, a programming shop's number one expense AND number one asset is its programmers. So it seems pretty clear to me that you aught to do everything you possible can to keep those programmers working well. The challenge is of course figuring out how you actually do this.

The book First, Break All the Rules, suggests that a manager's job is to discover the talents of his people and direct those talents to the business's goals. I like looking at it this way because it indicates that the manager should recognize what his people are good at, and then let them go be good at it. The trick then is to make sure the things they're being good at are also the things the business needs to be good at.

Joel Spolsky goes even further and says that a manager's "most important job is to run around the room, moving the furniture out of the way, so people can concentrate on their work." At least, he says that's how managers at Microsoft behaved. Again, there is a large focus on getting out of the way so your people can work.

Recently Fog Creek announced a series of training videos they are selling. The video series is $2000 and a little conceited if you ask me... But in the promo video, one of the Fog Creek dudes says, "Developers are assumed to know the right answer. So you don't start from a position of negotiating about whether or not you could possibly have the right answer. You assume they have the right answer and that's there job to explain to you why its the best solution, not why its the wrong solution or the right solution." In this we're seeing some of that "get out of the way" mentality but also a certain amount of built in trust.

I have my own theory on the best way a manager should behave, which is strongly influenced by these references as well as just about every word in Peopleware. I think it breaks down like this:

1. Trust your people
2. Value your people's time over just about everything

When I say trust, I'm not talking about blind trust. I'm not a complete idiot! But I am talking about a change in tone. A manager needs to set the goals and objectives people are working toward, and a manager needs to ensure those goals and objectives are being met correctly and effectively. But a manager should trust their people to do the work right and in the best way possible. As a manager, you can ask for proof of why what people are doing is the BEST way. But you need to be careful that you don't demand proof they are not completely wrong. There is a subtle difference here which has a huge effect on morale.

I think this is very important. Programmers want to be treated like experts. They want their opinions to matter. If you stifle that, you will end up with frustrated programmers, and frustrated programmers don't work as well. Worse, since programming is a creative activity, if you're actually stifling creativity you're ruining your product. The best way to improve morale and encourage creativity is to offer up some trust and treat people like adults.

If your people know that they are trusted, and that their ideas will be seriously considered, they're more likely to bring ideas to you. They're more likely to think outside the box. These can only be good things. If instead they feel untrusted and are afraid their ideas will always be shot down, they wont bring anything to you.

Programming can be detailed tedious work. Nothing pisses a programmer off more than the impression that the time and effort they have spent was wasted or simply unappreciated. This is why programmers hate dropping a project in the middle to work on something "new" and "higher priority" that "just came up." New higher priority things DO come up at the last minute. But if a manager just tosses it on a programmer's desk and says, "get this done first" they're not valuing the programmers time.

I firmly believe a manager's job is to push the furniture out of the way so the programmers can get their work done. But, we're not really talking about furniture. We're talking about all the complexity of a project. The inter-relations between different teams, the ever changing client demands, the relative priorities of different assignments. These are things the managers should be focused on working out so that programmers don't have to spend so much time worrying about them.

Again, much of this is just tone. When a manager is laying out the work that needs to be done and the order it needs to be done in, and who needs to do what work when... they can treat this as a power opportunity for themselves. To hand out assignments from on high with little regard for explaining the circumstances to the programmers and a simple expectation that the programmers should take it and get it done. Or they can treat this as an opportunity to indicate that the goal is to optimize the developers time so they can focus and do their best work instead of dealing with the sticky details of the real world that programmers simply don't like.

A manager has to do the same work either way. In the end, much of management really comes down to politics. And since I believe the programmers are the most important asset going for a business, I believe the politics should be oriented around keeping the programmer's morale high, avoiding frustration as much as possible, and engendering a corporate culture where the programmers feel their work is valued and important.

Engaged Employees

First, Break All the Rules is a fantastic book about management and companies and I highly recommend it for managers and non-managers alike. Toward the end it makes the case that good managers lead to engaged employees.

I was totally enthralled by that term engaged employees. It's a perfect description of one of the characteristics that can make one person so much better at their job than another. I'm sure you've seen examples of this too. One person can be ridiculously smart, and yet do bad work. Or they might be amazingly talented, and still they don't do good work. You see examples of this every day in the grocery store checkout clerk, or the person at the front desk of your hotel, or the cleaning staff at your office, or the police officer directing traffic.

Sometimes when a person isn't performing well in their work, the failure is attributed to them being lazy, or immature, or even that they're just not challenged enough and so are bored. That may all be true, but the problem is really that they are simply not engaged in their work, for reason or another.

It turns out this concept of employee engagement has a rich history dating back to at least 1993 when it was described as "an employee's involvement with, commitment to, and satisfaction with work."

This really goes to the heart of what Peopleware was all about. Every manager wants their employees to be involved and committed to their work, but they all too often forget that they also need to be satisfied with it. Peopleware talks about this all over the place. And probably the two best real world examples are Google and Fog Creek.

Companies with disengaged employees are simply bleeding money. Its like the difference between people who drive the speed limit and coast into all their stops and people who are constantly speeding up to the bumper of the car in front of them and hitting the brakes... The second dude is just throwing gas away and killing his mileage. Every time he pulls around someone and floors it he feels good, but then he has to slam on the brakes again. And when he fills up at the pump and calculates his mileage, he'll blame his car or traffic conditions for the low mileage, anything but himself. This same short sightedness is the reason why companies want to skimp on amenities for their staff or the quality of their product. But its costing them engaged employees.

So engaged employees are important, the challenge is in getting them. Some people need someone around to keep them actively engaged, but others don't. The people that don't need help are your "self-motivated" people. The kind who need to be engaged to be happy. I would argue that most people in the world fall into this category, they just might be engaged in things other than their work. Look at sports. I'm not into sports. At all. I enjoy watching a game, but I just can't get into the stats and the history and memorizing years and events and people and on and on and on... I'm simply amazed by people who can learn all that stuff and talk and argue about it endlessly. The amount of energy and dedication required for that is phenomenal and it takes some serious smarts. Imagine if you could harness just a little bit of that energy and apply it towards something slightly more productive.

And therein lies the rub. How do you get people to be engaged, and more importantly how do you get them engaged in the right things?

I've had the fortune of knowing and working with a lot of smart people. Most have been naturally engaged. But it's interesting to see what that results in. If there is a project at work that excites them, they're all in. But when it doesn't, or when some other factor is causing them to dislike it, they'll find other things to get into: side projects at home, or frameworks or "minor" bugs or related "enhancements" at work. Expending energy on these things is extremely valuable experience to the individual but maybe not so much to the company.

But on the other hand, the greatest breakthroughs can frequently come from people messing with unrelated stuff. Just look at Google's 20% time and the making of ad sense. Ad sense practically single-handedly funds Google, and it was invented by someone in their 20% time. So having some leeway is equally important.

There is a line to walk here. If you have a lot of people who are very engaged, but always in things that don't help you achieve your business goals, you're screwed. And if most of your people are not engaged at all, you're screwed.

Walking this line is going to be difficult, but I would guess that most companies aren't even aware of it. Maybe their employees are engaged anyway, because of good managers or just because of luck. But that doesn't seem too likely.

So how do you get people to be engaged? Peopleware, and First, Break All the Rules, and Joel Spolsky, and 37 Signals can answer that question better than I can, so I'll refer you there.

In the mean time I'm curious, does your company engage you?

Getting Stuff Done

Today I was reminiscing about college and doing a bit of compare and contrast between how I got stuff done then and how I get stuff done now. I was surprised to find that this was actually somewhat interesting, so I'm writing about it.

In college, each class gives you one assignment at a time, and each comes with a due date. You may have many things going on concurrently, but each is clearly prioritized based on how hard you know the assignment will be, and when that assignment is due.

This situation is quite manageable. You may have to pull some late nights to get your stuff done on time when you find yourself in the unfortunate position of having many assignments due all at once. You may also shoot yourself in the foot by procrastinating. But when that happens, you know it was your own fault. So, at least in my experience, college assignments were really quite doable. I had some periods of stress, and I certainly had times when I felt very busy, but overall, it wasn't so bad.

The real world is not like that. But, why not? What makes it so different?

Here's some of the characteristics of college that don't show up so frequently in the real world:

1. Well defined work
2. Fixed schedules
3. Clear priorities
   
4. No re-work
   
5. No support calls
   
6. No releases

This is pretty self explanatory I think, but I like to hear myself type, so, lets talk about it!

Well defined work is easy. In college, you get a detailed assignment. It tells you what your inputs are, what your outputs are, exactly how the programs should behave in every way. You write it, test it, and you're done! Well defined work. In the real world you try to get your customers to define the work. But your customers don't even know the work themselves, so its really hard for them to define it for you. This means you can spend lots of time working on things that you either don't need, or that don't work quite the way they really should. And you don't get to find this out until you're all done.

Fixed schedules is also easy. In college, you get two weeks to do your assignment, from start to finish and turn it in. The professor never tells you to stop a week in and do another project. Or worse, and more realistically, to add another project on top of the first. This seems to happen all the time with real work, and it makes it difficult to plan ahead and stay on track. You start to feel like a never ending stack trace...

Clear Priorities dove tails nicely with fixed schedules. Your school assignments are easy to prioritize because you know up front how much of your grade its worth, how difficult its going to be, and how long you have to do it. These factors allow you to easily decide which assignment should be given priority. Once again, real life is not so clear cut. Things pile up and they're all ASAP highest priority do or die holy crap! Good luck prioritizing that!

Ah yes, re-work... Here's a factor that you never ever ever ever ever encounter in college. If you suck it up on an assignment you get an F, you move on. Not so in life beyond the academic gates. If you screw it up it doesn't end there. It snow balls and things get worse and worse. And only if you're lucky do you actually get the opportunity to go back and fix what you screwed up. And now you've spent twice as long on that assignment, which has certainly pushed back some other absolutely top highest priority assignment.

And now we come to support calls. My favorite! I never had a professor call me weeks after I'd finished an assignment and ask me how to use it. Or ask me why it doesn't do something no one ever heard about before. But the best part about support calls is how much time they can eat up. Simple calls will take you about twice as long to answer as you feel like it actually took. And complex ones will eat your entire day.

Finally, with a college assignment you have a single release: the first one. And none after that. In the real world, you'll have lots of releases. How many depends on your specific situation. But, one of the tenets of Agile is release often, so I think lots of people are probably doing lots of releases these days. Releases take time. You have to migrate the database, release the client, push the update, test. And the bigger the system, or the more components it has, the harder this process will be. In fact, this can get ridiculously hard. And this is yet another thing you never had to deal with in school.

So, is it possible to get stuff done in an environment where you're being pulled in so many directions at once? Or is the only answer to bring some of that collegiate structure back?

Interestingly, if you read many of the Agile practices in this light, you see a lot of it is geared towards restoring some of these characteristics. For example, iterations. This practice manages to restore some of your well defined work, fixed schedules, and clear priorities. Also, the customer representative and release often practices are there to try to cut down on the amount of re-work.

So maybe the best you can do is try to bring back as much collegiate structure as you can, because its awfully hard to get anything done without it.

How Should I's

There are ten types of programmers: Those that understand binary, and those that don't.

Oh, no, that's not what I meant.

There are two types of programmers: Those that ask, "How do I make this go?" and those that ask, "How should this go?"

I like to call the first group High school programmers. These are the kind of people who when presented with a task take their first idea and start trying to make it work. When it doesn't, they just keep tweaking it until it does work. Then it's "done."

I call them High school programmers because this is what High school students do when presented with an error message. "Oh, that's weird. Well, I'll just change this over here and try it again. That didn't do it? Ok, what about if I do this? Still no? Well what about..." They just care that it works in the end, they don't really care how it works or why it works or why it didn't work in the first place.

The second group are the ones that not only want their code to work, but want it to work the best it can. These people are probably going to consider alternative implementation approaches, designs, and architectures. They're probably going to refactor their code to make sure it's as clean and efficient as possible. They may even go so far as trying different things before making up their mind (prototyping, if you will).

This distinction actually is important. You obviously want the How Should I's working with you and not the How Do I's. Simply because their work will be better: cleaner, more maintainable, more bug free, more extensible, and believe or not finished faster.

Its important because this is the quality you're actually looking for. I've read many blogs where people claim that you can't be a good programmer and leave work at 5pm. This is simply ridiculous. Certainly your How Should I's are likely to be more obsessive, and therefore more likely to get caught up and work longer. But there is absolutely no reason why a How Should I can't have a life outside of work, leave at 5pm, and still be a great developer.

I also think that being a How Should I is a necessary condition to qualifying as Smart and Gets Things Done.

And on top of that, being a How Should I is very likely to also make you a top 20%-er.

You can also see this quality in The Pragmatic Programmer's definition of a Pragmatic Programmer,

Tip 1: Care About Your Craft
Tip 2: Think! About Your Work

So if you're interviewing, or reviewing other people's work, or simply working with other developers, this is a quality you should look for and appreciate.

The Agile Customer

One of the main concepts of "Agile Development Methodologies" is that the customer be involved in the entire process. And not only that they are involved, but that they are actually there, alongside the developers, while the software is written.

This is, of course, ridiculous. Most customers don’t actually have that kind of time to spare. So in reality the word "customer" is used to mean "customer representative." That is, someone who should be sufficiently familiar with the real customer to be able to understand their needs and argue for them with the development team.

This concept is a primary ingredient in fulfilling the purpose of an Agile Methodology. That purpose, in my view, can be stated as follows:

Requirements will change and be wrong. We must be able to create excellent software despite those changes.

This is accomplished by designing and developing things in small pieces (in iterations) and by frequently getting those pieces to the "Agile Customer" where we can verify that they work out, and when they don’t work out, we make the required changes immediately.

That, at least, is the idea. But I don’t think it works, and I think the reason it doesn’t work is because this concept of an Agile Customer is a bit unreal. Here’s why:

1. Your client doesn't have time to be that involved in the development process, and even if they did, they wouldn’t care
2. The business process is more complicated than a single person can fully understand, even though they think they understand it
3. A "customer representative" will know even less than the customer
4. No one is going to get it all right the first time, and they wont notice its wrong until they try to really use it
5. In a software demo the client will find about 80% of the trivial problems with the software, but only 2% of the big stuff (missing pieces, incorrect process, bad assumptions, etc)
6. In a testing environment, the client will find about 80% of the bugs in your software, but only 10% of the big stuff.
   

Basically, we expect an Agile Customer (who is not just a single person of course...) to know and remember every little detail and to recognize when details are wrong and also when they are missing. Unfortunately, real live people are really bad when it comes to these kinds of details.

The result is your customer representative won’t know all the things they need to know (because it’s impossible for them to). Your client will love your demos but won’t notice all the things about their business that aren’t accounted for (because they didn’t tell you about them because they probably don’t even remember themselves). Your pre-release testing won’t uncover the problems because it only begins to scrape the surface (because it would take too much time and effort to really use the system and not just test the system).

What’s a poor agile methodology to do?

I don’t think you should stop trying to use the “Agile Customer” approach. But you should realize from the beginning that it isn’t going to get you to 100% wonderful working software.

To do that, you’re going to actually have to release it. Often. Even when it doesn’t quite do everything it needs to do yet. Then you’ll see if it’s 100% wonderful and working.

This means you are going to have to transition people to using it, and you’re going to have to train people on it. It’s going to take time and effort. But if you do it correctly, you can save yourself time overall by finding problems earlier and fixing them before it’s too late.

The obvious problem with this is that your users will hate you. You are giving them unfinished software and telling them it’s done. You are making them deal with frequent releases, which means frequent changes in the software, which means they have to keep relearning things you changed, as well as learning new things you’ve added.

However, we can mitigate this by setting the tone.

First off, DON’T tell them it’s done! Even if you think it’s done, it’s not done. So tell them it’s solid, tell them its tested, but tell them that you and they need to work together to ferret out anything that may have been missed, or misunderstood, or changed.

Then as problems are found, don’t get annoyed or frustrated. Instead, remember that you were expecting it, and frame it as such. Every problem you find makes the software that much better. So tell the client that, and then keep them up to date on how much better the software is becoming. Show them lists of what was found, when, and when it got fixed and released. If you’re proud of how quickly you’re iterating and improving you can trick them into being psyched about it too, I mean, make them feel psyched about it too.

Then as critical parts of the software “settle down” in release, you can start adding other parts to them. You’re going to have to plan these “parts” intelligently of course. You can’t release the order intake portion of your software without the portion that allows them to view, manage, and/or print the orders… So your parts need to make sense. But you can release the majority of the order stuff and leave out some of the next steps, or additional features. You may find that you don’t need that stuff at all. Or that you had it all wrong. Or, if you’re lucky, that you had it perfect, and then you can go ahead and develop it and release it. Then you’ll discover you had it wrong. But, hey, at least you tried. And at least you got the really important main stuff out the door earlier.

Basically, you’re not doing anything differently than you would have done it before. You’re just doing it with a different expectation and a different attitude. The result will hopefully be rock solid software, happy clients, and best of all, happy developers.

Now we come to the part of the article where I fess up to the fact that while I wrote all this as if I’d been doing it for years and it works perfectly, I actually haven’t had an opportunity to actually try to follow this advice. Instead, this is more the result of the problems I’ve observed and the best ideas for how to solve them I’ve been able to come up with. Thus, if you’ve had different observations, or different ideas for solutions, or if you just completely disagree with me and think I’m a stupid moron who speaks out of turn about things he doesn’t know anything about, I want to hear about it. And that’s what the comments are for.

Leadership

In High School and College teachers and professors (of the liberal arts...) pay a lot of lip service to "Leadership." Of course they never really tell you what it is, they just tell you to do it.

Usually this was most apparent in group projects. The group's first order of business was always to select a leader. This was done by someone asking, "Who wants to be group leader?" Everyone would give each other blank stares until someone finally responded with, "I'll do it, I guess."

Sometimes the group leader would simply end up being the group secretary, responsible for writing down the answers and turning in the final paper. Other times the poor bastard would actually try to "lead." Depending on who the leader was, this would take a few different forms. Some leaders would try to make all the decisions and tell people what work they were going to do, even before anyone had started brainstorming about the project. Other leaders would wait for someone else to come up with a good idea and would then say, "Okay, here's what we're going to do!" and repeat the idea.

So basically school was worthless when it came to understanding or teaching leadership, and group projects were only good as an exercise in frustration (which is clearly a good exercise for the real world).

Ultimately, I think leadership is simply organizing the behavior of a group of people. So, basically, it's telling people what to do. Good leaders tell people to do the right things and don't piss people off when they tell them. Good leaders also resolve disputes by helping others make good decisions. Good leaders are also able to take the best advantage of the skills of the members in the team.

Over the years in observing how people try to lead, and how other people seem to lead without trying, I've decided leadership has more to do with personality than anything else. These are the three ways I've identified that seem pretty common:

1. Lead by decree
2. Lead by consensus
3. Lead by respect

The people who lead by decree come into a meeting with their minds made up and tell everyone else what the team is going to do. Sometimes these people try to pretend they want other people's input, but instead of actually listening to what people are saying they just wait for them to finish talking so they can go back to convincing everyone to do what they want.

The lead by concensus type is more democratic. They bring their ideas to the table along with everyone else's and allow everyone to argue it out. Which ever idea the team decides on wins. The lead by consensus type of person typically doesn't throw their weight around to get what they want, even if they have any weight to throw. That doesn't mean they give up on their idea completely, but it means they usually defer to the group before trying to issue a decree. This can be a problem when a decision really needs to be made or when a firmer hand is warranted, because you don't want to waste forever arguing around in circles. Typically I think I fall into this category.

The lead by respect person is a rare and impressive find. These are people who everyone simple likes and respects so much that other people actually want them to lead. I've only known a few of these kinds of people and I've never been fully able to understand how they did it. Sometimes these people never even come across as telling people what to do.

These are the three "categories" I thought up, and as you can tell they focus more on how a single person actually behaves. I did some brief google research and found Lewin's Leadership Styles. Interestingly, these correspond somewhat well with my own take, but they capture more of the gray area:

1. Authoritarian Leadership - leading by decree
2. Participative Leadership - leading by consensus but retaining the final say
3. Delegative Leadership - leading completely by consensus and/or allowing everyone to make their own decisions

Some interesting thoughts. No one leadership style can be considered the best. It depends on the circumstances. For example, if the leader is the most skilled of the group, Authoritarian leadership makes the most sense. However, if the peers are primarily equal, Participative makes the most sense.

The leadership style also dramatically affects the group members. If the group members are very independent people, they may prefer a Delegative leadership style. But, while they may prefer it, it may not be the most effective.

Finally I've been in very few situations where there really was one leader. Leadership tends to float around, landing on different people at different types. The title doesn't float around, but the actual act of leading does. I think its this fact that makes effective leadership important. You have to recognize that leading doesn't mean controlling every detail. And you have to work well with other people when the leadership bubble has landed on them. Otherwise you're just going to piss everyone off.

Managing Projects

This is a follow up to a previous post, To Estimate or Not To Estimate.

I've been doing a lot of reading and learning and experimenting since that post and I now have a much clearer idea of what I'd like to see in a project management application.

I'll start with what kinds of questions I want to be able to answer:

1. Who has too much work, who needs more work? (Developer Workload)
2. How much work have we done, how much work is left (Iteration Status)
3. Is our scope slipping, are we getting things done, are we converging on a completion date? (Iteration Progress)
4. Are certain high level tasks slipping or eating more time than others, or more than expected? (Project Item Progress)
5. What are people working on/what should they be working on?
   

My current thoughts are structured like this:

• Project - top level
• Project Items - high level "tasks", or "features" that make up a project. These are assigned to one or more individuals. Estimates are added at this level in terms of complexity or actual hours.
• Features, Tasks, and Bugs make up Project Items.
• Tasks and Bugs are assigned to developers and estimated in hours (<=16 hours)
• If actual time spent is going to be tracked, enter it at the Project Item level.

There are a couple of important things to note here. First, from an estimation standpoint, it makes no difference how much time the developers have actually spent versus how much time they estimated. All that matters is how much time they think is left. With that information you can create a "burndown" chart. That chart will answer my Iteration Status, Iteration Progress, and Project Item Progress questions (assuming I can filter it appropriately). That information would also make it possible to build a work load chart similar to this one. All of this can be accomplished without worrying about actual time. We can also display a list of all the Project Items a certain developer is assigned to, or a list of all the developers assigned to a given Project Item. No more wondering what people are working on.

Now you have some nice and simple burn down charts. But are they accurate? No. You know your developer's estimates are not correct. You also know that some devs are better estimators than others. So you can't accurately predict a ship date from this data. You can get a rough idea, and you can see that the project is in good shape and is progressing nicely, but you can't determine a date.

If you want to be able to predict an actual ship date, then you need to track time, and you need Evidence Based Scheduling and you need FogBugz. Can EBS account for the fact that working with a third party scan library will take 4 times as long as what a developer usually works on? Kind of, but not really. Can it predict that a ton of redesign and new features may need to be added to the application after the client sees a demo? Kind of, but not really. So even though EBS is way more accurate with the data it has, it's still just giving you a rough estimate because it doesn't know what data may arrive tomorrow.

So the question becomes, how accurate do we need to be with this stuff? Are we satisfied knowing that our work is trending toward completion? Or do we need to know WHEN completion will actually happen?

From my current thoughts above you can tell that, right now at least, I think it's enough for me to know we're trending toward completion. Unfortunately, I'm not the only person in the equation. My boss needs to do two things I don't. First, he needs to guess before we even start a project how long the project will take. I call this a "blind estimate" and it's clearly impossible, but he still needs to do it because the client isn't going to hire you otherwise. Second, he needs to keep the client updated on when we're going to ship, because that is all a client cares about: when we'll ship, and how the budget looks (only some of them care if the app is any good).

Clearly, actual time matters a lot to him. And not just time spent working on tasks, but all "billable time." Time spent in design meetings, time spent with clients, time spent writing documentation, time spent researching tools, etc. Thus any time entry system which is going to be at all useful for integrating with billing has to allow me to enter time on things other than tasks or project items. It should allow me to just enter time and not require it to be "on something."

Furthermore, now that we're entering time, that time can be used to help with future "blind estimates." How long did it take to do x last time? It's likely to take a similar amount of time this time. That's why I think it makes sense to track time at the Project Item level. It's high enough to be "reusable" but low enough to be semi-accurate. Also, since we're estimating at the task level, and tasks are linked to Project Items, we can see how the remaining estimates and actual time spent compare to the original blind estimate and the budget.

I suppose the final question must be, is there a tool that will do all this? Not all of this, at least, not that I'm aware of. I know that a lot of this is very similar to Scrum. But the Scrum tools I've seen don't capture all of it (Scrum for Team System, escrum), especially not the time entry portions. Also, FogBugz comes pretty close, but it lacks the Project Item concept and doesn't have any burn down style reports. So if you were hoping I'd end this by recommending a tool, I'm sorry to disappoint. If I find one, or can make one work like I want, I'll let you know. And if you know of one, please let me know.

To Estimate or Not To Estimate

I have recently been spending a lot of time with two different "project management" applications: FogBugz and Team Foundation Server. FogBugz originally attracted my attention because I read Joel Spolsky's blog, Joel on Software, and he's the CEO of Fog Creek Software which makes FogBugz. TFS attracted my attention because it includes a new Source Control system which is a-w-a-y better than Visual Source Safe. I have been evaluating these products and trying to figure out which I'd rather use, its a harder task than it sounds.

The new version of FogBugz, 6.0, mainly consists of Evidence Based Scheduling. I have to be totally honest and say that EBS is not only very cool but makes total sense and I believe it would totally work. However it requires two things: 1) time tracking and 2) work item estimates.

Joel Spolsky recommends keeping your estimates under 16 hours, anything longer than that doesn't stand a chance of being accurate. That means you need a lot of very detailed tasks entered in the system (ex: develop function foo). Once you have estimates at this level you can start to get good predictions of when you'll ship AND you can start to pick and choose what tasks you want to do based on how much time you have left.

TFS doesn't really have any scheduling. It has Remaining Work and Completed Work fields (but only on the Task work item type, not on the Bug work item type...), but no reporting, no nice views, and no nice way to enter time (you have to subtract from the remaining work field manually when you add Completed Work...). The TFS developers seem unconvinced that estimating work at the level Joel recommends is at all worth while. Witness these articles on the Teams WIT Tools blog.

Personally I think estimating how long a task will take is a great idea. I think creating a list of tasks at that detailed level would not only help a developer think through what they're going to do before they dive into it but also help them get an idea of "where they're at" with all their work. I think it would make managing work loads a lot easier too. Not to mention the fact that the ship date estimates would make it possible for you to determine when your initial "blind" estimates were wrong or when scope creep is starting to get the best of you.

My question for all of you is, what do you do at your work? It doesn't even have to be programming work. Do you estimate work? At the detailed task level, or at a more abstract level? Is it helpful? And if you don't estimate work, do you wish you did?

UPDATE: I've posted a follow up post, Managing Projects