Unit Testing Pitfalls

Unit Testing and more importantly TDD is all the rage these days. If you go simply by the noise in the blogosphere, it looks like everyone is doing it. Of course, the reality is probably that almost no one is doing it, but many people think its a good idea, and some of them wish they could do it.

For example, I have written quite a few posts about Unit Testing, TDD, and related topics like Dependency Injection. I have even done some real TDD on the software I write at work. However, most of what I write I don't TDD, and don't Unit Test.

Yes, I'm big enough to admit it. I think its a good idea, and I wish I could do it. But the truth is I don't.

Fortunately, I have reasons. And those reasons are that for all the promise of TDD and Unit Tests, there are a number of Pitfalls.

Almost everything has dependencies, and those need to be mocked/stubbed, and mocking sucks.
Martin Fowler has a great article about Mocks and Stubs if you want to read up. I think that mocking sucks because:

1. It is a lot of work
2. It requires intricate knowledge of the internal coding of the thing you're testing

Stubs are slightly better in some ways in that you don't have to go with as much intricate knowledge. But they're more work than mocks, and they can be super complicated, and they still require a good deal of internal coding knowledge.

If mocking sucks, and almost everything needs to be mocked, then almost everything sucks to unit test.

If an interface doesn't exist for your dependency, you have to wrap it.
Say you're testing something that writes to the windows event log. The .NET framework doesn't have an IEventLog interface defined, it just has an EventLog class. So if you want to mock out that dependency with dependency injection, you have to create your own IEventLog. Then you have to create a concrete class that implements IEventLog. Finally, you have to forward every method and property call in the concrete class to the EventLog framework class.

This no fun to write and it's adding complexity and overhead to your code. Just because you want to test.

Note: Using a dynamic language would remove the need for the interface and therefore make this problem go away.

You can't use constructors on your dependencies
Suppose your dependency needs some required information to function and the class you're testing has that information and wants to provide it. Typically you would simply create a constructor on your dependency that took in the required info. Then the class you're testing would new it up and pass in the info. Simple.

You can't do this if you're using Dependency Injection because the dependency must be an interface and interfaces can't have constructors, plus an instance of the concrete class must be passed in to your class's constructor.

To get around this you have to pass in a concrete class that implements the interface. Then you have to send the required info in through properties. Now you have to write the dependency class so that it checks that the required info has been provided before it does anything that requires it. This check will have to go in every public method and possibly some of the properties. Thus the class is more complicated and has more overhead. Just because you want to test.

You can't new-up your dependencies.
Sometimes you may need to new-up a dependency. Then when something changes you may want to new-up a new object to replace the old one. You can't do this if you're using Dependency Injection since you have to pass the dependency in as a fully formed concrete class.

To get around this you're going to have to write the dependency so that it is reusable. Unless you need two instances at the same time. In that case, you'd have to make your dependency into a factory that provided you with instances of your actual dependency. Just because you want to test.

So far, these pitfalls have all been due to Dependency Injection, which like I talked about in an earlier post is powerful, but also kind of scary. We might be able to avoid all this injection of dependencies by using a framework like Typemock, but that's not free, and if I recall right, its not cheap either.

GUIs can't be tested.
It depends on what kind of applications you're writing. For the kinds of apps we work on where I work, the GUIs can be pretty complicated. In fact, usually the GUI is just about all there is to it (aside from retrieving and storing data). We're still writing lots of complicated code which it would be awesome to test, but it's all operating on GUI state.

When people ask me what they can/should Unit Test I always say "Find the algorithm." But when the algorithm is "set this value on this field when the user clicks on this but only when this condition is met, otherwise change the controls which are displayed to this and disable that" you're pretty much out of luck.

Some things aren't worth testing
If all your class does is order calls to other classes and react to errors, your tests are going to be of limited value. Mainly because you're not testing much. It may be 100 lines of code, but it's really not doing much of anything. No algorithms. And any regressions are likely to be because of changes to the dependencies, not because of changes to that class. So is it worth testing this?


I would love to see a book or article on unit testing address these issues. I mean, who is writing code to transfer funds from one account to another and is dealing with simple objects? Who is writing a Queue? Who is writing a web service to serve a music catalog? These examples may teach the concepts and principles of TDD and Unit Testing, but they don't help me to actually practice it. Am I the only one?