/ Testing

Unit testing - Myths and misconceptions

This article attempts to be a language agnostic view of unit testing, and which tries to be devoid of any sentiments regarding taste and non-functional preferences. I too have strong feelings when it comes to semantics, tools and technology, but these are trivial compared to the fundamentals of writing unit tests. It is in the basics of writing good unit tests that the important understanding lies.

Why is unit testing so hard?

There are so many developers that find unit testing a grueling and arduous task. Many see it as a somewhat (un)necessary chore that needs to be performed, but that takes time away from getting their production code released. Many understand that unit testing is required in all current software development methodologies, but they seem not to really understand why.

Many also have a difficult time finding out what to test, and how to do it. "What should I test?" and "Is this enough testing?" are reoccurring questions I have been asked by developers I have worked with. This shows an inexperience and uncertainty about both what the purpose is, and which benefits unit tests offer.

"I will write unit tests later" and "I don't have time to write unit tests" are known excuses used by developers. They are deceiving themselves. As the code base grows so will the complexity. If the code base becomes hard to maintain and difficult to refactor, the growing complexity will most certainly impede productivity. The amount of new functionality delivered per development cycle will decrease. Properly designed unit tests forces the development of maintainable code. Likewise, a good unit test suite will serve to protect the code from harm. Refactoring can be done with much reduced risk of breaking functionality. Changes to the code base can be performed quickly and safely.

It seem like many developers don't see unit testing as an intricate part of the software development process, but rather a doctrine used to keep QA happy. "I know my own code, and it works perfectly when I run it. So why do I need unit tests?". Many will argue that you should write unit tests to ensure that there are no bugs in your code. This is an oversimplification of purpose of unit testing.

For me, unit testing is not a practice primarily used for finding bugs. Of course it will flush out any obvious defects and shortcomings in the code. But I would not characterize these trivial errors as bugs. Bugs are errors that occur due to more complex interactions. They often require a whole range of conditions to be present before they show their nasty faces. As such, bugs are typically found in the interaction between units, modules and systems.

So what is a unit test then?

Project Test Cycles
First of all, what is a unit? There is no absolute answer to this question, and the answer will most certainly depend on who you ask. It will most often also depend on the programming language used. As a rule of thumb, a unit is a small piece of functionality that can be tested in isolation. This can mean that a unit is a single function, or a small group of functions that work in cohesion. For object oriented languages a unit will typically be a class.

Why should you write unit tests? In an overall sense unit testing can help you to verify that you have captured the wanted functionality of the module you have developed. They do not say anything about the correctness of system as a whole, but this is not the purpose of unit tests either. Written correctly, unit tests can however ensure that the tested unit of functionality can be used by other units without big fear of failure. As such, unit tests can serve as documentation of the usage patterns of the unit of functionality.

At what time should unit tests be written? Some say straight after the code is finished. Some say at the same time as you write the code. Some even say that you should write the unit tests first. I will not take any side on this, as this is a matter of opinion. And opinion is often governed by "politics" and "religion". By politics i mean the guidelines which are layed down by management and architects, often when it comes to vendors and suppliers. These guidelines are often set by people that have moved further and further away from the bleeding edge of development. As such they have distanced themselves from new tools, technologies and innovations. By religion i mean the firm and obsessive stance some developers take on their preference in tools, technology and methodology. They are often convinced that their view is the only true understanding, and that all that see things differently have no say about those matters. As history have shown us, such views are ignorant, not to say dangerous.

What is beyond any doubt is that unit tests must be written when you have the functionality of the unit fresh in your mind. If you leave the writing of unit tests until later there is a strong chance that you have forgot much of what you did, and the unit tests will become much less effective. Poorly written unit tests can almost be harmful. They can dull the perception of developers and management, thinking their production code is covered by unit tests, when it isn't.

The positive aspect of writing the tests very early is that you remember what you code does, and it is easier to know what and how to test the functionality. Another aspect is that you will get feedback on any defects at an early stage, which make error correction much easier. There should be an iterative unit testing process, where the functionality is gradually evolved towards eventual robustness. Another positive side effect is that through the process of writing unit tests you will see the need to develop code that is testable. It will promote software design with clear separation of concerns. Code that is divided into small units is easier to understand, easier to maintain and easier to test.

When your unit of functionality is covered by a suite of good unit tests, not only do you have a verification of the correctness of that functionality. What you also get with the test suite is a safety net for any changes that are made to the code. The unit tests should and will detected changes that break the functionality. This safety net will lower the risk associated with doing changes to existing code. The result is a much improved and effective refactoring process. Running the tests often during refactoring will give you instant feedback on the success or failure of the code changes.

What is important for unit tests?

When writing unit tests there are some important design rules that should be obeyed. The positive effects that come as a result of a suite of unit tests, which I discussed in the previous paragraph, are dependent on these rules. The list below name the rules I believe are the most important, and which are essential when writing good unit tests. I go through them one by one in the following paragraphs. Unit tests should be:

  • Fast
  • Isolated
  • Independent
  • Robust
  • Maintainable
  • Purposeful

Fast

A unit test should run very fast. Having fast unit tests means they can be run often. This is a very important quality, especially in the refactoring process, where instant feedback is crucial. Slow unit tests will cause them to be run seldom, making them next to useless as a refactoring tool.
Test Amount and Execution Time

Isolated

A unit test should test its assigned unit of functionality in isolation. What this means is that the unit should be isolated from other units of functionality during testing. Stubs, mocks and dummy components should be used in place of any dependency the unit might have.

Independent

A unit test should be self sustained, and not depend on any other test in order to be able to run. There is typically no guarantee on the execution order of unit tests, so any introduced causality between tests can be harmful.

Robust

A unit test should be designed as robust as possible. It should produce a deterministic result which is independent of external concerns. No matter how many times the test is run, the time of day it is run, what locale and environment it is run in, or any other influences, the test should give the same result.

Maintainable

A unit test should be easy to maintain. In the same way that production code is refactored, unit tests also have the need to be changed and improved. If the unit test is well written and easy to understand, it will make the task of maintaining it much simpler. As a general rule you should strive to have the same code quality in your unit tests as in your production code.

Purposeful

A unit test should have a clear purpose. It should test a small piece of functionality, and it should test it well. When writing a unit test you should know what the preconditions are, what you want to test and what the expected result should be. Using a sensible naming standard for unit tests will help in expressing the purpose of a test, and will make it easy for other to understand what the test does.

How can I write good unit tests?

Writing better and better unit tests happens as a consequence of practice. Knowledge and experience are driving factors for good unit tests. But even the most experienced developer started by obeying rules very similar to the ones I have listed.

But as the title of this paragraph asks: how do you write good unit tests? There is no black magic to writing good unit tests. I have put the Fast-rule first because the whole effectiveness of unit testing is dependent on conforming to this rule. You need to be able to run your test suite often. And in order to do so the tests needs to be fast. But what makes a unit test fast? If the code under test is slow then the test itself will be slow. This is a strong sign that you need to refactor your code. The same is true if you need a lot of stubbing/mocking in your test. If you spend a lot of code lines setting up the stubbing/mocking harness you should take it as a clear indication that your production code can be split up into smaller pieces. It should be relatively easy to stub/mock your dependencies before you can run the test, and there should seldom be a need for more than one or two stubs/mocks.

Unit Test Cycle
I'm no TDD'er, but Test Driven Development have some good virtues. It promotes creation of tests for every new piece of functionality. As a direct result you get high test coverage. If the tests are written while observing the defined rules you should have a close to perfect unit test suite. Then again, using high test coverage metrics as the only measure of the quality of a unit test suite will never give the full truth. Another positive aspect is that writing the test first forces you to think about how to structure your production code. High separation of functionality makes the code easier to test, but also easier to read and maintain. This is often called writing "testable" code.

So in order to get a good test suite you need to have good, clean and testable production code. So lets consider what makes code hard to test:

  • The code is too monolithic
  • The code and/or the test is non deterministic
  • Use of the new operator in your business logic
  • Putting business logic inside constructors
  • The code depends on global state
  • Too many static functions
  • Components with deep inheritance
  • Having too many conditionals

Most of these properties are anti-patterns of programing, and should obviously be avoided. But a code base that has a long lifespan will most certainly encounter many of these issues. Enforcing high test coverage, and obeying the rules of unit testing that I have mentioned can help in avoiding that the production code is degraded by these properties.