I have a love/hate relationship with test driven development and unit testing.
I’ve been both an ardent supporter of these “best practices,” but I’ve also been more than skeptical of their use.
One of the big problems in software development is when developers—or sometimes managers—who mean well apply “best practices” simply because they are best practices and don’t understand their reason or actual use.
I remember working on one such software project where I was informed that the software we were going to be modifying had a huge number of unit tests––around 3,000.
Normally this is a good sign.
This probably means the developers on the project implemented other best practices as well, and there is going to be some semblance of structure or meaningful architecture within the code base.
I was excited to hear this news since it meant that my job as the mentor/coach for the web development team was going to be easier. Since we already had unit tests in place, all I had to do was get the new team to maintain them and start writing their own.
I opened up my IDE and loaded the project into it.
It was a big project.
I saw a folder labeled “unit tests.”
Great. Let’s run them and see what happens.
It only took a few minutes and—much to my surprise—all the tests ran and everything was green. They all passed.
Now I really became skeptical. Three thousand unit tests and they all passed?
What is going on here?
Most of the time when I am first pulled onto a development team to help coach them, there are a bunch of failing tests if there are any unit tests at all.
I decided to spot check one test at random.
At first glance, it seemed reasonable enough.
It wasn’t the best, most explanative test I’d ever seen, but I could make out what it was doing.
But then I noticed something…
There was no assert.
(An assert statement is what you use in a test to actually test something. The assert statement asserts something is true or false or some condition is met, otherwise it fails. Without at least one assert statement there is pretty much no way the test can actually fail.)
Nothing was actually being tested.
The test had steps and those steps were running, but at the end of the test where it is supposed to check something, there was no check.
The “test” wasn’t testing anything.
I opened up another test.
Worse.
The assert statement, which was testing something at some point, was commented out.
Wow, that’s a great way to make a test pass; just comment out the code that’s making it fail.
I checked test after test.
None of them were testing anything.
Three thousand tests and they were all worthless.
There is a huge difference between writing unit tests and understanding unit testing and test-driven development.
What Is Unit Testing?
The basic idea of unit testing is to write tests which exercise the smallest “unit” of code possible.
Unit tests are typically written in the same programming language as the source code of the application itself and written to utilize that code directly.
Think of unit tests as code that tests other code.
When I use the word “test” here, I’m using it fairly liberally because unit tests aren’t really tests. They don’t test anything.
What I mean by this is that when you run a unit test, you don’t typically find out that some code doesn’t work.
It’s when you write a unit test that you find that information out.
Hey John, you just said those 3,000 unit tests were bad because they didn’t have asserts or the asserts were commented out. Who cares if I’m only concerned about knowing the requirement at the time of writing? Or, should unit tests become regression tests?
Ah, so you are paying attention.
Astute observation… and yes, your unit tests should become regression tests.
One of the main reasons to write the unit tests, besides clarifying exactly what the code should do and finding it when it doesn’t do that, is to make sure the code continues to do what it’s supposed to do.
Essentially, unit tests become regression tests that make sure new changes in the code don’t break the old functionality.
Think of unit tests kind of like those little supports you see on young trees that make sure they grow up straight and tall.
Just because you planted a tree straight up from the ground, doesn’t mean it won’t bend or go crooked over time.
It’s the same with your code.
Unit tests can initially tell you that your code is planted straight up and then can help you keep it that way–even if some junior developer brings a downfall of heavy rain upon your fragile code.
More on this later…
Yes, the code could change later and that test could fail, so in that sense, a unit test is a regression test. In general, however, a unit test is not like a regular test where you have some steps you are going to execute and you see whether the software behaves correctly or not.
As a developer writing a unit test, you discover whether the code does what it is supposed to or not while you are writing the unit test because you are going to be continually modifying the code until the unit test passes.
Why would you write a unit test and not make sure that unit test passes?
When you think about it this way, unit testing is more about specifying absolute requirements for specific units of code at a very low level.
You can think of a unit test as an absolute specification.
The unit test specifies that under these conditions with this specific set of input, this is the output that I should get from this unit of code.
True unit testing tests the smallest cohesive unit of code possible, which in most programming languages—at least object oriented ones—is a class.
What Is Sometimes Called Unit Testing?
Oftentimes, unit testing is confused with integration testing.
Some “unit tests” test more than one class or test larger units of code.
Plenty of developers will argue that these are still unit tests since they are white-box tests written in code at a low level.
You shouldn’t argue with these people.
Just know in your mind that these are really integration tests and that true unit tests test the smallest unit of code possible in isolation.
Another thing that is often called unit testing—but isn’t really anything at all—is writing unit tests that have no assert. In other words, unit tests that don’t actually test anything.
Any test, unit test or not, should have some kind of check—we call it an assertion—at the end that determines whether it passes or fails.
A test that always passes is useless.
A test that always fails is useless.
The Value of Unit Testing
Why am I such a stickler on unit testing?
What is the harm in calling unit testing “real testing” and not testing the smallest unit in isolation?
So what if some of my tests don’t have an assert? They are at least exercising the code.
Well, let me try and explain.
There are two major benefits, or reasons, to perform unit testing.
The first one is to improve the design of the code.
Remember how I said unit testing is not actually testing?
When you write proper unit tests where you force yourself to isolate the smallest unit of code, you find problems in the design of that code.
You might find it extremely difficult to isolate the class and not include its dependencies, and that might make you realize that your code is too tightly coupled.
You might find that the basic functionality you are trying to test is spread out across multiple units, and that might make you realize that your code is not cohesive enough.
You might find that you sit down to write a unit test and you realize—and believe me, this happens—that you don’t know what the code is supposed to do, so you can’t write a unit test for it.
And, of course, you might find an actual bug in the implementation of the code as the unit test forces you to think about some edge cases or test multiple inputs which you may not have accounted for.
By writing unit tests and strictly adhering to having them test the smallest units of code in isolation, you find all kinds of problems with that code and the design of those units.
In the software development lifecycle, unit testing is more of an appraisal activity than a testing one.
The second main purpose of unit testing is to create an automated set of regression tests which can operate as a specification for the low-level behavior of the software.
What does that mean?
When you change shit, you don’t break shit.
In that way, unit tests are tests: regression tests.
But the purpose of unit testing is not to merely build these regression tests.
In the practical world, very few regressions are caught by unit tests since changing the unit of code you’re testing almost always involves changing the unit test itself.
Regression testing is much more effective at the higher level as a black-box testing activity because, at that level, an internal structure of the code could be changed while the external behavior is expected to remain the same.
Unit tests test the internal structure, so when that structure changes, the unit tests don’t “fail.” They become invalid and have to be changed, thrown out, or rewritten.
Now you know more about the true purpose of unit testing than most 10-year software development veterans.
What Is Test-Driven Development (TDD)?
Remember that post awhile back where we talked about software development methodologies, and the waterfall methodology often didn’t work out practically because we never had complete specifications up front?
TDD is the idea that, before you write any code, you write a test that acts as a specification for exactly what that code is supposed to do.
This is an extremely powerful concept in software development but is often misused.
TDD usually means using unit tests to drive the creation of the production code being written, but it can be applied at any level.
For the purposes of this post, though, we are going to stick with the most common application: unit testing.
TDD flips things around so that instead of writing the code first and then writing unit tests to test that code (which we know isn’t the case anyway), you are going to write the unit test first and then write just enough code to make that test pass.
In this way, the unit test is “driving” the development of the code.
This process is repeated over and over.
You write another test that defines more functionality of what the code is supposed to do.
You change the code or add code to make the test pass.
Finally, you refactor the code—or clean it up—to make it more succinct.
This is often called “Red, Green, Refactor” because at first the unit test fails (red), then code is written to make it pass (green), and finally the code is refactored.
What Is the Purpose of TDD?
Just as unit testing itself can be a best practice that is misapplied, TDD can be as well.
It’s very easy to call what you are doing TDD and to even follow the practice and not understand why you are doing it or the value—if any—it is providing.
The biggest value of TDD is that tests happen to make excellent specifications.
TDD is essentially the practice of writing unambiguous specifications, which can be automatically checked, before writing code.
Why are tests such great specifications?
Because they don’t lie.
They don’t tell you code should work one way and then after you spend two weeks pounding Mountain Dew and everything works, tell you it should actually work another way and “it’s all wrong; that’s not what I said at all.”
Tests, if properly written, either pass or fail.
Tests specify in no uncertain terms exactly what should happen under a certain set of circumstances.
So, in that respect, we could say the purpose of TDD is to make sure we fully understand what we are implementing before we implement and that we “got it right.”
If you sit down to do TDD and you can’t figure out what the test should test, it means you need to go ask more questions.
The other value of TDD is in keeping the code lean and succinct.
Code is costly to maintain.
I often joke that the best programmer is the one who writes the least code or even finds ways to delete code because that programmer has found a surefire way to reduce errors and to decrease the maintenance cost of the application.
By utilizing TDD, you can be absolutely sure that you do not write any code that is not necessary since you will only ever write code to make tests pass.
There is a principle in software development called YAGNI, or you ain’t going to need it.
TDD prevents YAGNI.
A Typical TDD Workflow
It can be a little difficult to understand TDD from a purely academic perspective, so let’s explore what a sample TDD session might look like.
You sit down at your desk and quickly sketch out what you think will be a high-level design of a feature which allows a user to login to the application and change their password if they forget it.
You decide that you are going to first implement the login functionality by creating a class that will handle all the logic for doing the login process.
You open up your favorite editor and create a unit test, called “Empty login does not log user in.”
You write the unit test code to create an instance of a Login class (which you haven’t created yet).
Then, you write test code to call a method on the Login class that passes in an empty username and password.
Finally, you write an assertion or assert, which asserts that the user is indeed not logged in.
You attempt to run the test, but it doesn’t even compile because you don’t have a Login class.
You remedy that situation by creating the Login class along with a method on that class for logging in and another for checking the status of a user to see if they are logged in.
You leave the functionality in this class and methods completely empty.
You run the test and this time it compiles but quickly fails.
Now, you go back and implement just enough functionality to make the test pass.
In this case, it would mean always returning that the user is not logged in.
You run the test again, and now it passes.
On to the next test.
This time you decide to write a test called “User is logged in when user has valid username and password.”
You write a unit test that creates an instance of the Login class and try to login with a username and password.
In the unit test, you write an assertion that the Login class should say the user is logged in.
You run this new test, and of course, it fails because your Login class always returns that the user is not logged in.
You go back to your Login class and implement some code to check for the user being logged in.
In this case, you’ll have to figure out how to keep this unit test isolated.
For now, the simplest way to make this work is to hardcode the username and password you used in your test and if it matches, return that the user is logged in.
You make that change, run both tests, and they both pass.
Now you look at the code you created and see if there is a way you can refactor it to make it more simple.
So on you go, creating more tests, writing just enough code to make them pass, and then refactoring the code you wrote until there are no more test cases you can think of for the functionality you are trying to implement.
These Are Just the Basics
So, there you have it.
Those are the basics of TDD and unit testing—but they are just the basics.
TDD can get a bit more complex when you truly try and isolate units of code because code is connected together.
Very few classes exist in complete isolation.
Instead, they have dependencies, and those dependencies have dependencies and so on.
To handle situations like these, veteran TDDers make use of mocks, which can help you to isolate individual classes by mocking the functionality of dependencies with pre-setup values.
Since this is a basic overview of TDD and unit testing, we won’t go into detail here about mocks and other TDD techniques, but just be aware that what I presented in this post is a somewhat simplified view.
The idea is to give you the basic concepts and the principles behind TDD and unit testing, which hopefully you now have.
Hey John, does it ever make sense to go back and create unit tests for code that never had it in the first place?
Yeah, well… maybe.
That’s the million dollar question.
You really have to ask yourself why you are doing it.
Are you doing it because it makes you feel all good inside and warm and fuzzy to have a bunch of unit tests for all your code?
Or… are you doing it because you think that creating those unit tests is likely to help you better understand the code, make it more robust against a bunch of changes you are going to introduce, or some equally valid reason.
Don’t just create unit tests because “it’s a best practice” or because “all code should have unit tests.”
Try to at least be a little pragmatic and have a real reason for going back and creating unit tests–yes, I know this may challenge your OCD and completionist tendencies. Afterall, I only wrote this “Hey John” because one of my editors made a comment asking this question and I feel like I must address every single point every editor makes.
We’ll go to rehab together… someday… I promise.
Hey John, does it make sense to write unit tests without doing TDD? Do they go hand-in-hand?
Now you are just trying to get me riled up–but it’s not going to work.
I’m not taking the bait.
But, I’ll say this:
If you’ve read this post and agree with what I’ve said about what the purpose of unit tests are, you have to really question yourself if those purposes are severely compromised if you go back and write unit tests after the code is written.
Yes, in some cases they are still valid and serve as regression tests, but is it really a wise use of your time?
Would it be a wiser use of your time to spend a little more time and effort utilizing a TDD approach?
I don’t want it to seem like I’m leading you here, because you have to really answer these questions for yourself.
I’ve found instances where TDD and unit testing made sense.
I’ve found instances where neither made sense.
And I’ve found instances where TDD didn’t make sense, but going back and creating some unit tests did.
Just don’t do things because “you’re supposed to do them;” always be pragmatic.
Get it?
Good.
Author Bio
John Sonmez is the author of the perennial top-selling Soft Skills: The Software Developer’s Life Manual and the founder of Simple Programmer.
In The Complete Software Developer’s Career Guide, John shares the principles and knowledge that took him from teenage hacker to highly paid senior development and consulting positions—and by age 33, early retirement and a second career as an entrepreneur.
Today he runs the hugely popular Simple Programmer blog and YouTube channel, where he helps millions of developers every year to master the career and life skills that made all the difference in his success.
Bonus tip: Building digital products with Usersnap
I know, I just talked about test driven development. Last but not least, I wanted to give you a heads-up on Usersnap, which a great solution for development teams. Actually, it is used by startups, as well as companies like Facebook, Google, and Microsoft.
Capture user feedback easily. Get more insights and make confident product decisions.
Getting feedback has never been easier and we hope you’ve realized that after reading this article. Let us know what you think, your feedback is important.
And if you’re ready to try out a customer feedback software, Usersnap offers a free trial. Sign up today or book a demo with our feedback specialists.
