7 Different Types of Software Testing

Most developers have no clue about how testing is actually done, and how valuable the understanding of software testing basics can be for developers who really want to excel in their careers.

I owe a large amount of the success I have had in my career as a software developer to my background in testing.

Why?

Because I learned it the hard way.

Back in my days…

Quick Usersnap PSA: if you need a guide to customer feedback, skip jump and fly your way over there now! That’s all!

I’m John Sonmez, best selling author of “Soft Skills: The Software Developer’s Life Manual” and one of my first official jobs in the software development industry was that of a tester.

In this post, I will tell you about the 7 most used software testing approaches and how you can use them to become a better software developer.

My job entailed looking at stacks of papers that were printed out by a new printer we were testing at HP and comparing them to the “master” printouts produced by older printers.

I didn’t actually do the comparison of the pages myself. Instead, I would execute the tests, someone else would compare the printouts, and I’d look at the differences they flagged.

With each difference, I would review and decide, based on the test, whether the result was a true failure or defect. If it was the latter, I’d write up a defect report for a developer to look at—and possibly fix.

That background caused me to look at the code I was writing a little differently and to realize that my job as a software developer wasn’t just to implement features and fix bugs, but to make the software I was writing work correctly and as intended.

Seems like a simple and obvious idea, but if you don’t at least know the basics of testing, you probably aren’t going to have the best idea of what “working correctly and as intended” actually means.

The basic idea behind software testing

Usually, new programmers don’t understand testing. They don’t think it’s necessary.

At a surface level, it can seem a bit extraneous.

Do we really need to test that code? I ran it on my machine and it worked perfectly, so let’s just ship it.

[Tweet “Testing, at its core, is really about reducing risk.”]

The goal of testing software is not to find bugs or to make software better. It’s to reduce risk by proactively finding and eliminating problems which would most greatly impact the customer using the software. Users may experience frustration with your software if it lags and hangs. Lagging and hanging can be reasons for the overall slow performance of their machine.

This goal is especially relevant to testing enterprise software with complex digital workflows. In such a case, ensuring smooth functioning, usability, compatibility, and data consistency of the software is of paramount importance due to the higher impact of any issues uncovered after a release.

Customers can be impacted by the frequency of an error or undesired functionality, or by the severity of the problem.

• Impact by the frequency of an error:
  If you had a bug in your accounting software which caused it to freeze up for a second or two whenever a value higher than $1,000 was entered, it would not really have a huge impact; but that would be a high enough frequency to be very annoying to the customer.
• Impact by the severity of the problem:
  If you had a bug in the accounting software that caused all of the data to become corrupted every 1,000th time the data was saved, that would be a huge impact, but very low frequency

The reason I define software testing in this way is because—as any tester will tell you—you can never find all the bugs or defects in a piece of software and you can never test every possible input into the software. (For any non-trivial application.)

So, the idea is not to find every single possible thing that is or can go wrong, or even to verify the software against a spec—as some people like to define software testing—because both are impossible.

(Oh, also if you ever find a complete spec for any application in your experience as a software developer, let me know.)

Instead, the focus and main idea behind software testing is to reduce the risk that the customer is greatly impacted in a negative manner when using the software.

Typically this is achieved by first prioritizing what areas of the software are likely to have the biggest impact (i.e. risk), and then deciding on a set of tests to run which verify the desired functionality in that area.

When the actual functionality deviates from the desired functionality, a defect is usually logged and those defects are prioritized based on severity.

Some defects get fixed, other defects are low enough impact that they are just noted and left in the system.

Common software testing approaches

The world of testing and quality assurance is huge.

Just like the development world has many concepts and methodologies for creating software, there are many ways to think about how to test and the field is changing all the time.

Even in name.

Early in my career, it could be perceived as a slight or insult to call someone who worked in testing a tester; they preferred to be called QA (or quality assurance) professionals.

Just a year or two ago, I attended a testing conference and I made the mistake of calling someone a QA person. They corrected me and said that tester was the preferred term.

You can’t win them all.

Anyway, let’s talk about the different kinds of testing, so you can get a general idea of what someone is talking about when they throw around these terms—which you will hear often in the software development world.

This is not an exhaustive list by any means.

1. Black-box testing

Black-box testing is simply testing as if the software itself was a black box.

It’s one of the most common forms of testing—and really a way to describe a whole category of testing—is black-box testing.

When you do black-box testing, you are only concerned with inputs and outputs. You don’t care how the actual outputs are derived.

You don’t know anything about the code or how it works, just that for a given set of inputs into the software, a given set of outputs should be produced.

Most testing is done in this fashion because it is largely unbiased. It either works or it doesn’t.

[Tweet “Every Developer should know at least 1 of these 7 common software testing types”]

2. White-box testing

Real white-box testing is when you understand some of the internals of the system and perhaps have access to the actual source code, which you use to inform your testing and what you target.

White-box testing is pretty much the opposite of black-box testing.

With white-box testing, you have at least some idea of what is going on inside the software.

Oftentimes, unit testing is called white-box testing, but I disagree. Unit testing is not testing at all.

With black-box testing, you’d have no way to know these two conditions existed, so you’d be very unlikely to test for both of them unless you just got lucky.

3. Acceptance testing

The basic idea of acceptance testing is that you have some tests which test the actual requirements or expectations of the customer, and other tests that run against the system as a whole.

Sometimes it’s called user acceptance testing (short: UAT).

Sometimes it’s called system testing.

This kind of testing could be testing the functionality of the system or it could be testing the usability or both.

The idea is that acceptance testing tests what is expected versus what actually happens.

4. Automated testing

Automated testing is any testing in which the execution of the test and the verification of the results is automated.

So, you might automate the testing of a web application by running scripts which open up a web page, input some data, push some buttons and then check for some results on a page.

You could also automate the testing of an API by writing scripts which call out to the API with various data and then check the results that are returned.

More and more of testing is moving towards automated testing because manually running through test cases over and over again can be tedious, error-prone and costly–especially in an Agile environment where the same set of tests may need to be run every two weeks or so to verify nothing has broken.

5. Regression testing

This brings us to regression testing, which is basically testing done to verify that the system still works the way it did before.

The purpose of regression testing is to make sure the software doesn’t regress in functionality.

This is extremely important with Agile development methodologies where software is developed incrementally and there is a constant potential that adding new features could break existing ones.

Most automated tests are regression tests.

In fact, you could really make the argument that all automated tests are regression tests since the whole purpose of automating a test is so that it can be run multiple times.

6. Functional testing

Functional testing is another broad term used in the testing world to refer to testing activities where what is being tested is the actual functionality of the system.

This might seem obvious.

You might be thinking “duh, what else would you test if you didn’t test the functionality of the system.”

But, it turns out that you can test all kinds of things that aren’t related to functionality, like performance, usability, resilience, security, scalability — I could go on and on, believe me.

So, functional testing is the kind of testing where you are really concerned with the system doing what it is supposed to do from a functional perspective.

If I put in this input and push this button, do I get this expected output?

I don’t care how long it takes. I don’t care if the screen flashes bright red and the computer starts to smoke, do I get my result?

7. Exploratory testing

I like to make fun of exploratory testing and call it “lazy-ass testing.”

It really pisses testers off when I do that.

But, there is definitely some legitimacy to the idea of exploratory testing and perhaps I am a bit too harsh and judgmental.

The idea behind exploratory testing–when done correctly–is that you have some guidelines and a basic plan of which application areas you are going to test and ways you are going to test them.

Then, you go about without actual test cases and explore the application, looking for things that might be wrong or behavior that is unexpected.

Oftentimes, exploratory testing sessions are recorded, so that if an error is found, the problem can be reproduced by retracing the steps taken by the exploratory tester.

While I’m generally not a huge advocate of this kind of testing, I do have to acknowledge its merits, as exploratory testing can often uncover bugs which no rational test case would have ever been designed to exploit.

Other forms of testing

Truly we’ve only scratched the surface of all the different types and classifications of testing.

Many other forms of testing exist, including:

• Load testing- How an application performs under a heavy load
• Performance testing- Performance of the application based on certain scenarios
• Recovery testing- Recovery from error conditions or hardware issues
• Security testing- The security of the system
• Stress testing
• Usability testing
• Accessibility testing

The list goes on and on.

The testing process

Different organizations are going to have very different ideas of how testing should be done and what process should be followed.

You’ll also see plenty of formal specifications produced by various testing organizations out there which cover the “testing process.”

So, again, like a large amount of what I said about testing, the idea here is not to be prescriptive or to perfectly model the perfect testing process, but rather to give you an idea of what the testing process is like in general and what it entails.

I like the pragmatic approach to life–and testing.

Step 1 – Development of a test plan

Testing usually begins with the development of some kind of test plan.

• How will things be tested?
• What is our strategy for testing?
• What kind of testing are we going to do?
• What features are we going to test?
• What is the schedule?

These are all questions that are generally answered in the test plan, or if the test plan is not a formal document, the test planning for a project.

Step 2 – Design Tests

Next, the tests are usually designed at a high level based on the requirements or functionality of the system.

At this stage, a tester might be coming up with a list of general test cases which will be run, what kinds of conditions will be tested, and coming up with what will be needed to perform the tests.

Step 3 – Test Creation and Execution

After that, the tests are usually created and executed.

Sometimes this occurs as a single step.

Sometimes tests are written in test management software first and executed later.

Step 4 – Log Results

The results from the test execution are recorded and evaluated and any bugs or defects are usually logged into some kind of bug tracking system.

Bugs are prioritized and sent to developers to fix.

Fixed bugs are retested and this cycle continues until the software meets the quality standards criteria for a shippable code.

And that’s basically it.

Plan how to test, design the tests, write the tests, execute the tests, find bugs, fix bugs, release software.

How testing works on Agile teams

The standard process of testing tends to run into some problems on Agile teams where new features are being coded and implemented every couple of weeks or so.

Many teams try to either strictly follow the standard testing process or completely throw it out the window instead of working it into the Agile testing lifecycle of software development process.

Both of these approaches are wrong.

Instead, the focus really has to change to developing the test cases and test scenarios up front, before any code is even written and to shrink the test process into a smaller iteration, just like we do when we develop software in an Agile way.

This just means that we have to chop things up into smaller pieces and have a bit of a tighter feedback loop.

Instead of spending a large amount of time up front creating a testing plan for the project and intricately designing test cases, teams have to run the testing process at the feature level.

Each feature should be treated like a mini-project and should be tested by a miniature version of the testing process, which begins before any code is even written.

In fact, ideally, the test cases are created before the code is written at all–or at least the test design, then the development of both the code and the test cases can happen simultaneously.

Another major consideration with Agile testing is automation.

Since new software is released on very short iterations, regression testing becomes more and more important, thus automated testing becomes even more critical.

In my perfect world of Agile testing, automated tests are created before the code to implement the features is actually written–truly test driven development–but, this rarely happens in reality.

Testing and you, the developer

What about you, the software developer? What is your role in all this testing stuff?

Do you even have one?

Yes. Definitely!

One of the big failings of software development teams is not getting developers involved enough or taking enough ownership for, testing and the quality of their own code.

As a software developer, you should be more concerned with quality than anyone else.

You can not have the mindset that QA will find the bugs in your code.

Instead, you should absolutely make it your responsibility to find and fix the bugs before your code goes into testing.

The reason is fairly simple. The further along in the development of software a bug is found, the more expensive it is to fix.

Think about it this way.

If you test your own code thoroughly and find a bug in that code before you check it in and hand it over to QA, you can quickly fix that bug and perhaps it costs an extra hour of time.

If you take that same bug, and you don’t take the time to find it yourself and fix it, the process might go something like this:

?‍? A tester runs a test which finds the bug in your code.

?‍? The tester re-runs the test to make sure the bug is valid.

?‍? The tester files a defect in the bug tracking software.

?‍? A development manager decides that the bug is severe enough for you to work on and the bug is assigned to you.

?‍♂️ You try to recreate the defect, but it seems to work on your machine.

?‍? The tester reproduces the bug and puts more detailed steps in the bug report.

?‍? You finally are able to reproduce the bug and you fix it.

✅ You update the bug report with the fix.

?‍? The tester goes back and checks that the bug is actually fixed and marks the defect as resolved.

That’s a large amount of everyone’s time to waste… ?‍♂️

Not saying you’re a lazy ass, but…

Perhaps you should take that extra 10 minutes to test your own code before checking it in.

You won’t catch everything, but if you can even catch 10% of the bugs that would otherwise make it to QA, you’ll be saving quite a bit of time, don’t you think?

Ok, so by now, hopefully, you have a decent idea of what testing is, the purpose of testing, what kinds of testing can be done and your role in that whole process.

Word of truth

I just wanted to cover some of the basics here which you’ll hear about and see in everyday conversations as a software developer.

Hey John, I’m a bit confused. Black-box testing sounds a whole lot like functional testing. What’s the difference? Oh, and also the same question for regression testing versus automated testing. Aren’t all automated tests essentially regression tests?

Ok, shhh… I’m about to tell you a little secret that, well, kind of pisses off QA people–I mean testers.

So. Many of these testing terms are basically the same thing. Sometimes I feel like the whole testing profession feels the need to invent a bunch of terminology and add a bunch of complexity to something that is inherently simple.

Now, don’t get me wrong, testing is important–and it does require skill to be good at it–but it’s not all that complicated… really.

To address some of the specifics. Basically, functional testing could be either white-box or black-box, but usually it’s going to be black-box. Black-box and white-box testing just refer to how the functional testing or other testing is done. It’s really just a type of functional testing. Are you looking at the code to give you hints about what to test or are you treating the whole thing like a mysterious black box? Black-box testing is just the higher-level concept or idea of testing an application without being able to look at the internals to see how it’s implemented.

If you are doing effective functional testing, you are probably doing it in a black-box way, although it’s conceivable that looking at the code might give you an idea of some edge cases or special cases you might want to test which you might have missed otherwise. For automated testing versus regression testing, again, we are dealing with a higher concept and implementation.

Regression testing is the concept. It’s the idea that when something breaks–or before it does–you should create some set of tests that ensure the functionality of the system doesn’t go backwards, or regress. Automated tests serve this purpose really well since, well… they’re automated. So, pretty much all automated tests will be regression tests, but it is possible to have manually-run regression tests to make sure the software doesn’t regress in functionality.

If you decide to become a tester and want to pass a job interview for a testing position, you should probably know all this stuff and be able to expound upon why exploratory testing is actually a valid way to test things and how user testing and acceptance testing are not the same things.

But, if you are a software developer, honestly, it’s only important for you to have an idea of the concepts and what the vocabulary is, and to understand the real idea behind testing, which is risk reduction.

So, don’t worry about all the semantics, and focus on the big ideas. That’s what is important.

Author Bio

John Sonmez is the author of the perennial top-selling Soft Skills: The Software Developer’s Life Manual and the founder of Simple Programmer. This article is an excerpt from The Complete Software Developer’s Career Guide by John Sonmez. To get the entire book delivered to your inbox, go here. In The Complete Software Developer’s Career Guide, John shares the principles and knowledge that took him from teenage hacker to highly paid senior development and consulting positions—and by age 33, early retirement and a second career as an entrepreneur.

Today he runs the hugely popular Simple Programmer blog and YouTube channel, where he helps millions of developers every year to master the career and life skills that made all the difference in his success.

Bonus tip: Software Testing with Usersnap

I know, I just talked about the most common types of software testing. Last but not least, I wanted to give you a heads-up on Usersnap, which is a great solution for UAT testing and user testing, used by companies like Facebook, Red Hat, and Microsoft. Get started with Usersnap and deliver what users want. Collect your first 20 feedback items for free, upgrade to continue acting on the insights.

Resolve issues faster with visual bug reporting.

Simplify and reduce issue & bug reporting efforts with screen recordings, screenshots, and annotations.

And if you’re ready to try out a visual bug tracking and feedback solution, Usersnap offers a free trial. Sign up today or book a demo with our feedback specialists.