If you’ve ever authored test cases for a requirement, you know the drill for negative test cases. Your product owner is going to ask what negative conditions you have included in your test suite. It’s almost inevitable. For those without this experience, it’s good practice to test the negative paths of a requirement in addition to the positive or “happy path” test cases. The first time this question is asked should not be when your product owners are reviewing the test cases. This requires some critical thinking and an imagination to think of what could go wrong. Even with all of this, you’re still not guaranteed full coverage of the negative paths.
Let’s take a look at an example – A simple signup form. You have multiple elements: Username, Password, First Name, Last Name, and Email. For the sake of simplicity, the requirements are simple: All fields are required, Password must contain alphanumeric characters and at least 1 symbol, and the username must be available. This would be graphed as like the following:
Generating the test cases in the BBT Tool results in 1 positive test case:
All of this looks great and proves you can create an account when nothing is wrong and the user fills out the form correctly. What happens when they don’t? What happens when they forget to provide their last name? What happens when their password doesn’t meet the complexity requirements? Now we’re getting into negative path test cases. There are a lot of things that could go wrong and it’s a QA Analyst’s job to account for all of those situations. It’s not sufficient to only account for those situations, but you have to test them, regress them after defect fixes, and automate them. Suddenly, this just became a huge effort for such a simple requirement. Don’t let that scare you into cutting corners or “reducing scope” as some like to call it. BBT minimizes this huge effort into a quick and simple task. Without any additional effort, the BBT Tool generates the negative path test cases (31 total for this example):
BBT takes the provided positive contexts and creates all permutations of negatives path test cases with the negative condition of each context by default. This is without any additional work on your part! To understand what the BBT tool is doing here, let’s take a look at the truth table below. In it, we have defined all permutations of the graph above. You’ll see we have one positive test case and 31 negative test cases for a total of 32 test cases. An easy way to calculate the total number of permutation is to use the formula 2n where n equals the number of contexts. In this case, 25 = 32. The test case count grows exponentially larger with multiple positive or negative states. This formula only applies when you have only one negative state of each context. This is also excluding constraints which would reduce the overall number of permutations by masking invalid test scenarios. We’ll cover that more later. As you can see below in the provided truth table, each of the negative cases ties directly to a test case in the generated list from the BBT Tool.
Using the truth table above, we’re easily able to generate test cases for each line. For instance, the first negative test case reads like following:
Given [Not] Valid Username, [Not] Valid Password, [Not] Valid First Name, [Not] Valid Last Name, and Valid Password
When Click Create Button
Then [Not] New Account is Created
The [Not] tag simply represents a negative state of each context or outcome. [Not] Valid Username could mean many things depending on the requirements provided: Username is not long enough, Username is taken, Username contains spaces, etc. It’s up to the QA Analyst to define what that negative state means based on their knowledge and interpretation of the requirements. Regardless of the terms of the negative state, it still results in a negative state outcome: [Not] New Account Created.
In our next blog, we will go over how BBT handles multiple negative test conditions and how you can use it to get complete coverage of your requirements.