It’s hard not to compare BBT(behavior based testing), without raising some questions about BDD(Behavior driven development). In many aspects they do the same thing, but are very different in their own rights. BBT is designed and developed heavily on BDD concepts, just like BDD is based on TDD(test driven development). Like all new tools, holes are found that create the need and want for a more improved version. This is where BBT comes into play, using a lot of what works with some improvements to make it all its own. Throughout this blog I’ll be going over the differences with process, setup, and automation of both technologies.
Let’s start off with the process, for these two tools, they’re very different. For instance, BDD calls for the product owner to work with the developer as well as the QA resource to create feature files. Feature files are files that contain tests in a plain English format, this allows business to fully understand what’s being tested. From my experience I’ve noticed a huge problem with this approach. First being most product owners don’t like writing the feature files, and the ones that do, their tests require some modification before it could be used in any automation. This is where I feel BBT is far more superior. Product owners can continue to write acceptance criteria, and QA engineer will handle the rest. In the BBT process, all acceptance criteria are graphed out, this allows business to easily read a graph and identify addition pitfalls.
Keep in mind that these tools work very differently. In the BDD approach, the majority of the tests cases are just covering a specific acceptance criteria and that leaves a lot of room for error. With BBT, a truth table is devised from the graphed out criteria assisting the engineer to account for all test cases. Even with all these differences, a lot remains the same. For instance, BBT uses contexts and outcomes, which can be compared to the given, when, and then steps. Not only can they be compared but they’re pretty much the same thing, each step is reserved for a specific set of commands in automation that can be reused over and over when using the same phrasing.
Now on to setup, once again these two tools are verify different. In this case let’s start off with what’s similar about the two. For instance, both tools have a manual approach that require no coding at all. BDD has feature files, which is only one part of the process. The other being step definition files which contain the actual code with a regex linking the two together. BBT the open source manual approach starts with graphing and truth tables before any coding is done. Although SQAsquared has created a BBT graphing tool that will not only read the graph but create all possible positive and negative test cases for you. The major difference here is how the tests are setup. BDD requires the feature file, that looks for a test case title along with individual steps to go along with it. Whereas BBT looks only for the test case title which has key information about what’s needed to get to the appropriate outcome.
There are only a hand full of differences when it comes to the automation portion of these two tools. Like I mentioned earlier many new tools improved versions of existing tools that almost had it all right. For instance, BDD had the logic for automation down solid. It used regex to run human readable files which made it easy for anyone to add tests as well as provided excellent documentation. BBT uses the same logic heavily, all the context and outcomes are directly related to code which can be added to create additional tests. The only real difference here is how the code is called alongside with the terminology used. As stated earlier, BDD – given, when, and then, BTT – context and outcome. When it comes to automation, tests over time become stale and flat out useless. When working with the BBT graphing tool, it will assist with cleaning up all tests that are graphed. This can be achieved by just adjusting the graph to fit your new acceptance needs and simply generating the test cases. Now you have the most updated list of tests without having to go through an modify each and every one individually.
Lets walk through an example, here is a basic BDD test written in the gherkin format(BDD human readable language).
Now from this example we already know this isn’t enough to cover testing, we would then have to add additional tests for all possibilities. This is where BBT thrives, here is an example of the same exact criteria in BBT.
In this graph I have each step laid out, all my given statements on the left which we call contexts. On the right I have the then statement, as well as the action or the when statement needed to get there. So as you probably noticed already, we are doing the same thing in both cases just in a different style. Lets now take a look at the results when we generate test cases from our graph.
The first thing you should see is that it wrote the test with the same exact steps. But BBT doesn’t stop there, like I mentioned before, the tool also generates negative tests as shown below.
Now let’s visualize that your company is using BDD or another automation framework and your test suite has over 2000+ tests. First off that awesome, but now think what would happen if another step was introduced to the product which required an extra step that needed to be added to about 20% of those tests. Sounds like a nightmare to me, but with the BBT tool you have a major win. All you would have to do is update the graph with that extra step and click generate tests. All you tests are now updated. Its as simple as that. With
To sum things up, BDD is a great tool, but the majority companies, at least the ones I’ve worked at, don’t follow its intended use and it only cover what you can think of. BBT is an improved tool based on BDD, specifically in test case management, creation, and coverage; the main headaches of the QA industry. Since BBT is so heavily influenced by BDD making the transition is pretty easy, not many growing pains. The biggest transition for me was the shift in process, the graphing approach verse the identifying and realizing tests approach. Like BDD, BBT can be used in many different testing environments, whether its front end, back end, service based testing, etc. You can find more information and other blogs from how BBT works to BBT on the testing frontlines with Mobile, Business Intelligence, Security, and many others here.