When it comes to web-service testing, tests are pretty black and white, submitting data and expecting data or a response back. There is so much that can be tested which leave many guessing if they got everything, but this is where switching to BBT leaves you stress free. Keep in mind that BBT won’t replace any current development methodologies, only your current testing framework. This will allow seamless integration with any current methodologies BBT is designed to ensure full test case coverage, in the shortest amount of time possible. Think how long it would take you to write test case permutations on a web-service with 25+ fields/values to verify, most likely more than you’d like. BBT will help you make sure that all permutations are captured. Throughout this post I’ll be taking you through how BBT can work for you, not only in the test case identification process but the automation process as well.
Gathering requirements, every web-service has requirements to properly outline each expectation and functionality. Depending on the structure, the success of a web-service can be dependent on many other services, which at times can be a nightmare. When gathering requirements, I felt it best to always get specifics, not ranges or guesses. Failing to do so will most likely lead you down a rabbit hole full of bad habits and production defects. Whether you’re a QA professional or a developer all requirements need to be gathered in order to properly test anything. Every requirement has a specific outcome, and this is where we start. The great thing about BBT is that we can use any tool to help us graph out the outcomes(what we expect, or the verification) and from there contexts(a specific behavior or action). In the BBT process we always start out with all the outcomes first, what we expect. In many, if not all cases this will come in the form of acceptance criteria. From here it’s easy to map out which fields or values depend on or require one another. This will not only help in the automation process by creating a visual flow, but it will provide living documentation of how the web-service works and it will surface any additional questions that need answering.
Uncovering the test, if you don’t know already the manual approach to BBT has only a handful of steps:
- Identifying all possible outcomes
- Identifying all possible contexts
- Graphing out the outcomes and contexts
- Identifying constraints
- Generating a Truth table
- Identifying test cases based on the truth table
- Realizing test case steps
You can find more information about the BBT as a whole by reading up on BBT for dummies. Which will take you through each step thoroughly.
Let’s go over something simple, if I submit a request with a valid user ID, first name and last name, then I expect that users account information. Now from this alone, there are a few unknowns. For instance, what else is require or even optional, and what exactly is being returned? Each value in the request would then be a context, whether it be either required or optional. The response would then be the outcome, what do we expect to see, whether it’s a positive or negative response. Graphing out the logic behind each value now becomes easier to manage when broken down into its simplest form. Example shown below.
Now the example above is pretty easy, created in paint, but from something as simple as this, you can then create the truth table which will allow you to capture all possible positive and negative test cases leaving no stone unturned.
Here’s an example of the same test but with more information about the response. (Image provided by the SQAsquared BBT tool.)
Simply adding additions context, everything left of the successful response oval, can help modify the request made to a specific web-service. As well as adding additional outcomes to the successful response can will ensure all possible values are verified. Typically adding more information to graphs comes hand in hand with updating or enhancing requirements. When doing so, it adds in additional layers to the truth table mentioned previously. This will in turn give more permutations for each value on the graph. For example one additional outcome can generates a minimum of two additional test cases; a success and a failure case. Now depending on the amount of contexts, it can multiply enormously, but it will ensure you have all possible test cases. Keep in mind this process can’t avoid human error, bad logic will grant bad test cases. So be sure to review and double check your work before moving on the true tables and actual coding. Once we coding starts, there is one very important understanding to keep in mind. All test cases are sharing context and outcomes, just in different combinations. Which means each context and outcome need to be coded only once, which cuts down on automation time as well as making easy to manage code.
When it comes down to it, BBT helps saving time, money, creates living documentations, make managing test cases easier, and provides peace of mind. As mentioned previously, there are many graphing tools that can be used, but I found that the SQAsquared graphing tool is by far the best and easiest way of test case creation and graphing. The can also auto-generate the test cases in the BBT framework, leaving you to only apply the steps. The tool actually reads the graphs and based off all the connections, it will write all the test cases for you, no more true tables needed. By no means is this tool required, but it does make life easier and the job even faster. For more information about the open source BBT framework or the BBT tool SQAsquared has created please visit BBT.SQAsquared.com. Also, you can find more information and other blogs from how BBT works to BBT on the testing frontlines with Mobile, Business Intelligence, Security, and many others here