I reckon that any of us using a computer has the experience that the applications you’re using do not always act the way you expect it to do. This might be a matter of ignorance, i.e. simply not knowing how it works, but it also could be that they are not behaving as intended. The latter is what we call a bug. In the words of Wikipedia:

A bug is an error, flaw or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways.

I tend to narrow this down to:

A bug is an untested feature.

This short definition highlights that any kind of bug, be it a real fault, an unexpected result, or whatever, boils down to the fact that this behavior was not revealed by testing. No scenario was thought up to uncover this behavior before the application was released, only to be discovered in daily usage.

In many software implementation projects this discovering bugs in daily usage has been a substantial part of detecting bugs in the software. This is due to many reasons, just naming a number of obvious ones:

• restricted resources, both financial and human
• restricted (lead) time often causing a squeezed test schedule
• lack of in-depth knowledge of the functionality
• non-existence of tooling to support structured testing
• ignorance regarding the assurance of the quality

And too often because of something simple as no one really liking testing.

Having read the title of this article, it might not come as a surprise that I regard test automation a grand helping hand in overcoming this. Before elaborating on this I first would like to zoom in on testing software as such.

software testing

Software testing, in the current context, is also known as application testing. Verifying whether the software under test is doing what it was intended to do. Or in other words: checking whether a given input yields the expected output. Note that this should be exactly the same every time again, unless we deliberately change this behavior.

Basically with every change we impose on the software we should validate that its behavior is what was meant to be. Be it as before, as nothing has changed on that specific part, or, with an intentional alteration, as it now should be. In this way we assure that everything is still working as it should. Daily practice, however, shows often to be unruly for reasons mentioned above. We tend to hide behind them and not leverage this daily practice, with the user suffering the consequences first. Eventually all suffer, as any bug found downstream cost a multitude of when it was found upstream. You might read some more on this here.

So, how can we position ourselves, both implementers and end-user company, to improve this practice? How can we really ensure the software is still working? How can we promote that our tests do cover all code that make up the features being used? That they are repeated exactly the same in each test run?

automate what is repeated

These questions pose us a challenge that had always been there, but somehow evaded in many projects. In the cloud world we are all slowly moving to an additional challenge that changing the game: how to handle shorter update cycles?

To head both challenges you either have to extend your human resources or invest in automating those things that are repeated. As might be very apparent from the above, a major part of our test effort is recurring and, with the tooling available, automatable.

You might counter me saying this comes with a substantial cost that so far has not been part of your project budgets. Yes, I cannot deny that. However, if you have read the article pointed to, with respect to the cost up up-/downstream finding of bugs, you will very well have understood that sticking to your non-automated testing has always been a substantial, often, not budgeted cost. Manual testing is a recurring vast cost, where automated test become cheaper with every rerun and can be done at any time, as, in general, you do not need to replan resources for that.

new possibilities

Many of you might know me as an evangelist of test automation in the Microsoft Dynamics 365 Business Central world. One of the challenges I am facing in this role, is not so much to convince you of what has been discussed so far, but to let you understand that test automation brings in a whole lot of new possibilities you have never thought of, let alone experienced, before. These new possibilities will genuinely leverage your daily practice:

• as the major part of tests will now be run automatically, manual testing will shift focus to more meaningful matters; a tester can now really go and try the break the-damn-thing
• there will be an almost direct feedback on new development; no need to wait for the testers to get their hands on it
• a whole new tool is added to the toolset of a developer as each developer now can run relevant automated tests while working on a bug fix or a new feature
• each bug – a missed scenario – is quickly added to existing collateral and will be checked in every next test run
• a shift will happen from bug chasing to feature development
• continuous integration (CI) and continuous delivery (CD) come in play, allowing you to build and deploy your solution at (almost) any time and any place
• test automation has a need for detailed test cases before coding starts, which will lead to a better discussion of the requirements

Test automation will surely make development less stressful as the release of the software will lesser be “hoping for the best”. You will be more in the known of the quality. And it’s fun, it really is. And it goes without saying that any bug prevented to go out in production will save the end-user a lot of hassle, frustration and possible resistance to your software.

educate yourself

Like any of your skills test automation needs to be learned. It’s not rocket science. Experiences are that it still is a threshold to take for all involved. That’s why we pay attention to this skill in our junior training.

Test Automation is not a buzz word, but a grand helping hand.

read more

If you are in search of more arguments to get test automation on the agenda in your organization read chapter 1 of my book Automated Testing in Microsoft Dynamics 365 Business Central.