Test automation and automated test design

Nowadays time to market is reduced while the complexity of applications increases. There is no question for test automation – it’s a must. However, test automation is not enough, test design should also be automated.

Obviously, it doesn’t mean that a tool will generate the tests. Currently it’s impossible. Test design automation means that tests are generated form models and you should write these models, instead of the test cases.

There are many types of models, but basically it can be graphical or text-based. Good models should be understandable for every stakeholders and can be created without programming knowledge. One of the possibilities is using acceptance criteria.

What are the acceptance criteria?  In Agile, acceptance criteria refer to a set of predefined requirements that must be met in order to mark a user story complete. However, we can use acceptance criteria in any software development methodology. Generally, acceptance criteria for a given requirement are statements by which the implemented requirement can be validated. Some needs for acceptance criteria:

  • they should be testable;
  • they should be clear and concise;
  • everyone must understand your acceptance criteria.

What does it mean ‘it should be testable’? It means that executing the software you can validate the fulfilment of an acceptance criterion. This means that it should be concrete. Here is an acceptance criterion taken from the web:

Given The email address is valid.

When The email address is authenticated.

Then The email address is authenticated.

Is it testable? No! We cannot validate this for each email address. It’s a general description, but it’s not testable. Let’s consider a similar one:

When I fill in a valid “Username” and “Password” fields with my authentication credentials
and I click the Sign-In button
Then the system signs me in”

Is it testable? Yes, it is as I have concrete username and concrete password. Can I use this for test automation? No, as the actual user name and password are not in the acceptance criterion.

Now let’s consider the third one created by me:

WHEN Username IS Smith AND password IS 2a4b6c

THEN Message IS ‘you are logged in’

It is obviously testable and can be automated as the necessary values are available. Here the IS keyword connects value (called choice) to a parameter (called category).

Why is it so important to write acceptance criteria instead of test cases directly? When you write test cases, you easily may make mistakes. A test case should contain everything.  This means that it contains the precondition, which makes it possible to go to program point of the feature you want to test. It also contains values which do not relate to the requirement to be validated, but makes the test non-understandable. The acceptance criteria should contain only the essence of a test case, which relates to the requirement.

Here is a figure showing the difference between a test case and an acceptance criterion:

Now, let’s consider the process of testing from test design to test execution. The starting point is a set of requirements or user stories.  From these we can design the acceptance criteria. However, we can design acceptance criteria in different ways. This is because we have to decide when a set of acceptance criteria  is able to validate a requirement. If a requirement is very important, i.e. the related risk is very high, we should make more ACs. Thus, the inevitable step to design ACs is the risk and complexity analysis.

Based on the requirements and taken into account the result of the risk and complexity analysis we can make the ACs. From these ACs a tool can generate manual and executable test cases. Here is example:

Feature Login – happy path

Logged in:

WHEN Login name IS Hall AND Password IS 2@A9ih

WHEN Login button IS #pressed 

THEN Message IS successful login

Generated code:

SetValue( “Login name”, “Hall” );

SetValue( “password”, “2@A9ih” );

ClickOn( “Login button”);

VerifyValue( “Message”, “successful login”);

The code usually contains a keyword, a parameter and a value, sometimes only the keyword and a parameter.

The advantage of this method is obvious. You don’t need to write any code. You can also design the ACs before the implementation starts. Another key advantage of using ACs is maintenance. You should never modify, delete or insert a test case. When a requirement changed, you should modify the related acceptance criteria. That’s all. To learn more read our book or visit the website of the related tool: https://4test.io/

2 thoughts to “Test automation and automated test design”

Leave a Reply

Your email address will not be published. Required fields are marked *