Skip to content

Test Automation As A Service – Return on Investment

2012 March 15
by jonathon.wright

After over 10 years of framework development experience implementing a number of frameworks I finally had a chance to spend some time investigating into the real return on investment numbers (not the ones vendors would like you to believe).

The driving force behind this was when a client ask how to automate over 10,000 business process tests (currently in the form of manual assets) into test automation assets without investing any real additional effort or money? to accelerate regression testing phase in a few weeks time?

Normally I would discuss the options around 5th generation frameworks using Business Process Modelling (BPMNv2 stored in xPDL) to represent Business Process Scenarios (BPS stored in AXML/Workflow v4) to convert the current test assets Business Process Tests (all 10,000 of them).

However, due to the considerable amount time it would take to not only convert existing assets into a usable format to drive the automation solution exceeded the clients times scales. This started me thinking around one of the concept that I have previously discussed on companies becoming ‘Automation Ready’ very similar concept to the CMMi/TMMi as an independent assessment  to provide as a guideline for companies to adhere too:

Automation Maturity Model Index (AMMi)

  • Solution assessment looking directly at the companies Automation challenges and possible opportunities for improvement.
  • Test Asset index to evaluate how the test artefacts are stored and the encapsulate the process that manages them.

The example here was how to take a company using no automation solutions with test assets stored in a legacy file template. In this case the AMMi index  was around the D4 rating with A1 been the very best automation solution and process integration that you could possible achieve   (based on the moment in time when the assessment was completed i.e. Q1 2012. based on the below follow principle):

“The best aspects of proven test approaches that have evolved over the past decade, and this echoes some of the changes towards more lean and agile business methodologies. Which are in a constant state of evolution – just as the underpinning technology evolves over time.”

With this in mind the best possible automation solution today may not be the right solution for every client. Therefore, sometimes half the battle is just starting on the Automation journey by taking them one step closer to becoming ‘Automation Ready’.

In this example the recommendation to follow the age old approach of Manual’mation to encapsulate the manual execution phase using proven FastForward technologies to capture a journal that can be used to drive ‘Automation by Example’ in the future to in a generic XML format extracting all the necessary test artefacts to allow them to be able to run all 10,000 scripts for regression purposes.

Check out the following presentation for further information: