AI-Enabled Test AutomationMaintenance Leveraging Self-Healing

ust-global.com

Abhilash. G & Shamir Ahamed. S, UST Global

Copyright © 2020 UST Global Inc 1

Typical conditions that makethe automation scripts fail?

Design Considerations

Self-Healing at work

Test automation teams have scaled many heights in automating enterprise applications at large. However, they run into challenges at execution time when applications undergo up-dates. A minor revision in the application user interface can result in test failures and become roadblocks for the test automation execution. Changing dynamics of end-user expectations are driving organizations to accelerate code promotion to production. Agile model of devel-opment is being endorsed more than ever, thus leading to an absolute requirement to imple-ment measures to reduce test automation maintenance efforts. A self-healing mechanism to auto fix the scripts by identifying errors and triggers would help the testing team minimize maintenance efforts. This also calls for an AI-enabled solution that can learn the nuances of errors to fix them and reduce manual interventions.

Multiple reasons cause an automation project to fail like non-standard coding, unreasonable expectations, lack of understanding of the func-tionality, ergonomic tools selection etc. However, if we consider primary contributors for an automa-tion script failure will be invalid loca-tors for appli-cation elements, network blips during script execution and failures in the system infrastructure. Amongst the faults mentioned above, the network and infrastructure parameters can be addressed by defined processes as they can be predicted based on Analytics or rule books. However, with regards to the failure related to application object locators, the same cannot be said. It would not be possible to predict the occurrence of the type of object failure that may arise when a test automa-tion script executes.

Taking the above into consideration, UST Global designed a highly scalable and extensible tool-ag-nostic framework. The framework supports the end to end automation spanning across multiple technologies like Web, Mobile, REST API, Desktop, Legacy etc. with the flexibility to script in Java, C# and Python. For more details on Skriptmate, please do visit https://www.skriptmate.io/. Skript-mate has an auto-healing engine that can auto-fix issues when it encounters object-related failures during script execution. After analyzing the differ-ent kinds of object-related faults, categorizing them and devising the possible healing mecha-nism to each of those failures. We drafted a blue-print for developing a plugin for achieving unhin-dered execution. This plugin is what we call today as Skriptmate Self-Healing engine.

When a Software Development Engineer for Test, runs the test scripts developed using Skriptmate framework, the Self-Healing engine would contin-uously monitor the execution. In the event of a failure, the Self-Healing engine is auto-triggered and will first halt the execu-tion sequence and quickly analyze the failure event. Based on its in-built script failure classification concept, it auto-matically decides the appropriate corrective action. Skriptmate Self-Healing engine is capable of handling multiple object-related failures.

For the timeout-related failures, the Skriptmate self-healing engine will auto adjust the script execution rate the application responsiveness. When failures related to an object state oc-curs, the Skriptmate self-healing engine should facili-tate a time-interval based validation of the object state like clickable, visible, etc. For scenarios that result in the element locators not being identified, the Skriptmate self-healing engine using its intelli-gence to track the location of the object in the application captures and validates an alternate property of the failed object. There are multiple properties of an object that are verified by the engine. The Skriptmate self-healing engine then shortlists the most suited locator property using Decision tree learning and Inductive Logic Programming techniques to uniquely identify the object and then proceed with the test script execution. The priority order of shortlisting the best-suited object identification locators is not fixed. Instead, it's auto-updated and auto-refined based on the automated script execution runs.

Copyright © 2020 UST Global Inc 2

Skriptmate Self-Healing is facilitated at script execution time without halting the script execution process. The corrective actions taken by the Skriptmate Self-Healing engine are dynamic. They will be taken in due to course as a response to the failure events that unfold during execution or script healing actions. For ex; in the event of a time-out failure caused due to lags in controls loading in a web page, the Skriptmate self-healing engine would accelerate or slow-down the script execu-tion in accordance to the application response rates. Once this is fixed, the Skriptmate Self-Heal-ing engine would perform the necessary valida-tions like verifying object state and object identifi-cation properties to ensure that the step in the automation script where the failure occurred does not fail again due to some other factor.

The Skriptmate self-healing engine also handles scenarios such as alerts, popups, windows and network-related failures that interjects an automa-tion script. The user can customize the actions to handle such interjections. In the event of a browser crashing during execution, the Skriptmate Self-Healing engine would gracefully exit and hand over the control back to Skriptmate to perform the next sequence of actions.

Though self-healing assists in handling the majori-ty of the object failures, all the steps mentioned above would slightly extend the script execution duration every time, the self-healing engine is called upon to perform an object heal. Hence considering the performance aspects of the script execution, the decision to heal or not heal the object can be configured in Skriptmate framework configurations layer. The approach ensures the freedom to activate/deactivate self-healing in Skriptmate, rests entirely with the users.

The Skriptmate self-healing engine also generates comprehensive reports with analytics on the different types of object failures and the healing measures applied. It also contains de-tailed Impact analytics on application pages that are chief contributors to script failures. These analytics will act as suggestions for 'SDET's in exercising caution during future test script development process. This information would act as a reference guide to facilitate pre-emptive corrective mainte-nance.

The key benefits of provisioning auto-healing of script failures are

• Reduced probability of automation script failure and smoothens execution of scripts during test cycles• Improves readiness to carry out execution any time• Reduced test script maintenance efforts by ~70% as the number of test cases requir-ing changes is reduced• Analytics to update Developers/Testers on appli-cation changes to support future applica-tion/script development

In this day and age, where DevOps is embraced more than ever, there is a definite need to ensure that the testing cycles are concise and effective. In order to achieve this, enterprises target to imple-ment higher percentages of automation. This might inturn result in additional maintenance efforts of the developed automation suite. In order to retain the effectiveness of the automation pack, we believe that the self-healing engine could be of great help by minimizing test maintenance efforts across enterprises that use Skriptmate.

UST Globalust-global.com

T: (949) 716-8757

E: info@ust-global.com