Hello and welcome to our next post in this DevOps series. Today we explore the test stage and show you how our smart tools help organizations understand what to test and why.
We will cover three specific areas associated with testing in this series:
• Smart impact analysis - 100% coverage for just 15% effort
• Test System Validation – validating the QA system is set-up correctly.
• Test Audit – Understanding what has been tested and by who.
Watch our webinar recording on the six key stages of smart DevOps here.
Smart Impact Analysis
Today we look at LiveCompare’s smart impact analysis app, Impact, and how this reduces SAP application testing by at least 85%.
You may be asking “how does LiveCompare reduce testing?"
Let’s take a look…
To reduce testing by this much, LiveCompare looks at three key factors, in order of importance:
• How critical these objects are to the business.
• Their depth. And by depth, LiveCompare will analyse all the dependencies between changing objects and used objects to give you an optimal set of objects to test to ensure all changes are covered.
• How often objects are used by the business.
LiveCompare crunches through your SAP data and provides reports which will tell you exactly what to test and why.
In addition to running the Impact app on-demand, further efficiencies can be achieved by automating more of the impact analysis process. Several customers are working towards a continuous delivery model where every night:
- DEV changes are promoted to QAS
- LiveCompare Impact is scheduled to run and picks up all the new QAS changes
- Identifies most-at-risk objects to test
- Integrates with ALM repository to determine:
- What’s testable
- What’s untestable
- Email report to test engineers before the start of their day
Integration between ALM products and test automation tools (e.g. Micro Focus ALM, Visual Studio Team Services and Worksoft Certify) mean that all available lights-out test assets can be run automatically during the night.
When the test team shows up in the morning they have answers to:
• What should be tested and why?
• What’s testable and untestable?
• What passed and failed?
Risk-Based Routes to Production
A more advanced implementation of this involves the adoption of risk-based routes to production.
With this approach, low risk changes may skip dedicated test cycles and go direct to user acceptance testing (UAT).
An observation of this approach showed that there is tremendous pressure on the build and test teams to accept all changes as low risk. Unfortunately this leads to increasing PRD defect counts.
A better way is to assess the change and route it appropriately.
Impact provides multiple data points that can be used to objectively assess the risk of every change. The factors include:
• # of Users
• Frequency of Change
Maintain the Health of Automated Tests
With traditional approaches, testing is given a little bit of time at the end of the release to do its job.
Pressure to test more with less drives the adoption of automated test systems. However, there is a problem with “lights-out” tests that we call the half-life problem. It looks like this:
Put simply: with each release only half of the automated tests can be used.
Why is this?
Automated tests are sensitive to UI changes. Adding a mandatory field to a screen will break an automated test.
There’s no time to fix-up the test.
IntelliCorp’s smart DevOps fixes this problem by giving test engineers early access to what will impact their automated test assets. LiveCompare’s Component Impact Analysis app:
• Compare screens to find significant changes
• Find automated tests that depend on the significantly changed screens
• Mark automated tests as impacted
With smart DevOps we can start testing earlier, do more testing (to eliminate more production defects), maintain the health of our automated tests (so they truly become assets) and still deliver working software faster.
• Know the specific screen and object impacted by a change
• Enable accurate early testing, eliminate more production defects
• Eliminate half-life problem of lights-out tests
• Deliver working software faster
If you want to find out more, watch our webinar recording.
If you want more information now, contact us here to schedule a call or demo.