Building the right Performance Testing strategy

In our digitally-connected world where companies are churning out umpteen number of products or services to be accessed over digital devices, performance becomes a sticky issue. The word ‘sticky’ has been used deliberately as the performance of a product or service depends on a host of factors. And delivering a glitch-proof product is no guarantee of its smooth performance. In fact, factors like responsiveness, network bandwidth, and others can deal a body blow to the performance of an application. Just think of an airlines portal catering to thousands of transactions every second. If the portal does not follow a robust performance testing strategy and selects a threshold for load, it is likely to crash midway or show latency. This can adversely impact the user experience.

What is performance testing?

It is a form of software testing where the application under test needs to be checked for its load handling capacity. The load performance testing process determines the stability, speed, throughput, and responsiveness of an application or system when working under a load. It is a non-functional testing that comprises various tests such as load test, capacity test, scalability test, soak/endurance test, volume test, and spike test.

To build the right performance testing strategy, one must be aware of the associated challenges. These include

·         Testing the software under realistic scenarios

·         Picking the right tool to simulate real users

·         Lack of test data

·         Limited test coverage area

·         Poor scalability not allowing the handling of concurrent tasks

·         Poor network configuration

·         Insufficient hardware resources like low performing CPUs

·         Insufficient capacity to handle the workload

The performance testing methodology should define the scope, load policy, and SLAs surrounding the application. Since everything about the software cannot be tested, a decision should be taken to identify the specific area, feature, or functionality of the application for testing. It has been observed that around 10-15% of test scenarios contain around 75-90% of bugs. So, it is better to focus on those scenarios and mitigate the lion’s share of glitches.

How to build the right Application Performance Testing strategy?

Selecting the right strategy would determine if the performance of the application covers all the right areas and load scenarios.

Test environment: Set up a robust test environment by identifying elements like hardware, network configurations, and test tools. This will help the QA team to identify challenges pertaining to performance testing early on in the SDLC. In fact, QA experts can have a number of options in setting up the test environment. These include:

·         A production system containing a smaller number of servers of lower specification

·         A production system comprising a smaller number of servers of similar specification

·         Using the actual production system or its replica

Baseline for user experience: At the beginning of conducting load performance testing, the QA testers should know the level of user satisfaction. So, it is not merely about the number of load times or responsiveness of the application. Identify a baseline for user experience that the application should be tested for.

Performance metrics: Choosing the right performance metrics is important to plan for testing. These include throughput, response time, and constraints. For example, response metrics deal with measuring the performance of an application from a user’s perspective. While volume metrics is about measuring traffic generated by the load testing tool vis-a-vis the application.

Realistic tests: Allowing umpteen number of virtual users to face a server or cluster of servers may not be the right way of measuring load in a real-world scenario. While setting up such a scenario, the focus should be on choosing the devices or environments that would be used to access the application in real-time. This is important because the traffic is likely to come from a host of mobile devices, operating systems, and browsers. Also, the realistic test scenario should vary the test environment. This is to allow the organization to become resilient to the changing market dynamics.

Performance testing a part of Agile: Performance testing should not be conducted at the end of the development process. This makes it too late and expensive to fix glitches. So, load performance testing should be integrated with the development process a la Agile. In fact, performance engineering should be a part of the scrum and entrusted with measuring the performance of code during development.

Conclusion

In a competitive business environment where the quality of software application determines the latter’s success, building the right strategy for performance testing is critical. It ensures the smooth performance of application against a definitive user load scenario.