Digital Transformation is disrupting the way software applications are being developed, amplifying the need for building better software testing capabilities. In fact, the complexities with applications and software are increasing and there is a growing need for third-party integration. The main effect of this transformation is that the patterns, methodologies, and tools for testing are being evolving with the changing software testing needs. For instance, Agile testing and DevOps can be identified as the key drivers for growth in the software testing market, considering the pressure due to the release new software version at speed and with sustain quality at the same time.
In this new challenging context, Workload Automation Quality Assurance Team is evolving itself, enabling new processes for which software testing starts way ahead in the development lifecycle instead of being introduced at the end of the development process, as shown by the following picture:
Our current quality assurance testing strategy
Our quality assurance strategy is centered on continuously evolving our testing process to prevent bugs from being created as the software is developed or, worse yet, from being discovered by our customers.
The adoption of the Agile methodology rapidly provides customers what they need to compete while maintaining high-quality deliverables. Using the Agile methodology, we can reach the following goals:
- Faster feedback
- Radical simplification of the code: developers write code in response to
the requirements of the tests
- Identification of the error/problem quickly
- Bugs reduction
- Quality improvement
- Reduction of the development Time-to-Market
We do extensive use of automated test suites to validate that our products function properly allowing the products stabilization of the new implemented features and a continuous regression during all the development phase. Many thousands of automated test cases are daily run to test all the product interfaces and the main product functionalities. Jenkins is used as orchestrator to achieve a continuous testing, according to the following picture:
Moreover, we are used to exercise our products in real-world scenarios as they're being actively developed, which helps us to identify problems before our customers do. To enable this approach, we aggressively use customer feedback to evolve our testing plans as we resolve issues identified in our support forums, custom services, and formal support chains.
More specifically, the key differentiators of our Testing strategy are:
Transplant Testing paradigm for System Verification Test (a.k.a. SVT), a methodology to bring customer applications, data, scenarios and procedures into our test labs. This paradigm sees its finalization in the implementation of the Corporation methodology to emulate business-driven scenarios in production-like environments.
This methodology is built on the following processes/best practices:
- Learning from customers how to change our test approach
- Providing our consulting for the deployment of our solutions
Continuous Performance Testing based on the following elements:
- A single node is able to scale up to 175 concurrent users
- Clustered solution with 4 nodes is able to scale up to 700 concurrent
users preserving acceptable average page response time for all the transactions
environment (for instance, more than 60 * 10^6 rows in the product historical
field into quality goals to be assessed during each new functional fix
pack and/or main release development cycle
- Publishing periodically Performance Report to provide guidelines on how
to configure the product to optimize the performance and the scalability
Security Testing based on the following implementations
New challenges for the immediate future
The main mid-long term goal is to adopt a continuous testing approach to enable a full DevOps development process.
The main benefits of this new shift-left paradigm will be:
In this way, the team will improve software quality, gather feedback faster than ever, and manage properly the Workload Automation solutions proliferation due to Hybrid Cloud puzzle.
From a testing perspective, as long as testers have access to those systems, the automation tools needed for continuous testing (test automation, service virtualization, deployment automation, and so on) need to do their robotic magic as well. Understanding this requirement at the outset of a project will help prevent barriers to achieving continuous testing (QA+).