Software Testing for Devops and Microservices Era - Common Misconceptions and Industry Realities

Sachin Mulik, VP, Quality Engineering, Amdocs

Sachin Mulik, VP, Quality Engineering, Amdocs

For a long time, Business organizations were limited by speed of software development by their IT organizations. Even though IT was seen as an enabler, businesses felt that they couldn’t get their products and services rolled out fast enough to the customers. As IT embraced newer methodologies and IT architectures such as DevOps and microservices, businesses are feeling more confident about getting important features and services into hands of customers at a faster pace.

Because of these changes, the world of software testing is undergoing a massive transformation. Every single aspect – people, tools and technologies, methodologies, delivery models—is rapidly evolving. There is overwhelming amount of information floating around all of us. It can be very difficult to keep a track of all relevant, useful information and to separate fact-from-fiction, theory-from-real world experience. Some of the most common misconceptions are around role of a software tester and role of software testing in general. While it’s true that eventually for organizations that have fully transformed into DevOps/microservices world, role of software testing and software tester will be completely absorbed within other roles; most of the organizations will take several years to get there. And many organizations will decide to operate on a hybrid model–partly operating on newer methodologies, and partly continuing to maintain their legacy systems. For these organizations, it’s important to understand common misconceptions and industry realities for software testing for DevOps and microservices era.

This transformation journey is not to be taken lightly. It needs a lot of thinking, planning, execution rigor –and above all, a well thought through change management process

Misconception #1: Testing is not required in the Microservices world

Industry Reality:
• Testing is done at a microservices level and moves inside the ‘pods’ (scrum or agile teams working on a specific story)
• Types and methods of testing change e.g. instead of usual systems integration test, contract testing (certifying that a specific microservice is fulfilling its functional, performance and security ‘contracts’ under given conditions) becomes more important

Misconception #2: The Software Tester role is no longer needed

Industry Reality:
• Traditional ‘black box’ tester role transforms into ‘Software Development Engineer in Test’ (SDET) or ‘Development Quality Assurance Engineer’ (DevQE)
• SDETs / DevQEs are integral part of pods, actively participate in every development cycle from day-1
• While SDETs / DevQEs have deep expertise on development technologies being used within a pod, their primary focus is to look for errors in products being developed and quality assurance process compliance within that pod
• SDETs / DevQEs will also automate as many test scripts as they can, which enables CICD pipeline and rapid deployments
Misconception #3: Development and Testing is done by a single vendor (in organizations that rely on vendors / partners)

Industry Reality:
• Microservices /DevOps based pods work in ‘One Team’ model
• Vendor boundaries disappear and everyone is playing their specific role in a pod
• While it’s true that if all roles within a pod are being delivered by a single vendor then they can benefit from colocation; this scenario is very rare in global delivery model anyways – more and more teams are embracing multi-location multi-vendor pod structures as long as time overlaps and communication-collaboration tools are put in place
Misconception #4: A Microservice is completely standalone and once it’s tested, one doesn’t have to worry about integration or end-to-end tests

Industry Reality:
• Well, it’s not completely a misconception. For certain organization who are ‘Digital first and digital only’ (Think Google, Facebook, Wix), this is indeed true. In these organizations, the IT architecture has been built from ground up to be a ‘services-oriented architecture’ and each of these services can be dealt with in isolation.
• But majority organizations have not been fully transformed yet (or can never fully transform). For these organizations, microservices constantly have to interact with other interfaces within their IT application landscape that can be a mishmash of legacy systems, middleware, modern 3rd party applications, and ERP systems.
• Hence, for these organizations, it becomes important to test at least a thin layer of cross-microservices business flows or interactions between microservices and other systems.

As IT organizations go through this transformation, it’s also important to address some fundamental question, such as:

Question 1: How will software testing roles within my IT organization change?
• Traditionally, IT organizations have had ‘Functional Tester’ and ‘Test Automation Engineer’ as two primary roles. These roles will evolve into following 3 roles:
• SDETs/DevQAs: Functional testers will evolve into SDETs/replaced by SDETs with ability to conduct functional testing and automation within pods. They will also support performance and security test activities within a pod.
• Test Architect: Responsible for early engagement during the technical requirements and product backlog creation phase. Responsible for ATDD approach and making sure that the stories are developed in a proper testable sequence.
• Technical Test Architect: Envision, create and enhance the continuous automation platform, performance-security test platforms

Question 2: How will the new IT organization structure look like from a Software testing perspective?
• The pod structure will become a self-sufficient mini-org that can fulfill its own needs. The size of these pods will tend to follow the famous ‘2 pizza team’ rule–mostly between 6 to 8 people. Majority of these people will be dedicated to a pod. Some roles may cut across multiple pods such as ‘Test Architect’ or some specialty SME skills.
• There will continue to be a thin layer dedicated to end-to-end tests and maintaining continuous regression and test automation platform.
• The overall structure may look something like this:

Question 3: Will there be a dramatic shift in testing tools?

Yes, absolutely yes. Major impact will be on test tools used for functional testing, with Performance and Security testing tools undergoing some minor adjustment (or updates to existing tools). The changes can be summarized as:

• API / Services testing tools (such as SOASTA, SOAPUI) will gain prominence
• Open source Java, Selenium based frameworks will become mainstream
• Development tools / languages will be reused for testing e.g. Python
• There might still be a need to create a ‘plug-n-play’ kind of wrapper framework that can be used across the IT organization (and across multiple pods and microservices) that can drive synergies and standardization of continuous automation platform

This transformation journey is not to be taken lightly. It needs a lot of thinking, planning, execution rigor –and above all, a well thought through change management process.