Rethinking Software Testing in a DevOps World
Faster time to market. Some say it’s not the next big thing; it’s the only thing. Still, even with everyone pushing for DevOps, Agile and Continuous Integration, there is one key component that consistently gets lost in the shuffle — software testing.
Analysts call for a new approach to testing in Agile environments, but all we see are point solutions — unit testing, functional testing, browser testing, performance performance testing, load testing, and security testing — each a separate and distinct product, with little or no integration. In addition, many of these solutions require a great deal of script development, and that means script maintenance and debugging.
Incorporating five or six different point solutions doesn’t sound very agile!
That being said, the last three years have brought us the first attempts to address aspects of DevOps testing in a new way. An example of this would be companies taking open source software (like Selenium) and putting a slicker façade on it to undertake Web-based functional testing.
Although this can certainly help with functional testing, where does it leave performance, security, database, client-server and all other types of software testing? How are you supposed to execute on DevOps, global development, and delivery models, along with Agile, when a dozen different solutions have to be integrated?
Rethinking Software Testing in a DevOps and Agile World
Testing has seen limited innovation in the last 20 years. Big technology brands, such as Amazon and Google, have the budget and time to build custom, broad-based, rapid automated testing systems tied directly to their product. Many enterprises making big investments in DevOps and moving to Agile development, however, don’t have the budget or expertise to do the same. In other words, without rethinking testing, the entire DevOps productivity gain is in jeopardy.
There is a significant need for a next-generation Agile software testing platform — one that brings different types of tests together as a single “write-once” solution rivaling what organizations such as Google have built from scratch. Delivering such an integrated, automated test suite, one that works with the entire lifecycle (including Puppet, Jenkins, JIRA, etc.), represents the future of how organizations think about testing.
Old-school testing companies are coining new phrases such as “Integrated Functional Testing” rather than building unified testing solutions that meet the rapidly evolving needs of businesses. For this reason, enterprises and analysts alike need to start looking at creating a new category for evaluating testing solutions — Unified Testing or Agile/DevOps Testing.
While every whitepaper or report written on software testing in the last two years emphasizes the challenges of testing in a DevOps and Agile world, no one actually addresses the issue or offers a true solution. It’s the elephant in the room. And the fact of the matter is that tools they are evaluating don’t solve the problem, nor do they improve QA productivity and reduce workload for coders.
To realize the productivity gains promised by DevOps, you need to kick off hundreds of tests — from functional to performance to security — at each build and get results in minutes. This may sound like a fantasy to traditional QA teams, but it’s not for companies who architect unified solutions from scratch, reducing QA costs by 10x (or more) while increasing the quality of their code and their confidence in Continuous Integration deployments.
Removing the Silos
Solving the software testing conundrum is essential for anyone hoping to get full value from a true DevOps methodology. This means getting rid of siloed QA functions and understanding how to employ automated software testing in a DevOps world.
It’s important to start unified testing. Is it a new category? I’m not sure. I am sure, though, that without addressing automated testing across multiple categories, no one will see the productivity gains, increased efficiencies, and reduced costs promised by the move to DevOps.