How implementing a

modern testing practice

accelerates software
delivery
How implementing a modern
testing practice accelerates
software delivery

EXECUTIVE SUMMARY

Digital transformation is near the top of every CIO’s agenda

today. To reap the benefits, it’s critical to get new products
to market faster. Agile and DevOps methods have ushered
in a new era of rapid development and delivery. Yet testing
remains a bottleneck.

To achieve the desired speed, it’s critical to align business

goals with both testing methods and quality ambitions.
Those who do stand to gain a significant competitive
advantage by modernizing a critical element of accelerated
software delivery that is still often overlooked. This guide
explores how CIOs can eliminate traditional testing
roadblocks to achieve true digital transformation.
 INTRODUCTION

A recent survey of 500 IT executives finds 58% of enterprises deploy a new software build daily, and
26% at least hourly. That’s why Agile and DevOps are so important. These are the methods that enable
companies to develop and release software on a daily, and sometimes, hourly, basis.

Software is now at the forefront of most companies’ strategies. This game-changing shift, from creating
physical goods and experiences to smart software development, is a key element of digital transformation
and is now the primary driver of sustainable innovation — even in traditional industries that have made
their fortunes outside of technology.

Despite this, software (and what’s required to efficiently create and deliver it) gets surprisingly little
attention in the C-suite. According to a recent McKinsey article, companies pay a steep price for dismissing
software’s importance. This dismissal exposes them to several vulnerabilities:

• Tech-based disruption: According to Salesforce’s State of the Connected Consumer report, 73%
of customers say that a single extraordinary experience with one company raises their expectations
of other companies. This highlights a tricky reality for businesses: today, you are in competition
with every other company, regardless of industry.

• Customer churn: Three-quarters of customers expect companies to use new technologies to create
better experiences. We are now entering an era where a company’s ability to provide innovative
experiences is as important as the quality of its products. Salesforce found that 84% of customers
now consider these factors equally when deciding who to buy from.

• Higher costs and lower margins: Beyond customer experience, software is pivotal in helping
optimize operations and rein in costs.

These risks should put the constant push for faster software delivery in perspective. But what many
executives fail to realize is the power of the right testing approach for increasing delivery speed without
compromising quality.

To achieve the desired speed, it’s critical to align business goals with both testing methods
and quality ambitions. Those who do stand to gain a significant competitive advantage by
modernizing a critical element of accelerated software delivery that is still often overlooked.

This guide sheds light on testing’s critical role in today’s CIO agenda and shares best practices for
modernizing testing to enable high quality at high speed. Executives don’t need to write test scripts (and
their developers and testers may insist they don’t!), but they must understand modern testing practices
to determine the right approach for the company.

tricentis.com 3
 EVERY CIO HAS A DIGITAL TRANSFORMATION INITIATIVE UNDERWAY

According to a survey of CIOs conducted by Morgan Stanley Research in October 2020, today’s top
CIO priorities are:

Cloud Data Warehousing/ AI/Machine Digital Security

computing BI/Analytics Learning/Process Transformation Software
Automation

You might be surprised to learn that testing has significant impact across 4 out of 5 of these
initiatives. Here’s how the right testing strategy can help:

Cloud computing: Enterprises must have strategies for both testing cloud-native applications, as well as
for minimizing risk during migration of on-premises software to the cloud.

Data warehousing/BI/analytics: There isn’t time to manually test for data integrity as it passes through
the data warehouse to other systems. But making business decisions without confidence in the integrity
of the data behind them is a recipe for disaster. Automated BI and data warehouse testing help solve the
data integrity problem in real time.

AI/machine learning/process automation: CIOs are investing in AI, machine learning, and process
automation with the goal of making smarter business decisions. In the testing domain, AI is being
leveraged to pinpoint the highest risk areas of an application and the best test cases for addressing them
— drastically reducing the volume of testing that needs to be completed before each release

Digital transformation: Digital transformation is a must, both across the critical business systems your
organization relies on, as well as in the arena of developing software that creates new revenue streams
and improves the customer experience. An enterprise-wide testing modernization can unify quality
objectives and accelerate testing across both of these areas.

Once the limited domain of testing teams, software quality is now gaining ground as an enterprise-wide
priority, driving a transformation that federates the ownership of quality — and places it squarely on the
map of CIO priorities. Because of the collaborative nature of Agile and DevOps methods, it is also now
deeply embedded in the processes of IT leaders, product owners, and development and operations
teams.

SPEED & AGILITY: THE KEYS TO DIGITAL TRANSFORMATION

Agile introduced a radically new concept of software development: iterative builds which accelerate
feedback loops, allow faster response to market need and catalyze digital transformation. But Agile can

tricentis.com 4
 only succeed if downstream processes can accommodate the change. Without alignment, testing and
operations teams will not be prepared for the influx of smaller and more frequent releases — which can
cause delays and impede the end goal of Agile efforts.

DevOps has emerged to help organizations realize the goals outlined in Agile by connecting development
with testing and operations, mainly through the introduction of automation. And automation differentiates
the most advanced DevOps teams. According to DORA’s 2019 Accelerate State of DevOps report, the most
advanced teams “automate everything” by committing to continuous testing, integration and delivery (CI/
CD) practices. These elite DevOps teams deploy code 208 times faster, and release software 106 times
faster than the lowest performers surveyed. And DevOps practices are catching on quickly. Almost half
of all respondents in GitLab’s 2019 survey report using continuous code deployment at least somewhere
in their organizations.

THE TRADITIONAL TESTING APPROACH: THE #1 SPEED INHIBITOR

According to GitLab, development teams report that testing represents the most significant delay in the
software delivery pipeline — even on DevOps teams.

For the average organization, 80% or more of the testing effort is performed manually. And
this effort is not without cost: Testing still represents 23–28% of IT budgets, according to the
2019-20 World Quality Report.

Recent research from Tricentis and the World Quality Report shed light on this finding. Both reports show
that most testing still occurs manually. A majority of organizations automate 20% or less of their total
testing effort.

Taken collectively, this data makes it clear that testing stands out as the most significant barrier to
achieving innovation. Therefore, it also represents the biggest opportunity for improvement in delivery
timelines. So why have more organizations not leveraged testing’s untapped potential?

To put it simply, digital transformation requires a scalable approach to test automation. Below, we’ve
outlined the key steps required for accelerating release speed, reducing testing costs, and improving
software quality.

tricentis.com 5
 SCALING TEST AUTOMATION TO INCREASE DELIVERY SPEED AND QUALITY

1. Implement a tool to assess risk

A risk-based approach can expose business risks associated with each release, so testing is targeted
on the highest-risk areas. This significantly cuts down on the amount of testing that is required and the
overall time spent.

Tricentis offers two key solutions for addressing risk:

Tricentis Tosca, powered by Vision AI offers a risk-based testing approach that reduces testing efforts
by exposing risks to the user experience and/or critical business processes. A risk-based approach
catalyzes the benefits of automation. With risk-based testing, testing teams can identify and create the
tests that are most important for minimizing business risk — significantly reducing the number of test
cases that need to be created and increasing confidence in release readiness. The resulting test suite is
perfectly suited for continuous execution as part of the delivery pipeline, and it’s easily updated as the
application evolves. And all of this is accomplished without coding or scripting.

Tricentis LiveCompare provides AI-powered impact analysis for updates to SAP and other enterprise
applications, exposing the risks to critical business processes posed by any standard or custom change.
LiveCompare pinpoints the most at-risk areas that need to be tested, which reduces testing timelines
by 85%, on average, while ensuring 100% risk coverage. Any testing gaps discovered are automatically
added as requirements in Tricentis Tosca so that automated test cases can be created to fill the gaps.

2. Streamline test management, design, and build processes

It’s critical to implement an agile test management solution that centralizes testing strategy, whether
you’re using open source tools, proprietary test automation tools like Tricentis Tosca or a combination of
the two. Test management should integrate with each test automation too, as well as with both upstream
with agile planning tools like Jira Software and downstream with CI/CD pipeline tools like Jenkins. When
it’s integrated across the delivery pipeline, testing can be executed efficiently, feedback can be delivered
in real-time, and all stakeholders gain full-scale traceability, from development to delivery.

Tricentis qTest centralizes quality strategies across the organization with modern, enterprise-grade
agile test management. Through agile test management, enterprises can manage, design, and build test
suites for full-scale traceability across the software delivery pipeline, from agile planning tools like Jira
to test automation tools and CI/CD tools. Tricentis qTest integrates with any test automation tool or
framework to enable central management, kickoff, and reporting across test automation initiatives, and
to offer complete visibility across teams.

tricentis.com 6
 3. Automate at scale

Many organizations struggle to scale test automation, either because they lack the required coding skills,
or because a proliferation of tools makes it difficult to track ROI and assess overall progress towards
higher test automation rates.

Tricentis Tosca, powered by Vision AI provides scalable, AI-driven test automation technology that
allows teams to automate the application user interface, independent of the underlying technology.
With machine learning, Vision AI will see and steer any interface just like a human user. This makes test
automation scalable, future-proof, and as adaptable as the human brain. For the first time, teams can
truly shift UI testing to the left. With Vision AI, teams can create test automation based on a simple mockup
or UI description before any code is written — enabling teams to test much earlier in the development
lifecycle.

With Tricentis Tosca, enterprises can quickly scale functional test automation across hundreds
technologies, from mainframe to mobile and everything in between. Tricentis Tosca facilitates testing
across IT and development teams, with support for enterprise applications (SAP, Salesforce, ServiceNow,
Oracle, Workday, etc.), data warehouses, and custom-developed software, including web and mobile
applications.

4. Implement modern performance testing for modern application development

Enterprise organizations need continuous performance testing to cover everything from monolithic and
packaged apps to microservice-based apps, APIs, and more. A performance engineering solution that
spans these critical technologies across various skillsets is crucial for DevOps success.

Tricentis NeoLoad provides the predictability, validation, and assurance at the speed and scale of today’s
faster release cycles and is designed for the complexities of modern enterprises. It is purpose-built to
be flexible and fast, working with complementary testing solutions and toolchains to enable continuous
performance testing. NeoLoad simplifies and scales performance testing for everything from APIs and
microservices to end-to-end application testing.

5. Ensure cross-project visibility

Testing reporting should offer executives a rapid, real-time assessment of release readiness. It should
also be able to quickly demonstrate overall progress towards test automation goals, demonstrate ROI,
and pinpoint areas for improvement — at the project and team levels, as well as for the organization as
a whole.

tricentis.com 7
 Tricentis Analytics provides portfolio-level visibility through a predictive, BI reporting engine to
consolidate, manage, and analyze activities across the Tricentis Continuous Testing Platform, as well as
any open source tools in use. With Tricentis Analytics, executives instantly evaluate current and future
business risks associated with the health of their applications, enabling them to release faster and with
more confidence.

SCALING TEST AUTOMATION TO INCREASE DELIVERY SPEED AND QUALITY

Based on many years of R&D, Tricentis has developed the industry’s #1 test automation platform, which
effectively addresses today’s most common test automation challenges and enables enterprises to rapidly
increase automation rates, reduce testing costs and significantly accelerate software delivery timelines.

EdgeVerve reduced Varian reduced Aegon now executes

test cycle times execution time by automated testing to
by 60%, with 93% and achieved verify data integrity
continuous testing, a 35% cost savings, across hundreds of
and now delivers after replacing millions of records
30% more features manual testing with in just 11 hours—an
per release, with a test automation, effort that would
50% reduction in take 84.5 hours to
production defects. complete manually.

DISCLAIMER: Note, the information provided in this statement should not be considered as legal advice. Readers are cautioned not to place undue
reliance on these statements, and they should not be relied upon in making purchasing decisions or for achieving compliance to legal regulations.

tricentis.com 8
 ABOUT TRICENTIS

Tricentis is the global leader in enterprise continuous testing, widely credited for reinventing
software testing and delivery for DevOps and agile environments. The Tricentis AI-based,
continuous testing platform provides automated testing and real-time business risk insight across your
DevOps pipeline. This enables enterprises to accelerate their digital transformation by dramatically
increasing software release speed, reducing costs, and improving software quality. Tricentis has been
widely recognized as the leader by all major industry analysts, including being named the leader in
Gartner’s Magic Quadrant five years in a row. Tricentis has more than 1,800 customers, including the
largest brands in the world, such as Accenture, Coca-Cola, Nationwide Insurance, Allianz, Telstra, Dolby,
RBS, and Zappos.

To learn more, visit www.tricentis.com or follow us on LinkedIn, Twitter, and Facebook.

AMERICAS EMEA APAC

2570 W El Camino Real, Leonard-Bernstein-Straße 10 2-12 Foveaux Street
Suite 540 1220 Vienna Surry Hills NSW 2010,
Mountain View, CA 94040 Austria Australia
United States of America office@tricentis.com frontdesk.apac@tricentis.com
office@tricentis.com +43 1 263 24 09 – 0 +61 2 8458 0766
+1-650-383-8329

tricentis.com 9
v. 0821