3/18/23, 7:06 PM Advanced Topic - Agile Testing - Scaled Agile Framework

© 2023 Scaled Agile, Inc. All rights reserved.

https://scaledagileframework.com/agile-testing/ 1/5
3/18/23, 7:06 PM Advanced Topic - Agile Testing - Scaled Agile Framework

“ If you don’t care about quality, you can meet

any other requirement.

—Gerald M. Weinberg

Agile Testing

Note: This article is part of Extended SAFe Guidance and represents official SAFe content that
cannot be accessed directly from the Big Picture.

Agile Testing is an approach to systems and software testing that develops and tests systems in small
increments, often developing tests before writing the code, Story, or Feature. Although traditional
development often uses a deferred, big-batch testing approach, tests help elaborate and better define
the intended system behavior, even before the system is implemented. Quality is built-in from the
beginning. Like Agile development, Agile testing is a collaborative, team-oriented process. All team
members share responsibility for testing the system.

This test-first approach applies to all types of Agile work. That includes capabilities, features, stories,
NFRs, and code. It can even be used for the hardware components of a system. In the same way, tests
are written during coding, and acceptance tests for capabilities, features, and stories are written during
elaboration. The just-in-time practice of elaborating the proposed system behavior also mitigates the
need for overly detailed requirement specifications and sign-offs. Even better, unlike conventionally-
written requirements, these tests are automated wherever possible. But even when they are not, they
still provide a definitive statement of what the system does rather than a statement of early thoughts
about what it was supposed to do.

Details
Agile testing is a continuous process integral to Lean and Built-In Quality. Agile Teams and Agile
Release Trains (ARTs) cannot go fast without high quality. Without constant, primarily automated
testing, it’s challenging to implement a fast, Continuous Delivery Pipeline and achieve Release on
Demand. Wherever possible, that means testing—and even test automation—comes first.

In Agile testing, everyone on the team is a tester. Using Behavior-Driven Development (BDD), Product
Managers and Product Owners collaborate with their teams to create tests for features and stories.
Developers create tests for code changes using Test-Driven Development (TDD). With these general
tests created by their team members, testers can focus on exercising edge cases and the less intuitive
interactions with the system.

Agile testing strives to be test-first. This approach has many benefits:

https://scaledagileframework.com/agile-testing/ 2/5
3/18/23, 7:06 PM Advanced Topic - Agile Testing - Scaled Agile Framework

Multiple perspectives provide a broad view of the required system behavior and the best
approach to testing it
English
Collaboration creates alignment across the team and a shared understanding of how to
implement the behavior
It forces developers to think broadly about a change before implementing it

Other tests—such as performance, security, and user acceptance—are typically less collaborative and
more challenging to write first. Nonetheless, developers should still perform these tests early and often
to minimize surprises later in development.

The Agile Testing Matrix

At scale, many tests are necessary to ensure quality: tests for code, interfaces, security, stories, larger
workflows, and so on. Brian Marick (Extreme Programming (XP) advocate and Agile Manifesto [1] co-
author) helped pioneer Agile testing by describing an Agile testing matrix that guides the reasoning
behind various tests and provides a taxonomy for organizing tests. This approach was further developed
in Agile Testing [2] and extended to scaling the Agile paradigm in Agile Software Requirements [2, 3].

Figure 1 describes the latest matrix [4] with guidance on what to test and when.

Automated Manual and

and Manual Business Facing Automated

• A/B Tests • Exploratory Testing

• Story Tests • Workflows
• User Experience (UX) • System Integration
• Prototypes • Usability Testing
Critique the Product
Guide Development

• Simulations • User Acceptance Testing

Q2 Q3

Q1 Q4

• Unit Tests • Performance Testing

• Component Tests • Load Testing
• Testing Connectivity • Security Testing
• Quality Attributes (ilities)

Technology Facing Automated

Automated
and Tools

© Scaled Agile, Inc.

Figure 1. Agile testing matrix

The horizontal axis of the matrix contains business (top) or technology (bottom) facing tests Written in
https://scaledagileframework.com/agile-testing/ 3/5
3/18/23, 7:06 PM Advanced Topic - Agile Testing - Scaled Agile Framework
The horizontal axis of the matrix contains business (top) or technology (bottom) facing tests. Written in
business terminology, business-facing tests are designed to be understandable by the user. Written in
the developer’s language, technology-facing tests are used to evaluate whether the system delivers the
behaviors the developer intended.

The vertical axis on the left side guides development by having the development team think about how
they will test a story or section of code before they write it. To find defects or missing features, the right
side contains tests critiquing the solution by evaluating the system against the user requirements.

Creating tests across the four quadrants (Q1 – Q4) fosters a comprehensive strategy that helps ensure
quality. (Note: The quadrant number is arbitrary and implies no priority to creating tests.)

Q1 (Unit and component tests) –tests are written to run before and after code changes to
confirm that the system works as intended.

Q2 (Functional tests) – tests for stories, features, and capabilities validate functionality works
the way the Product Owner (or user) intended.

Q3 (System-level acceptance tests) – validate that the whole system’s behavior meets usability
and functionality requirements. These may include exploratory tests, user acceptance tests,
scenario-based tests, and final usability tests. Because they involve users and testers engaged in
actual or simulated deployment scenarios, these tests are often manual. They are frequently the
final system validation before delivery of the system to the end-user.

Q4 System qualities test – verify the system meets its Nonfunctional Requirements (NFRs).
Typically, they’re supported by a suite of automated testing tools (such as load and performance)
designed for this purpose. Since any system changes can violate NFR, they must be run
continuously, or at least whenever it’s practical.

SAFe’s TDD and BDD articles detail practices for quadrants one and two, respectively. Continuous
Integration provides more information on quadrant three. NFRs, Continuous Integration, and Release
on Demand offer more details on quadrant four.

Learn More

[1] Manifesto for Agile Software Development. http://AgileManifesto.org/

[2] Crispin, Lisa, and Janet Gregory. Agile Testing: A Practical Guide for Testers and Agile Teams. Addison-
Wesley, 2009.

[3] Leffingwell, Dean. Agile Software Requirements: Lean Requirements Practices for Teams, Programs,
and the Enterprise. Addison-Wesley, 2011.

[4] Gregory, Janet, and Lisa Crispin. More Agile Testing: Learning Journeys for the Whole Team. Addison-
Wesley, 2015.
https://scaledagileframework.com/agile-testing/ 4/5
3/18/23, 7:06 PM Advanced Topic - Agile Testing - Scaled Agile Framework

Last update: 27 February 2023

he information on this page is © 2010-2023 Scaled Agile, Inc. and is protected by US and International copyrigh
ws. Neither images nor text can be copied from this site without the express written permission of the copyrig
older. Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc. Please visit Permission
FAQs and contact us for permissions.

https://scaledagileframework.com/agile-testing/ 5/5