Customer

SAP Solution Manager – Test Suite

Test Planning
Digital Business Services, SAP SE
Disclaimer

This presentation outlines our general product direction and should not be relied on in making a
purchase decision. This presentation is not subject to your license agreement or any other agreement
with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and SAP’s
strategy and possible future developments are subject to change and may be changed by SAP at any
time for any reason without notice. This document is provided without a warranty of any kind, either
express or implied, included but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally of grossly negligent.

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 2
Test Plan Management
Entry point: Launchpad  tile „Test Plan Management“

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 3
Test Planning
Systematic approach to distribute Test Scope to Tester

Solution Documentation Test Plans Test Packages

 Repository / Library of Test Cases  Selection of Test Cases for specific Release,  Selection of Test Cases to be assigned to
 Structured by Processes, Steps or Executables Projects, Test Phase and Test Level one or more testers
 Reusable across Releases, Projects and Test  Could be used to reflect further aspects like  Typically used to reflect further
Cycles Business Area, Country, etc. organizational aspects like department,
 Relevant entity for Reporting user-role, etc.

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 4
Typical Test Process in SAP Solution Manager
Test Process supported by Test Plan Release Status Schema
Example: DEFAULTS

Sign-Off Sign-Off
Test Scope Test Execution

Business Process Expert Test Manager Tester Test Manager

Test Preparation Test Planning Test Execution Test Reporting

• Create/ Update • Create/Copy/Update • Get E-Mail • Access Test Status

Business Blueprint Test Plans Notification Overview
• Create/Update Test • Create/Update Test E-Mail • Access Tester • Perform Test Status E-Mail
Cases Packages notification Worklist Analysis notification
• Re-Use Templates • Assign Tester • Perform Test • Run Test Progress
• Manage Defects Report
• Document Result

Re-Open urgent

Release Status New Released Protected

 Test plan is open for changes  Test plan is closed for changes  Test plan is closed for changes
 Test plan is closed for  Test plan is open for execution  Test plan is closed for execution
Impact
execution  BI-reporting data updated

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 5
Test Planning for integrations test
Systematic approach via Test Sequence to support integration tests

Test Plans Test Sequence Test Packages

 Selection of Test Cases

for specific Release, Projects, Test Phase
and Test Level  Select Test Cases to be part of an
 Could be used to reflect further aspects like integration test and define sequence.
Business Area, Country, etc.  By default the order of steps in a process is  Test Package can directly be created based on a test sequence
 Relevant entity for Reporting used with the option to adjust it.  Each test case can be assigned to a different tester

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 6
Test Plan Management
Test Case Selection via attribute filters

Challenges:
• High volume of Processes and Test Cases to be managed

• Different variants of same Business Process may need to be reflected in

Test Planning

• Different Type of Tests may require different Test Case selection

• Single Functional Test
• Functional Integration Test
• User Acceptance Test
• Regression Test
• Automated Test
• …

• How to find the right Test Cases for Test to be planned ?

SAP Approach:

Classification of Business Processes and Test Cases via Attributes

These Attributes can be used in Test Case selection when generating Test
Plans or Test Packages

Profiles (combination of selection criteria) can be saved to for repeated

planning activities.

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 7
Steps to create a Test Plan
Test Plan Creation
General Data
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

 Enter a unique Test Plan ID (35 char)

 Enter a Test Plan Description 100 characters
 Enter a combination of Solution, Branch and View
 Enter a system Role. Based on the system role entered ,
the landscape to be used for Test execution is derived
and displayed below

• You can launch Quick help to know about the

application. This will give you Tab specific
information

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 9
Test Plan Creation
Settings
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

A Test Plan has multiple attributes

explained in subsequent slides
 Release Schema
 Workflow
 Executables Settings
 Test Classification
 Test set
 Document Type
 Planning Information

Once the mandatory attributes in General Data and Settings

are provided , a Test Plan can be saved before the addition of
Test cases.

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 10
Test Plan Creation
Settings → Status Schema
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• A Release Schema controls the lifecycle of a

Test Plan
• Changes to the Test Plan can be blocked
using certain status values
• Test execution can be blocked based on
3 status settings
• Its possible to assign a Signature strategy to
status values to use Digital Signatures
• SAP delivers 2 schemas by default (
Simplest and Default)
1
• Customers can create their own status
values and configure a custom schema
2 using IMG

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 11
Test Plan Creation
Settings → Workflow
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Test coordinator can decide if the Emails to testers

should be triggered during Test execution. This can be
done by setting the Workflow flag
• In case sequences are defined , the next tester gets an
email notification when previous tester has finished test
execution
• For emails to go successfully , Email address should be
maintained in Business Partner of Testers
• In case Testers do not get emails , the Test
Coordinator(Person Responsible for Test Plan) gets
notified
• Its possible to customize the smartform used to send
workflow emails

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 12
Test Plan Creation
Settings → Executables as Test Cases
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

Setting Test Case Executable Explanation

Show Test Cases Visible Hidden In Test case selection , all executables are
Only hidden. This means Test coordinator can
select only formally scripted Manual Test
cases or Automated Test cases

Show Executable Visible Hidden if Test System hides the executables if same node
when Test case case present has both executable and Test cases else the
executable is shown. This setting helps the
missing on same Test coordinator to be more flexible and
Node select Test cases when both Test cases and
executables are present else select
executables

Show Test cases Visible Visible System does not hide any entity and shows
Once the Hierarchy is generated in Tab 3 ( Test case both Test Cases and Executables so that the
and Executables
selection) or the Test Plan is saved, this setting can not be Test Coordinator can decide which one
changed should be selected in a Test Plan

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 13
Test Plan Creation
Settings → Executables as Test Cases
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Test Classification is customer defined set of values that can be used

at both Test plan level and Test Case level
• Test Classification values can be defined via the Administration – Test
Suite Tile
• These values can be used in Filters and Reporting

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 14
Test Plan Creation
Settings → Test set
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Test Set is customer defined set of values that can be used in

Reporting and Analytics
• This is an optional Attribute
• Customers can create Test sets using the IMG path shown below

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 15
Test Plan Creation
Settings → Document Type
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

 Using Document Type Administration ,

Document Types are included in the Scope
of Solution
 One of the document Type is then
designated as the doc Type to be used
when a Test Note or Test Result is created
during Test Execution

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 16
Test Plan Creation
Settings → Effort Planning
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

 It is possible to set the Planning level at

Test Plan , Test Package or Test Case
 It is also possible to Plan effort in Days,
Hours or Minutes

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 17
Select Test Cases
Test Plan Creation
Test Case Selection
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• The Solution/Branch/View is shown in a Hierarchy

• Nodes which do not have Test cases or executables
are hidden
• The Hierarchy is influenced directly by the executable
preference selected in Settings tab
• The attribute assignment Type set in Solution
Documentation also influences the display of nodes (
Additive will display child nodes if they have test
cases)
• Filter capability is provided to select the test cases or
executables
• If the Solution has changes since the time Test Plan
was saved , this tab shows change information via
flags
• The attribute ‚BP Test‘ is shown but has no direct
influence when test cases are selected manually

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 19
Test Plan Creation
Test Case Selection
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• The test cases or executables selected here are the selection

set which will be available during Test Sequence creation and
Test package creation
• This view has 2 modes , display and edit.
• Display mode shows only the selected elements at
earlier save
• Edit mode allows selection of elements
• You can select any row and click on to expand and to
collapse sub elements
• To improve the performance of this Tab, customers should
• Use ‚Views‘ in Solution Documentation to limit
information
• Use executable setting ‚Show Test case only‘ if they
do not want to consider executables
• Use parameter AGS_SMT_MAX_NODES to limit
maximum number of nodes that can be expanded

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 20
Handling Solution changes after Test Plan creation

Solution Test Plan

Solution Solution
Changes that are Flagged
1..n folders 1..n folders
• Structure Changes
Scenario 1 Scenario 1
Update
• Test Document Version
Process Process
Changes
Process Step Process Step
• Test Data Variant Changes
Scenario 2 Addition
Scenario 2
Process
Process
Process Step
Process Step
Process Step
Library
Process Step
Process Step1 Library

Process Step1
Process Step 2

Executable Executable
Library Library

Executable Executable

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 21
Select Test Cases – Handling Changes
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

Flags get changed in

General Data

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 22
Select Test Cases – Handling Changes
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• If a Test case or a Test Configuration variant

changes after a Test plan is saved , its flagged
with

• If a Node gets added its flagged with

• If a Node which was selected earlier in Test plan

gets deleted , its flagged with

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 23
Using Filters
Test Plan Creation
Test Case Selection → Using Filters
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Switch to Edit Mode in a Test Plan

• Click on Apply Filters
• Select the Filter Criteria
• Save the Filter Profile
• Click Ok to Apply the Filter Criteria

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 25
Test Plan Creation
Test Case Selection → Using Filters
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Create Groups to create complex combinations of Filter Criteria

• Entities within a Group are combined via ‚OR‘
• Entities across Groups are combined via ‚AND‘ / ‚OR‘ depending on chosen option

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 26
Test Plan Creation
Test Case Selection → Using Filters
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• Two Modes provided – Append or Overwrite

• Append - This helps the Test coordinator build Test case selection incrementally. The test cases selected via filters get added to the
earlier selection
• Overwrite – This helps the Test coordinator start over. The test cases selected via filters overwrite the earlier selection

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 27
Creating Test Sequences
Test Planning for integrations test
Test Sequence approach compared with standard approach

Standard approach: Several Testers (Tester Pool) are assigned to a

Tester 1 collection of Test Cases (Test Package)
Test
TestPackage
Package
Test Package New in 7.2
Test Case 1 Multiple Testers can set status of the same test
Tester 2
Test Case 2 case without overwriting other testers status
Test Case 3 information.
Aggregation rules available:
• Worst wins
Tester 3
• Last wins

Sequence approach:
Tester 1

Test
Test Package
Package
Test Package
Create Sales Order In addition to standard approach you can assign
Outbound Delivery Tester 2 each Test Case to a Tester and the sequence of
test cases can be process as workflow
Post Goods Issue
Create Billing Doc.

Tester 3

Example: As soon Test Case 1 has been processed successfully by Tester 1, Tester 2 will be notified by E-Mail that Test Case 2 is ready to be tested

Benefit: Less coordination effort during complex scenario tests and accelerated test execution

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 29
Test Sequence Creation
What is a Test Sequence
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• A Test sequence represents an order in which Manual Test

cases ,Automated Test cases or Executables need to be
executed
• Test sequence creation is an Optional step in Test Plan
Management
• The selection of test cases made in earlier step is made
available during Test Sequence creation
• The order in which you click the entities automatically
generates sequence numbers, which can be adjusted if
required

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 30
Test Sequence Creation
Multiple Test cases or 1 E2E Test Case
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

Use Case 1: Individual Test cases in a Sequence

Test sequence specifies the order of Test cases. Manual and Automated Test cases can be combined in
the same Test sequence

Use Case 2: One E2E document describing the flow. Test sequence is composed of Executables

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 31
Test Sequence Creation
Use Case 1 ( Without E2E Test Case)
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

1. Navigate to Test Sequence Tab

2. Click on Create
3. Enter Test sequence attributes
4. Click the entities in the desired order of execution
5. Adjust order if required
6. Save the Test Sequence

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 32
Test Sequence Creation
Use Case 2 ( With E2E Test Case)
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

1. Navigate to Test Sequence Tab

2. Click on Create
3. Enter Test sequence attributes
4. Select the End to End Test Case
5. Click on ‚Assign E2E Document‘
6. Click the executables in the desired order of execution
7. Adjust order if required
8. Save the Test Sequence

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 33
Creating Test Packages
What is a Test Package
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

A Test Package is a subset of entities selected in a Test Plan

• Test package helps break a Test plan into smaller manageable
units
• Same Test case can be assigned into multiple Test packages
within the same Test Plan
• Test Package creation is a mandatory step to start Tester
assignment
• Test package retains the Solution Documentation Context where
Test cases are assigned
• Test Package can be created in 3 ways
• Without a Test Sequence
• Using a Test Sequence as Ref
• Using a Test Sequence as Template

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 35
Test Package Creation
Test Packages – with or without Sequences
Solution Test Package with
Sequence as Ref.
1..n folders

Scenario 1

Process 1

Process Step 1

Process Step
Library

Process Step 1 Test Sequence

Process Step 2

Process Step 3

Executable
Library

Executable

Test Plan
Test Package without
Manual Test Case Automated Test Case Executable Tester Sequence

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 36
 Test Package Creation
Test Sequence - Usage
• Test sequence creation and usage is optional • Test cases must be executed in the specified order
• One Tester assigned to each test case
• Removing of Test case not possible

Test Test Test Test

Case 1 Case 2 Case 3 Case 4

Test Test Test Test

Case 1 Case 2 Case 3 Case 4

Test Test Test Test Test

Case Case Case Case Case
1 2 3 4 5

• Test cases can be executed in any order

• Tester assigned to all test cases
• Removing or adding of Test case is possible

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 37
Test Package Creation
Test Sequence Usage – Reference vs Template

Sequence as Ref Sequence as Template

Usage Use Sequence as it is to execute test Use created sequence as a starting point
entities in defined order or a collection of test cases

Order Test cases need to be executed in strict Test cases can be executed in any order
order defined in Sequence

Tester Assignment Tester assignment at Test Case level Tester assignment at Test Package level

Changes No changes allowed in Test case selection Addition and deletion of Test case
allowed
Sequence ID Seq ID appears in Test package Seq ID does not appear in Test package

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 38
Test Package Creation
Settings
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• The settings from Test Plan are displayed in Test Package but
none of them cant be changed except dates
• The dates of Test plan are defaulted to Test package but can be
adjusted for a Test Package
• A Test Plan attachment can be referred in Test Package and it can
be ensured that latest updates in Test plan attachment are visible in
Test package

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 39
Test Package Creation
Test Case Selection
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• The test cases available in Test plan are available for selection in a
Test package
• Filter capabilities are available similar to filter capabilities available
in Test Plan
• If a Test Package is created using sequence as Ref, , no changes
are allowed in Test case selection
• In all other cases , its possible to add or delete test cases , as long
as they are available in Test Plan

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 40
Test Package Creation
Tester Assignment
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

• For Test packages with seq as Ref. , Tester assignment happens at Test
case level
• In other cases, the Tester is assigned to the entire Test package
• It is possible to hide a Test package from a Tester by Locking assignment
and later releasing it

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 41
Release Test Plan for execution
Test Plan Creation
Release Test Plan for execution
Enter General Data Enter Settings Select Test Cases Create Test Sequences Create Test Packages

After Test Planning has been completed the Test Plan need to be released for execution while changing the
Release Status accordingly
© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 43
© 2016 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate
company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.

National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and
services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as
constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop
or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future
developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time
for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-
looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

© 2016 SAP SE or an SAP affiliate company. All rights reserved. Early Knowledge Transfer Customer 44