STRSW ILT SANADM - StudentGuide
STRSW ILT SANADM - StudentGuide
ONTAP SAN
Administration
Student Guide
Content Version 3
NetApp Learning Services
COPYRIGHT
© 2021 NetApp, Inc. All rights reserved. Printed in the U.S.A. Specifications subject to change without notice.
No part of this document covered by copyright may be reproduced in any form or by any means—graphic, electronic, or mechanical,
including photocopying, recording, taping, or storage in an electronic retrieval system—without prior written permission of NetApp, Inc.
TRADEMARK INFORMATION
NETAPP, the NETAPP logo, and the marks listed at http://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and
product names may be trademarks of their respective owners.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Welcome......................................................................................................................................................... 1
Module 1: NetApp ONTAP SAN Fundamentals ........................................................................................ 1-1
Module 2: NetApp ONTAP SAN Resource Provisioning .......................................................................... 2-1
Module 3: ONTAP Storage Virtual Machine Administration .................................................................... 3-1
Module 4: SAN Availability and Data Protection ...................................................................................... 4-1
Module 5: Management of NetApp ONTAP SAN Environments .............................................................. 5-1
Appendix A: SAN Migration with Foreign LUN Import ............................................................................ A-1
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Be sure that you have your Student Guide and Exercise Guide
ONTAP SAN Test your headset and microphone (virtual sessions only)
administration
Provide yourself with two screens (virtual sessions only)
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Admin Network
JUMPHOST
192.168.0.x
Cluster Network
DC1 Cluster 1 (2-node HA-Pair)
demo.netapp.com
10.10.10.x
iSCSI Network
e0f
RHEL1 e0g
10.10.20.x
0c,0d
Launch your exercise equipment kit from your laptop or from the classroom desktop. To connect to your exercise
equipment, use Remote Desktop Connection (RDC) or the NetApp Learning Services portal.
The Windows 2016 Server is your windows domain controller for the DEMO windows domain. The Windows Server hosts
the domain DNS server.
Your exercise equipment consists of several servers:
1. One 2-node NetApp ONTAP cluster
2. Two Windows 2016 R2 Servers
3. One Red Hat Enterprise Linux Server
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
The NetApp Learning Services overview page is your front door to learning. Find training that fits your learning map and
your learning style, learn how to become certified, link to blogs and discussions, and subscribe to the NetApp newsletter
Tech OnTap. http://www.netapp.com/us/services-support/university/index.aspx
The NetApp Learning Services Community page is a public forum for NetApp customers, partners, and employees.
NetApp Learning Services welcomes your questions and comments.
https://community.netapp.com/t5/NetApp-Learning-Services/ct-p/netapp-university
The NetApp Learning Services Support page is a self-help tool that enables you to search for answers to your questions
and to contact the NetApp Learning Services support team. https://netapp.service-now.com/learningcenter/
Are you new to NetApp? If so, register for the New to NetApp video series to acquaint yourself with the facts and tips that
help to ensure that you have a successful support experience.
https://mysupport.netapp.com/site/info/new2support
The NetApp Support page is your introduction to all products and solutions support: http://mysupport.netapp.com. Search
for products, downloads, tools, and documentation or link to the NetApp Support Community
https://community.netapp.com/t5/Products-and-Services/ct-p/products-and-solutions
Join the Customer Success Community to ask support-related questions, share tips, and engage with other users and
experts. https://forums.netapp.com/
Search the NetApp Knowledge Base to use the accumulated knowledge of NetApp users and product experts.
https://kb.netapp.com/support/index?page=home
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
About this module • List the ONTAP components that are integrated in a SAN
environment
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Corporate
LAN
Operating systems and applications request data at either the [1] file level or the [2] block level:
Network-attached storage (or NAS) provides file-level access to data on a storage system. Data is accessible from a
network that uses ONTAP services. These services include [3] Common Internet File System (or CIFS), which is a
dialect of Server Message Block (SMB) and [4] Network File System (NFS). In a NAS environment, [5] ONTAP
controls the file system.
A storage area network (SAN) provides block-level access to data, typically in units of 512 bytes. SAN solutions can
use the [6] iSCSI, [7] FCoE, [8] FC, and NVMe over Fabrics (NVMe-oF) protocols. In a SAN environment, the [9] host
operating system controls the file system.
[10] A storage system that includes both NAS and SAN protocols is called a “unified storage system.”
This course focuses on the use of SAN protocols to access block data.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Host (client)
FAS drives
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Host (client)
Drives
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
TCP/IP FC
Driver iSCSI Driver
Driver
SCSI Driver
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Data SVM
LIFs are virtualized interfaces that are assigned to storage virtual
machines (SVMs). SAN data LIFs have the following qualities:
Cluster Interconnect
HA Pair
• Are assigned a home node and port LIF1
HA Pair
LIF2
• Can be grouped into port sets
Data SVM
SAN data LIFs do not migrate or fail over as NAS data LIFs do. However, the SAN data LIFs can be moved to another
node or port in the SVM.
To move a data LIF with SAN protocols, use the network interface modify command:
1. To view the current status of a LIF, use the network interface show command.
2. Change the administrative status of the LIF to down (offline).
3. Use the network interface modify command to change the location of the LIF to a new node or port.
4. Change the status admin of the LIF to up (online).
5. Verify the changes by using the network interface show command.
NOTE: Although the LIFs must be offline to move, if they are configured by using NetApp best practices, the host
continues to serve data nondisruptively.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
LUNs
Target
Initiator
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
LUNs on a NetApp storage system can be accessed through either of the following:
An FC SAN fabric that uses the Fibre Channel Protocol
An Ethernet network that uses either the FCoE protocol or the iSCSI protocol
In all cases, the transport portals (FC, FCoE, or iSCSI) carry encapsulated SCSI commands as the data transport
mechanism.
Similarly, an NVMe namespace can be accessed through an FC SAN fabric that uses the NVMe-oF protocol or through
an Ethernet network that uses the NVMe protocol.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Aggregate
• SVM
Components common to
• Volume NAS and SAN
• LIF
• LUN or namespace
The first four components listed are also used in NAS environments, so they should be familiar. For SAN environments,
several new components have been introduced.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
≈
A LUN and a namespace each represent an array
of data blocks that are presented to a host.
LUN or
Hard Disk
Namespace
Spinning hard drives are accessed by specifying the cylinder, head, track, and sector where the data resides.
In direct memory access, memory (NVMe storage) is accessed with a logical block address (LBA), starting at 0 and going
up to (n-1).
Because data is accessed in 4k blocks, a 4-MB namespace has 1024 LBAs, 0000x – 03FFx.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
An igroup and a subsystem are a collection of addresses (WWPNs, IQNs, or NQNs) that are enabled to
communicate with LUNs on an SVM.
LUN
LUN0 svm_a
Mapping
Host A WWPN LUN1
FC LUN2
Host A
igroup1
LUN
Host B IQN Mapping LUN0
iSCSI LUN1
Host B
igroup2
svm_b
Namespace0
Namespace1
NVMe Host C NQN
Namespace Namespace2
ID
Host C
subsystem3
A LUN in ONTAP software is a logical representation of an attached SCSI drive. As was mentioned, SAN is often called
“block-based storage.” The “block” refers to the logical blocks that the host writes to, just as it would write to an attached
SCSI drive. Traditionally, the logical blocks are 512 bytes per sector.
Drive manufacturers have started to use 4,096-byte (4KB) sectors, called Advanced Format, in new drive geometry.
Currently, ONTAP LUNs use the SCSI standard of 512 bytes per sector.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
The “SAN Administration Guide” shows you how to configure and manage LUNs, igroups, and targets that use the iSCSI
and FC protocols and namespaces and subsystems using the NVMe over Fibre Channel (NVMe/FC) protocol.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
https://www.netapp.com/us/media/tr-4080.pdf
https://www.netapp.com/us/media/tr-4684.pdf
Many more documents are at the following link:
https://www.netapp.com/us/documentation/index.aspx
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
AFF A800
Specifications
• Hardware Universe includes the following
ONTAP 9.8 software
object count maximums:
• Volumes System Maximums and Limits
• LUNs
Maximum storage devices
• LIFs per port 240 (drives)
(high availability)
• SVMs
• Namespaces Platform Maximum and Limits Configuration: High Availability
Maximum number of
256
namespaces
The example shown is for an AFF A800, which is an enterprise class system. Smaller systems should be expected to
have lower maximums.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. LUN
c. Namespace
d. Datastore
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. Subsystem
c. Namespace
d. Initiator group
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Module summary • List the ONTAP components that are integrated in a SAN
environment
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Volumes and LUNs provide a set of configuration options that determines how the volumes and LUNs use and provide
space. Use the right combination of LUN and volume configurations for your environment.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Thin provisioned volumes do not have any extra space reserved when the volume is created.
• When the aggregate runs out of free space, the volume might not be writable.
When a thinly provisioned volume is created, ONTAP software does not reserve any extra space when the volume is
created. As data is written to the volume, the volume requests the storage that it needs from the aggregate to
accommodate the write operation. Using thin-provisioned volumes enables you to overcommit your aggregate, which
introduces the possibility of the volume not being able to secure the space that it needs when the aggregate runs out of
free space.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
aggr1
Space is not reserved for
Characteristics of non-space-reserved LUNs with thin the volume in the aggregate
volume provisioning: at creation.
- Must be monitored closely in their environment to ensure Space is not reserved for
that they do not run out of space and go offline the LUN in the volume at
vol1 creation.
If you do not allocate the storage for a FlexVol volume or a LUN up front, you might overcommit the storage object that
supplies the volume or LUN storage. Overcommitting a storage object can increase your storage efficiency, but
overcommitment also requires that you actively monitor your free space to prevent writes from failing due to lack of space.
A storage object is said to be overcommitted if the objects to which it supplies storage are collectively larger than the
amount of physical storage that it can currently supply. For example, suppose that you have three FlexVol volumes with
“none” guarantees and they are associated with a 100TB aggregate. If each of the FlexVol volumes has a nominal size of
40TB, the aggregate is overcommitted. Each of the volumes can continue to receive data provided that, collectively, their
physical storage requirement is not more than 100TB. This situation is possible only if the volumes have a volume
guarantee of “none” (they are thin-provisioned). If all the volumes have a volume guarantee of “volume,” you cannot
create the third volume.
Similarly, you can overcommit a volume that contains more than one LUN if the LUNs are not space-reserved.
The ability to overcommit the supplying storage object is the promise that is offered by not preallocating storage for an
object up front. However, overcommitment requires that you manage the supply of physical storage resources carefully to
avoid running out of free space. You must manage storage in any configuration. However, if your supplying storage object
is overcommitted, you can run out of free space by issuing a write to space that appears to have been allocated.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Thick provisioned volumes are allocated sufficient space from the aggregate to ensure that any block can be
written to at any time.
• Storage efficiency capabilities, such as compaction, compression, and deduplication, can be used to offset
the larger upfront storage requirements.
When a thick-provisioned volume is created, ONTAP sets aside enough storage from the aggregate to ensure that any
block in the volume can be written to at any time. When you configure a volume to use thick provisioning, you can employ
any of the ONTAP storage efficiency capabilities, such as compression and deduplication, to offset the larger upfront
storage requirements.
Setting the -space-slo option to thick ensures the following:
The entire volume is preallocated in the aggregate. You cannot use the volume create or volume
modify command to configure the volume's -space-guarantee option.
100% of the space that is required for overwrites is reserved. You cannot use the volume modify command to
configure the volume's -fractional-reserve option.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
aggr1
Space is reserved for the
Characteristics of space-reserved LUNs with volume in the aggregate
thick volume provisioning: at creation.
+ Guarantee space for the LUN
+ Require less monitoring
+ Are good for small installations
- Are the least efficient configuration
You should understand how the space reservations setting and the volume guarantee affect how space is set aside for
LUNs. If you understand how space is set aside, you can better understand when space is set aside for your LUNs.
Certain combinations of LUN and volume settings do not work together.
When a LUN has space reservations enabled, and its containing volume has a volume guarantee: free space from the
volume is set aside for the LUN at creation time. The size of the LUN determines the size of the reserved space. Other
storage objects in the volume (such as other LUNs, files, and Snapshot copies) are prevented from using the space.
When a LUN has space reservations disabled (a non-space-reserved LUN), no space is set aside for that LUN at creation
time. The amount of space that is required for a write operation to the LUN is allocated from the volume when it is needed
if sufficient free space is available.
If a space-reserved LUN is created in a none-guaranteed volume, the LUN behaves the same as a non-space-reserved
LUN. The LUN behaves that way because a none-guaranteed volume has no space that it can allocate to the LUN. The
volume itself can allocate space only as it is written to, due to its none guarantee. For this reason, creating a space-
reserved LUN in a none-guaranteed volume is not recommended. Using this configuration combination might seem to
provide write guarantees that are impossible.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
When a volume using semi-thick provisioning is created, ONTAP sets aside storage space from the aggregate to account
for the volume size. If the volume is running out of free space because blocks are in use by block-sharing technologies,
ONTAP tries to delete protection data objects (Snapshot copies and FlexClone files and LUNs) to free up the space that
they are holding. If ONTAP can delete the protection data objects fast enough to keep pace with the space that is required
for overwrites, the write operations continue to succeed. This process is called a “best effort” write guarantee.
Setting the -space-slo option to semi-thick ensures the following:
The entire volume is preallocated in the aggregate. You cannot use the volume create or volume
modify command to configure the volume's -space-guarantee option.
No space is reserved for overwrites. You can use the volume modify command to configure the volume's -
fractional-reserve option.
Automatic deletion of Snapshot copies is enabled.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
ONTAP provides three basic options for provisioning volumes. Each option uses different ways to manage the volume
space and the space requirements for ONTAP block-sharing technologies. Understanding how the options work enables
you to select the best option for your environment.
Thin provisioning for volumes
When a thin provisioned volume is created, ONTAP software does not reserve any extra space. As data is written to the
volume, the volume requests the storage that it needs from the aggregate to accommodate the write operation. Using thin
provisioned volumes enables you to overcommit your aggregate. However, if the aggregate is overcommitted, the volume
might not be able to secure the space that it needs when the aggregate runs out of free space.
Thick provisioning for volumes
When a thick provisioned volume is created, ONTAP software sets aside enough storage from the aggregate to ensure
that any block in the volume can be written to at any time. When you configure a volume to use thick provisioning, you can
employ the storage efficiency capabilities of ONTAP software, such as compression and deduplication, to offset the larger
upfront storage requirements.
Semi-thick provisioning for volumes
When a volume that uses semi-thick provisioning is created, ONTAP software sets aside storage space from the
aggregate to account for the volume size. If the volume is running out of free space, ONTAP software tries to delete
protection data objects (Snapshot copies, FlexClone files, and FlexClone LUNs) to free the space that they are holding. If
ONTAP software can delete the protection data objects fast enough to keep pace with the space that is required for
overwrites, the write operations continue to succeed. This situation is called a "best effort“ write guarantee.
You cannot employ storage efficiency technologies such as deduplication and compression on a volume that is using
semi-thick provisioning.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Enables host applications to • Can be created in one of • Is the default setting when creating
support SCSI features, including two ways: a LUN
the following: • In NetApp ONTAP System
• LUN space reclamation Manager, select
thin provisioning.
• LUN space monitoring
• Use the -space-reserve
• Is enabled in ONTAP software command line option.
by using the
-space-allocation option
The definition of a “thin provisioned LUN” change, depending on the context. The T10 SCSI standard uses one definition.
Historically, NetApp has used a different definition.
SCSI thin provisioned LUNs
The T10 SCSI standard defines two types of LUNs: Thin provisioned LUNs and fully provisioned LUNs. ONTAP supports
both types of T10-standard LUNs.
SCSI thin provisioning (sometimes called T10 thin provisioning) is a set of SCSI features and capabilities that ONTAP
software supports. The SCSI host software must support these SCSI features. SCSI thin provisioning enables host
applications to support SCSI features, including LUN space reclamation and LUN space monitoring capabilities for block
environments.
If your host software supports SCSI thin provisioning, you can use it with space-reserved and non-space-reserved LUNs
and with any volume provisioning type. You use the ONTAP space-allocation setting to enable SCSI thin provisioning on a
LUN.
NetApp thin provisioned (non-space-reserved) LUNs
Historically, NetApp has used the term “thin provisioned LUN” to mean a LUN for which space reservations are disabled
(a non-space-reserved LUN). A non-space-reserved LUN shares some important characteristics with a thin provisioned
volume. Its storage is allocated as it is used, rather than at creation time, and its containing storage object can be
overcommitted. The term “non-space-reserved LUN” is used for this configuration.
If you create a LUN in ONTAP System Manager and select the thin provisioning checkbox, you are creating a non-space-
reserved LUN.
When a LUN has space reservations enabled (a space-reserved LUN, the default setting) and its containing volume has a
volume guarantee, free space from the volume is set aside for the LUN at creation. The size of the LUN determines the
size of the reserved space. Other storage objects in the volume (such as other LUNs, files, and Snapshot copies) are
prevented from using the space.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
There are specific combinations of FlexVol volume and LUN configurations that you can use, depending on your
application and administration requirements. Understanding the costs and benefits of the combinations can help you to
determine the right volume and LUN configuration combination for your environment.
Non-space-reserved LUNs with thin volume provisioning
Benefits:
There are no restrictions on storage efficiency and data protection technologies on the volume.
Space is allocated only as it is used.
Costs and restrictions:
Writes are not guaranteed. If the volume runs out of free space, LUNs go offline.
You must manage the free space in the aggregate effectively to prevent the aggregate from running out of free space.
Space-reserved LUNs with thick volume provisioning
Benefits:
All writes are guaranteed. LUNs do not go offline due to insufficient space.
You do not need to monitor your free space.
There are no restrictions on storage efficiency and data protection technologies on the volume.
Costs and limitations:
Enough space must be set aside from the aggregate up front to support the thick provisioned volume.
Space equal to twice the size of the LUN is allocated from the volume at LUN creation time.
Space-reserved LUNs with semi-thick volume provisioning
Benefits:
Less space is reserved up front than for thick volume provisioning.
A best-effort write guarantee is still provided.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
1. To guarantee overwrites or provide best-effort overwrite assurance, you must enable space reservations on
the LUN or file.
2. Protection data includes Snapshot copies, FlexClone files, and FlexClone LUNs that are marked for
automatic deletion (backup clones).
3. Storage efficiency capabilities include deduplication, compression, any FlexClone files, LUNs that are not
marked for automatic deletion (active clones), and FlexClone subfiles (copy offload).
The table on this slide summarizes the major differences between the three volume provisioning options.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Can you
Write accept a
guarantees “best-effort” write No Space-reserved LUN
Writes are guaranteed to
guarantee for with thick volume
succeed.
some space provisioning
savings?
Answering a few basic questions about your environment can help you to determine the best FlexVol volume and LUN
configuration for the environment.
You can optimize your LUN and volume configurations for maximum storage utilization or for the security of write
guarantees. You must select the optimization that makes the most sense for your installation, based on your requirements
for storage utilization and your ability to monitor and replenish free space quickly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Autosize
• Fractional Reserve
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Possible settings
• 5% (new default for NAS as of ONTAP 9.0 software,
previously set to 20%)
• 0% (current default for SAN within ONTAP System Manager,
Snapshot reserve
not the current default for CLI)
• 0% to 100%
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Vserver : epic_1
Volume : tme01
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
With Fractional Reserve = 100% space is reserved so that the entire LUN can be overwritten. This setting is referred to as
2X + Δ (delta).
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Description
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
To use the ONTAP CLI to create a Storage VM, you can specify the following parameters:
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
After the SVM is created, you must configure the SVM for a protocol:
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Specify the following parameters to use the ONTAP CLI to create a LIF:
• SVM name • IP address and network mask
(iSCSI LIFs)
• LIF name (unique for every LIF)
• Home node and home port
• Data protocol
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
For every LUN, you should create a volume by entering the following parameters:
• Volume name (unique for every volume)
• Hosting SVM
• Containing aggregate
• Size
Snapshot policy
Snapshot space percentage
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
cluster1::> igroup create -vserver svm02 -igroup ig_svm02 -protocol iscsi -ostype
windows -initiator iqn.1991-05.com.microsoft:win1.demo.netapp.com
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• For the host to be able to see a LUN, the LUN must be mapped to an igroup
that contains the host initiator worldwide port name (WWPN) or iSCSI Qualified
Name (IQN).
• To map a LUN to an initiator group, enter the following parameters:
• SVM name
• LUN path (unique for every LUN)
• Initiator group name
• LUN ID (unique for every LUN in the igroup)
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
For every namespace, you should create a volume by entering the following parameters:
• Volume name (unique for every volume)
• Hosting SVM
• Containing aggregate
• Size
• Snapshot policy
• Snapshot space percentage
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• For the host to be able to see a namespace, the host NVMe Qualified Name
(NQN) must be mapped to the subsystem.
• To map a host to a subsystem, enter the following parameters:
• SVM name
• Subsystem name
• Host NQN
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• For the host to be able to see a namespace, the namespace must be mapped
to a subsystem that contains the host NQN.
• To map a namespace to a subsystem, enter the following parameters:
• SVM name
• Subsystem name
• Namespace path
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
On clusters with more than four nodes, you should limit the number of paths that are used to access each LUN. Limiting
the number of paths helps to avoid exceeding the operating system path count limitation. Selective LUN map and port
sets can be used.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• iSCSI access control enables administrators to use an access control list to enhance iSCSI security.
• A list of approved IP addresses or a range of IP addresses can be specified to restrict unapproved access to a
LUN.
cluster1::> vserver iscsi security add-initiator-address-range -vserver vs1
-initiator-name iqn.1993-08.com.example:01:e3f87c7cf2e4 -initiator-address-range
192.168.2.1-192.168.2.255
• Challenge-Handshake Authentication Protocol (CHAP) can also be used to gate iSCSI connections.
The vserver iscsi security create command uses the –auth-type parameter to configure the authentication
type. The syntax is:
[-auth-type | -s {CHAP|deny|none}] - Authentication Type
The following specifies the authentication type:
- CHAP: Authentication is by using a Challenge-Handshake Authentication Protocol (CHAP) user name and password.
- none: The initiator can access the Vserver without authentication.
- deny: The initiator cannot access the Vserver.
An access control list can be defined with the following command:
cluster1::> vserver iscsi security add-initiator-address-range -vserver vs1 -initiator-
name iqn.1993-08.com.example:01:e3f87c7cf2e4 -initiator-address-range 192.168.2.1-
192.168.2.255
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Admin network
JUMPHOST
192.168.0.x
Cluster network
DC1 Cluster 1 (2-node HA pair)
demo.netapp.com
10.10.10.x
iSCSI network
e0f
RHEL1 e0g
10.10.20.x
0c,0d
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
10.10.10.1
JUMPHOST
10.10.20.1
Cluster network
DC1
demo.netapp.com
Cluster 1 (2-node HA pair)
10.10.10.30
10.10.10.31
e0f
iSCSI network
10.10.10.2
RHEL1 e0g
10.10.20.2
10.10.20.30
10.10.20.31
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. SVM name
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. active/optimized
b. active/nonoptimized
c. direct/optimized
d. inactive/nonoptimized
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
d. CHAP
e. LUN mapping
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. True
b. False
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Description
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
Direct: Active/Optimized FC or Ethernet
Indirect: Active/Non-Optimized HA Interconnect
Cluster Interconnect
During normal operation, the direct paths provide the fastest path of access to the LUN. The direct paths are the paths
that use LIFs on the same node as the LUN. All the indirect paths have to cross the cluster interconnect to access the
LUN.
Direct paths are marked as Active/Optimized and indirect paths are Active/Non-Optimized in asymmetric logical unit
access (ALUA).
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA Interconnect
Cluster Interconnect
Introduced in ONTAP 9.7, symmetric active-active is a multipathing solution in which all paths are active/optimized. If
there is a storage failover, the host does not need to wait for the ALUA transition of the failover paths to resume I/O. This
behavior reduces time to fail over. Symmetric active-active is available on only All SAN Arrays. ANA continues to operate
in an asymmetric manner, with Active Optimized and Active Non-Optimized paths in an All SAN array.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA Interconnect
Cluster Interconnect
During a high-availability (HA) failover event, the LIFs on the impaired node are seen as offline. During the event, the LIFs
on the HA partner that has taken over for the impaired node become direct paths. The state changes automatically
because of ALUA path inquiry. No administrative changes are required.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA Interconnect
Cluster Interconnect
In a port or switch failure, no more direct paths are available. Path priority remains the same and the Microsoft Multipath
I/O (MPIO) software on the host selects alternate indirect paths until a direct path becomes available again. The ALUA
path states do not change.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA Interconnect
Cluster Interconnect
When a volume or LUN is moved to a different node, the direct paths are available until the cutover occurs. It is important
to add the new nodes or HA pairs to the reporting-nodes list.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA Interconnect
Cluster Interconnect
After the cutover is complete, the ALUA path states are updated and the host accesses the LUN through the new direct
paths. All paths remain available always.
For a volume move, the cutover occurs after the data has been moved to the new location.
For a LUN move, the cutover occurs before all the data has been moved. New writes go to the destination. If the data has
not been moved yet, reads are passed to the source node.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Selective LUN Map (SLM) reduces the number of paths from the host to the LUN.
• With SLM, when a new LUN map is created, the LUN is accessible only through paths on the node that owns
the LUN and its HA partner.
• If you are moving a LUN or a volume containing LUNs to another HA pair within the same cluster, you should
modify the SLM reporting-nodes list before initiating the move.
• This action ensures that active, optimized LUN paths are maintained.
::> lun mapping add-reporting-nodes -vserver svm01 -path /vol/vol1/lun1
-igroup igroup1 -destination-volume vol2
Most customers opt to remove the former owning nodes from the reporting-nodes list after the LUN or volume has been
moved. This decision can require a rescan on the host to recognize the current path status.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Disk
FC or Ethernet
HA interconnect
Cluster interconnect
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• SnapVault
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• LUNs that are being backed up need to be in an application-consistent state before the Snapshot copy is
created.
• Products like NetApp SnapCenter software create consistent Snapshot copies.
1
Host Native API
Plug-In 6
Quiesce 2 5 Thaw
Host
Plug-In
Application
SnapCenter FC, iSCSI, Writer
Server NVMe/FC
Backup Workflow:
1. A NetApp SnapCenter backup is begun based on the policy that is assigned to the resource group that contains the
databases. The policy includes a set of rules that governs the SnapCenter backup, such as backup type, schedule
frequency, retention, and prescript and postscript with arguments. SnapCenter software (the requestor) sends a
command to the Microsoft Volume Shadow Copy Service (VSS) to do the following:
a. Enumerate the writers
b. Gather the writer metadata (like the Exchange Database [EDB] file, logs, and checkpoint file for each database
that is selected for backup)
c. Enable the creation of Snapshot copies
2. The operating system native API then sends a command to the writer to prepare for a Snapshot backup. To enable
the SnapCenter backup operation, the Exchange writer temporarily “freezes” the database and puts it into read-only
mode. All data in server memory is flushed to the database. The temporary freeze holds for a couple of seconds,
during which all writes to the databases on disk are stopped and all the write I/O is queued.
Note: The “freeze” action enables data to be written in a consistent order and a consistent Snapshot copy to be
created in the SnapCenter software.
3. The API then sends a command to NetApp storage through SnapCenter Plug-in for Microsoft Windows (VSS storage
provider) to create a point-in-time Snapshot copy of the volumes that host the database.
4. When the NetApp Snapshot operation is complete, completion is acknowledged. The logs are truncated, and the
database headers are updated.
5. To resume regular operations, the VSS “thaws” the database and releases the writer. All queued write I/O that is held
in RAM is written to disk again.
6. The API then acknowledges to SnapCenter software the successful backup operation. SnapCenter Server updates
the backup metadata with the status. The application database also truncates the transaction logs and records the
time of the most recent backup for the database.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• After a Snapshot copy is created on the primary storage, it can be replicated as needed to meet
business requirements.
Snapshot Snapshot
copies copies
Building 1 Building 2
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
LUNs that have been moved do not bring with them any associated Snapshot copies that might exist in the source
volume.
LUNs that have been moved might not participate in data protection relationships that are associated with the original
source volume. A new SnapMirror relationship might be required.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. The application was busy when the server encountered a system panic.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Description
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Move and LUN Copy • The destination SVM has a different worldwide node name
(WWNN) or iSCSI target name.
• A LUN move is a nondisruptive operation. Changing the
parameters might cause the LUN to become inaccessible.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Storage efficiency and data • LUNs that have been moved do not bring with
them any associated Snapshot copies.
protection considerations
• LUNs that have been moved do not necessarily
participate in any data protection relationships
that are associated with the source volume.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
Offloaded data transfer (ODX) also supports copying data between Windows LUNs and CIFS shares.
The copy process is performed within the ONTAP storage system. If the destination is a different cluster, the copy offload
process is not used, and data is copied across the network.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Volume size
Volume and LUN • Manual
reconfiguration
• Automatic
• Volume move
• LUN size
• Manual
System Manager can be used to move, resize volumes, and enable automatic volume sizing.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• SAP HANA
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Databases:
Systems that enhanced • Oracle
• Virtual systems:
• VMware
• Hyper-V
• Xen
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. Volumes
b. SVMs
c. Clusters
d. Aggregates
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. HPE UX
b. ESXi
d. Microsoft Windows
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
a. The node that owns the disks that contain the aggregate with the LUN
b. The node that owns the LUN and that node’s HA partner.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
• Description
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
If you encounter an issue, notify your instructor immediately so that the issue can be resolved promptly.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
https://metricsthatmatter.com/netappinternal
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-1 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-2 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-3 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
SAN Host
Target Initiator
Initiator Ports Ports
Ports
FC SAN Destination
Target
Ports Initiator Target
Ports Ports
Third-Party Array
Source
A-4 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-5 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-6 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-7 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
6. Clean up zoning.
A-8 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
The Foreign LUN Import IMT is specific to Foreign LUN Import interoperability
Foreign LUN Import Interoperability Matrix Tool (IMT) before and during LUN imports. The IMT is used primarily to verify that the
end-state is supported.
NetApp Active IQ OneCollect is a data collection tool that collects data from
NetApp Active IQ OneCollect storage, hosts, and switches. The collected data is used for troubleshooting,
solution validation, migration, and upgrade assessments.
A-9 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-10 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-11 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-12 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-13 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-14 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
https://metricsthatmatter.com/netappinternal
Your feedback is important for ensuring the quality of NetApp courses. Your instructor will give you instructions about how
to find the survey for this class and about how to use the survey website.
A-15 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-16 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.
A-17 ONTAP SAN Administration: SAN Migration with Foreign LUN Import
© 2021 NetApp, Inc. This material is intended only for training. Reproduction is not authorized.