Storage Technology Basics

Foreword

⚫ Data is the most important asset for every enterprise. This course describes
how and where data is stored, and provides the key data storage
technologies in cloud computing.

2 Huawei Confidential
 Objectives

Upon completion of this course, you will be able to:

 Understand mainstream data storage modes and network topologies.
 Master RAID and Huawei RAID 2.0+ block virtualization technologies.
 Distinguish between centralized and distributed storage.
 Understand storage protocols and application scenarios.

3 Huawei Confidential
 Contents

1. Storage Basics
◼ Definition of Storage
 History of Storage
 Mainstream Disk Types
 Storage Networking Types
 Storage Forms

2. Key Storage Technologies

4 Huawei Confidential
 What Is Storage?

Database server
User
Application server

Mail server

Online storage Nearline storage Offline storage

Employees Based on the data access frequency

File server

Data Data Data

generation processing management

5 Huawei Confidential

• Storage in a narrow sense: CDs, DVDs, ZIP drives, tapes, and disks...

• Storage in a broad sense:

▫ Storage hardware (disk arrays, controllers, disk enclosures, and tape

libraries)

▫ Storage software (backup software, management software, and value-

added software such as snapshot and replication)

▫ Storage networks (HBAs, Fibre Channel switches, as well as Fibre Channel

and SAS cables)

▫ Storage solutions (centralized storage, archiving, backup, and disaster

recovery)
 Contents

1. Storage Basics
 Definition of Storage
◼ History of Storage
 Mainstream Disk Types
 Storage Networking Types
 Storage Forms

2. Key Storage Technologies

6 Huawei Confidential
 History of Storage

Controller

This century
1990s • Distributed
• Storage storage
1980s network • Cloud storage
• External
1950s storage
• Traditional
storage

7 Huawei Confidential

• The storage architecture has gone through the following development phases:
traditional storage, external storage, storage network, distributed storage, and
cloud storage.

• Traditional storage refers to individual disks. In 1956, IBM invented the world's
first mechanical hard drive that has fifty 24-inch platters and the total storage
capacity of just 5 MB. It is about the size of two refrigerators and weighs more
than a ton. It was used in the industrial field at that time and was independent of
the mainframe.

• External storage refers to direct-attached storage. The earliest form of external

storage is JBOD, which stands for Just a Bunch of Disks. JBOD is identified by the
host as numerous independent disks. It provides large capacity but low security.

• A storage area network (SAN) is a typical storage network that transmits data
mainly over a Fibre Channel network. Then, IP SANs emerge.

• Distributed storage uses general-purpose servers to build storage pools and is

more suitable for cloud computing. This will be introduced later.
Storage Development - from Server Attached Storage to
Independent Storage Systems
External disk array Intelligent disk array

Disk in the server CPU Server Server

CPU

Server RAM
RAM
CPU
Disk Disk
RAM

SCSI card
SCSI card
Disk
Controller

Just a Bunch Of Disks (JBOD)

A controller provides the RAID function and

Restrictions: JBOD logically connects several large-capacity cache, and enables the disk
• Disks become the system performance physical disks to increase capacity. array to have multiple functions for better
bottleneck. read/write performance and data security.
• The number of disk slots is limited, thereby
limiting capacity. Problems solved:
• Data is stored on a single disk, lowering data Problem solved: •Disks become the system performance
reliability. bottleneck.
• The number of disk slots is
• Storage space utilization is low. • The number of disk slots is limited, thereby
• Data is scattered in local storage systems. limited, thereby limiting capacity. limiting capacity.
• Data is stored on a single disk, lowering data
reliability and read/write performance.
8 Huawei Confidential
 Storage Development - from Independent Storage Systems
to Network Shared Storage
SAN NAS

Server Server
Server
CPU
Server
File system File system
RAM

SAN LAN

Disk
RAID File system
SCSI card

Controller Problems solved:

• Disks become the system performance bottleneck.
• The number of disk slots is limited, thereby limiting capacity.
• Data is stored on a single disk, lowering data reliability and read/write performance.
• Storage space utilization is low.
• Data cannot be shared.

9 Huawei Confidential

• The direct connection between the controller and server resolves the problems
caused by the limited disk slot quantities, single-disk storage, and poor disk
interface performance.

• However, other problems remain, such as low storage space utilization,

decentralized data management, and inconvenient data sharing. We will learn
how the network shared storage in SAN or NAS mode solves these pain points.
 Contents

1. Storage Basics
 Definition of Storage
 History of Storage
◼ Mainstream Disk Types
 Storage Networking Types
 Storage Forms

2. Key Storage Technologies

10 Huawei Confidential
 Introduction to Disks
⚫ Disks can be considered the most important storage device of a computer.
⚫ A disk interface is a component used to connect a disk to a host. It transmits data between the disk cache and the
host memory. The disk interface type determines the connection speed between the disk and the computer, the
program running speed, and system performance.

SATA SAS NL-SAS SSD

Rotational speed
7,200 15,000/10,000 7,200 N/A
(rpm)
Serial/Parallel Serial Serial Serial Serial

Capacity (TB) 1 TB/2 TB/3 TB 0.6 TB/0.9 TB 2 TB/3 TB/4 TB 0.6 TB/0.8 TB/1.2 TB/1.6 TB

MTBF (h) 1,200,000 1,600,000 1,200,000 2,000,000

SAS disks are designed to meet Solid state disks (SSDs) are made
enterprises' high performance NL-SAS disks are enterprise-class up of solid-state electronic storage
Being developed from ATA requirements, and are compatible SATA drives with SAS interfaces. They chip arrays.
disks, SATA 3.0 supports up with SATA disks. The transmission are applicable to storage tiering in a Each SSD consists of a control unit
to 600 MB/s data transfer. rate ranges from 3.0 Gbit/s to 6.0 disk array, which simplifies the design and a storage unit (DRAM or flash
Remarks
Gbit/s, and will be increased to 12.0 of the disk array. chip). SSDs are the same as the
The annual failure rate of Gbit/s. common disks in the regulations
SATA disks is about 2%. The annual failure rate of NL-SAS and definition of interfaces,
The annual failure rate of SAS disks is disks is about 2%. functions, usage, as well as the
less than 2%. exterior and size.
11 Huawei Confidential

• MTBF: Mean Time Between Failure

• Increasing order of price: SATA and NL-SAS disks, SAS disks, and SSDs
 Disk Key Indicators
⚫ Disk capacity IOPS Bandwidth
Disk Type (4 KB random (128 KB
⚫ Rotational speed (HDD only) write) sequential read)

SATA 330 200 MB/s

⚫ Average access time
⚫ Data transfer rate SAS 10K 350 195 MB/s

⚫ Input/Output operations per second (IOPS) SAS 15K 450 290 MB/s

Higher IOPS,
SATA SSD 30,000 to 60,000 540 MB/s
IOPS

better performance

SAS SSD 155,000 1000 MB/s

Maximum read
bandwidth
Maximum NVMe SSD 300,000 3500 MB/s
write
Bandwidth bandwidth

12 Huawei Confidential

• Disk capacity: The capacity is measured in MB or GB. The factors that affect the
disk capacity include the single platter capacity and the number of platters.

• Rotational speed: The rotational speed is the number of rotations made by disk
platters per minute. The unit is rotation per minute (rpm). In most cases, the
rotational speed of a disk reaches 5400 rpm or 7200 rpm. The disk that uses the
SCSI interface reaches 10,000 rpm to 15,000 rpm.

• Average access time = Average seek time + Average wait time

• Data transfer rate: The data transfer rate of a disk is the speed at which the disk
reads and writes data. It is measured in MB/s. The rate consists of the internal
data transfer rate and the external data transfer rate.

• Input/Output operations per second (IOPS): indicates the number of input/output

operations or read/write operations per second. It is a key indicator to measure
disk performance. For applications with frequent random read/write operations,
such as online transaction processing (OLTP), IOPS is a key indicator. Another key
indicator is the data throughput, which indicates the amount of data that can be
successfully transferred per unit time. For applications that require a large
number of sequential read/write operations, such as video editing and video on
demand (VoD) at TV stations, the throughput is more of a focus.
 Contents

1. Storage Basics
 Definition of Storage
 History of Storage
 Mainstream Disk Types
◼ Storage Networking Types
 Storage Forms

2. Key Storage Technologies

13 Huawei Confidential
Introduction to DAS
⚫ Direct attached storage (DAS)
⚫ Time: 1970s

Server
⚫ Background: Data explosion drove up huge

Fibre demand for storage. A simple storage

SAS SCSI
Channel
architecture, DAS, was then introduced.
Controller Controller Controller
⚫ Connection mode: Fibre Channel, SCSI, or SAS
⚫ Access mode: The connection channels between
DAS and server hosts often use SAS.
Disk array
⚫ Link rate: 3 Gbit/s、6 Gbit/s、12 Gbit/s
⚫ Provides functions, such as snapshot and
backup.

14 Huawei Confidential
 Introduction to NAS (1)
⚫ Network attached storage (NAS)
⚫ Time: early 1990s
NAS system architecture
Unix Linux Windows
⚫ Background: Developing networks drove the need
for large-scale data sharing and exchange, leading
to dedicated NAS storage devices.
NFS NFS CIFS ⚫ Access mode: Multiple front-end servers share
space on back-end NAS storage devices using CIFS
Dedicated IP or NFS Concurrent read and write operations can
storage network be performed on the same directory or file.
NFS and
CIFS File system
RAID ⚫ The file system is on the back-end storage device.

NAS storage device

15 Huawei Confidential

• Network File System (NFS) is an Internet standard protocol created by Sun

Microsystems in 1984 for file sharing between systems on a local area network
(LAN).

• Linux NFS clients support NFSv2 [RFC1094], NFSv3 [RFC1813], and NFSv4
[RFC3530]. NFSv2 that uses the User Datagram Protocol (UDP) is outdated due
to its limited data access and transmission capabilities.

▫ NFSv3, released in 1995, is widely used because Transmission Control

Protocol (TCP) is added for transmission.

▫ NFSv4, released in 2003, achieves better performance and security.

• NFS uses the Remote Procedure Call (RPC) protocol.

▫ RPC provides a set of operations to achieve remote file access that are not
restricted by machines, OSs, and lower-layer transmission protocols. It
allows remote clients to access storage over a network like accessing a local
file system.

▫ The NFS client sends an RPC request to the NFS server. The server transfers
the request to the local file access process, reads the local disk files on the
server, and returns the files to the client.

• Common Internet File System (CIFS) is a network file system protocol used for
sharing files and printers between machines on a network. CIFS is mainly used to
share network files between hosts running Windows.
 Introduction to NAS (2)
⚫ NAS supports the centralized management of scattered and independent data, facilitating
access to various hosts and application servers.

NAS
Ethernet port Ethernet port

NFS
File system
Volume management module Volume management module

LUN0 LUN1 LUN2 LUN3 LUN4 LUN5

LUN6 LUN7 LUN8 LUN9 LUN10 LUN11

16 Huawei Confidential

• NAS can serve as a network node and be directly connected to the network. In
theory, NAS can support various network technologies and topologies. As
Ethernet is the most popular network connection mode nowadays, we mainly
discuss the NAS environment on the Ethernet.

• NAS supports multiple protocols (such as NFS and CIFS) and supports various
OSs. Users can conveniently manage NAS devices by using Internet Explorer or
Netscape on any work station.
 Introduction to SAN
⚫ A storage area network (SAN) is a dedicated storage network that connects one or more
network storage devices to servers.

Clients

LAN

Servers

Ethernet switch Fibre Channel switch

Storage devices
17 Huawei Confidential

• A SAN is a high-performance and dedicated storage network used between

servers and storage resources. It is a back-end storage network independent from
a LAN. The SAN adopts a scalable network topology for connecting servers and
storage devices. The storage devices do not belong to any of the servers but can
be shared by all the servers on the network.

• The SAN that uses Fibre Channel Protocol (FCP) to set up connections between
servers and storage devices through Fibre Channel switches is called an FC SAN.
Fibre Channel is especially suitable for SANs, because it supports long-distance
and large-block transfer. The SAN mainly applies to high-end and enterprise-
class storage applications, which have demanding requirements for performance,
redundancy, and data availability.

• With the development of storage technologies, IP SANs based on TCP/IP also

gains popularity. IP SANs feature high scalability, flexible interworking, long-
distance data transmission, easy management and maintenance, and cost
advantages.

• The major difference between NAS and SAN is that NAS provides a file operation
and management system while SAN does not. SAN provides only data
management, which is the layer below file management. SAN and NAS do not
conflict with each other. They can coexist on the same network. However, NAS
implements storage space management and resource sharing through a public
interface, while SAN provides only a quick dedicate back-end channel for servers
to store data.
 Introduction to FC SAN
⚫ Fiber Channel storage area network (FC SAN)
⚫ Time: middle and late 1990s
Server ⚫ Background: To solve the poor scalability issue of
DAS, storage devices was networked. More than 100
servers can be connected in a network.
Fibre Channel ⚫ Connection mode: Fibre Channel link; Fibre Channel
switch switch
Fibre Channel
link ⚫ Access mode: The storage space on the back-end
FC SAN storage device can be divided into multiple LUNs.
Controller Each LUN belongs to only one front-end server.
⚫ Link rate: 2 Gbit/s, 4 Gbit/s, or 8 Gbit/s
⚫ Provides advanced data protection functions, such
as snapshot and disaster recovery.
Disk array

18 Huawei Confidential

• Fibre Channel (FC) is a standard data storage network used to transmit 100
Mbit/s to 4.25 Gbit/s signals over fiber or copper cables. It is a high-speed
transport technology used to build SANs. Fibre Channel is primarily used for
transporting SCSI traffic from servers to disk arrays, but it can also be used on
networks carrying ATM and IP traffic.
 Introduction to IP SAN
⚫ IP storage area network (IP SAN)
⚫ Time: 2001
⚫ Background: IP SAN is designed to solve the price and
Server management issues of the FC SAN.
⚫ Connection mode: Ethernet link; Ethernet switch
⚫ Access mode: The storage space on the back-end storage
Ethernet switch device can be divided into multiple LUNs. Each LUN
belongs to only one front-end server.
IP link ⚫ Link rate: 1 Gbit/s, or 10 Gbit/s
⚫ The IP SAN provides advanced data protection functions,
iSCSI storage controller such as snapshot and disaster recovery.
⚫ iSCSI is a mainstream choice because:
 Mature IP network management tools and
infrastructure can be used.
 IP networks are widely used, which can reduce a large
Disk array number of construction, management, and personnel
costs.

19 Huawei Confidential

• Internet Small Computer System Interface (iSCSI) is a storage technology based

on the Internet and SCSI-3 protocol. It transmits the SCSI protocol, originally used
only for local hosts, over the TCP/IP network to extend the connection distance.
In the following course, we will learn about the protocol encapsulation, working
principles, and application scenarios.
Comparison Between Storage Networking Types
SAN
DAS NAS
FC SAN IP SAN
Transmission SCSI, Fibre Channel,
IP Fibre Channel IP
mode and SAS

Data type Block-level File-level Block-level Block-level

Application
Any File servers Database applications Video security
scenario
High scalability and
Easy to understand; Easy to install; Strong scalability;
Advantage performance;
robust compatibility low cost low cost
high availability

Difficult management;
Low performance; Expensive and complex
limited scalability; low Low performance
Disadvantage inapplicable to some configuration; poor
storage space
applications networking compatibility
utilization

20 Huawei Confidential
 Contents

1. Storage Basics
 Definition of Storage
 History of Storage
 Mainstream Disk Types
 Storage Networking Types
◼ Storage Forms

2. Key Storage Technologies

21 Huawei Confidential
 Centralized Storage
⚫ A centralized storage system refers to one set of storage system consisting of multiple devices. Enterprises often
deploy their storage devices on a centralized environment. For example, the Huawei storage system may need
several cabinets to house devices. In terms of technical architectures, centralized storage is classified into SAN
(including FC SAN and IP SAN) and NAS storage.
⚫ Centralized storage has a simple deployment structure, which means you do not need to consider how to deploy
multiple nodes for a service, or the distributed collaboration between multiple nodes.

Server

Ethernet switch

IP link

iSCSI storage controller

Disk storage

22 Huawei Confidential

• Disadvantages of centralized storage:

▫ Isolated storage resources: Storage devices are connected to a limited

number of servers through a dedicated network.

▫ Scale-up by adding disk enclosures: The hardware controller performance

(a single controller with disks) becomes a bottleneck.

▫ Scale-out by connections between controllers: The hardware controller

performance becomes a bottleneck.

▫ No resource sharing: Storage devices and resources are provided by

different vendors, and resources cannot be shared among devices. Storage
pools are isolated in data centers.

▫ Centralized metadata management: The system concurrency is limited by

the metadata service performance. The metadata service becomes the
performance bottleneck.

• How to solve the capacity expansion and performance bottleneck issues of

traditional centralized storage?
 Distributed Storage
⚫ A distributed storage system stores data on multiple independent devices. It adopts a scalable system architecture
and enables multiple storage servers to share the storage load, improving scalability, reliability, availability, and
access efficiency. As distributed storage is becoming more popular, some applications requiring high performance,
such as databases of financial systems, also use distributed storage.

Server1 Server2 Server3

Volume1 Volume2 Volume3 Volume10 Volume11

P1 P2 Px
...
P1 P2 Py

Disk Disk Disk Disk Disk Disk

Resource pool 1 Resource pool 2
Host1 Host2 Host3 Host4 Host5 Host6

23 Huawei Confidential

• Distributed storage uses software to simulate the functions of the original

hardware controllers, avoiding the disadvantages of the hardware controllers.

• Resource pool: A resource pool is similar to a RAID group in SAN storage.

 Storage Service Type
iSCSI/FC NFS/CIFS HTTP/REST/S3
Protocol layer

Object
File Object
Object Object Data
system Key
system Object

Storage Metadata
layer
...
User-defined
metadata
... ...

Block storage File storage Object storage

⚫ Advantages: direct access, minimized ⚫ Advantages: easy management ⚫ Flat structure and nearly unlimited capacity expansion
overhead, and high efficiency and interconnection with
⚫ More intelligent self-management
applications
⚫ Disadvantages: high cost and poor ⚫ Disadvantages: limited scalability ⚫ Standard Internet protocol of the industry, cross-regional
scalability ⚫ Application scenarios: enterprises' transmission capability
⚫ Application scenarios: enterprise internal application integration ⚫ Application scenarios: Internet-oriented storage,
databases, such as Oracle and file sharing
archiving, and backup

24 Huawei Confidential

• Users can access data in an object storage as fast as in a SAN storage and can
share data as easy as in a NAS storage. Object storage has high reliability and
secure data sharing between platforms. The following describes the comparison
among the object storage, block storage, and file storage:

▫ Block storage directly accesses the storage layer, featuring fast speed,
minimum overhead, and maximum efficiency. However, block storage has
the high cost and poor scalability. Block storage employs iSCSI and Fibre
Channel. Therefore, it is difficult to transmit data across networks. Block
storage is applicable to enterprise databases, such as Oracle.

▫ File storage creates a file system on the basis of block storage. Data is
organized in the directory-directory-file mode, facilitating data
management. The objects operated by most application programs are files.
Therefore, file storage enables easier interworking with application systems.
File systems are restricted by directory trees. Therefore, a file system can be
typically expanded to dozens of PB at most. The scalability is limited. File
systems are applicable to application integration and file sharing in an
enterprise.
Storage Service Type
iSCSI/FC NFS/CIFS HTTP/REST/S3
Protocol layer

Object
File Object
Object Object Data
system Key
system Object

Storage Metadata
layer
...
User-defined
metadata
... ...

Block storage File storage Object storage

⚫ Advantages: direct access, minimized ⚫ Advantages: easy management ⚫ Flat structure and nearly unlimited capacity expansion
overhead, and high efficiency and interconnection with
⚫ More intelligent self-management
applications
⚫ Disadvantages: high cost and poor ⚫ Disadvantages: limited scalability ⚫ Standard Internet protocol of the industry, cross-regional
scalability ⚫ Application scenarios: enterprises' transmission capability
⚫ Application scenarios: enterprise internal application integration ⚫ Application scenarios: Internet-oriented storage,
databases, such as Oracle and file sharing
archiving, and backup

25 Huawei Confidential

▫ Object storage creates the object management layer above block storage.
Compared with the file system, the object system is flat with little
expansion limitation. An object consists of a unique key, file, data (file),
metadata, and user-defined metadata. An object contains self-management
information. Therefore, object storage is more intelligent. Using compatible
standard Internet protocol interfaces, object storage supports cross-region
transmission. Object storage applies to storage scenarios for Internet
services, and internal arching and backup scenarios for enterprises.
 Contents

1. Storage Basics

2. Key Storage Technologies

◼ RAID Technologies
 Storage Protocol

26 Huawei Confidential
What Is RAID?
⚫ Redundant Array of Independent Disks (RAID) combines multiple physical disks into one logical disk in
different ways, improving read/write performance and data security.

 RAID levels based on combination methods

RAID 0 Data striping, no parity

RAID 1 Data mirroring, no parity

RAID 3 Data striping, with dedicated parity

RAID 5 Data striping, with distributed parity

RAID 6 Data striping, with double distributed parity

 RAID levels by using two different RAID modes

RAID 0+1 Create RAID 0 and then RAID 1, providing data striping and
mirroring.

RAID 10 Similar to RAID 0+1. The difference is that RAID 1 is created

before RAID 0.

RAID 50 Create RAID 5 and then RAID 0, effectively improving the

performance of RAID 5.
27 Huawei Confidential
 RAID Data Distribution
⚫ Disk striping: Space in each disk is divided into multiple strips of a specific size. Written data
is also divided into blocks based on the strip size.
⚫ Strip: A strip consists of one or more consecutive sectors in a disk, and multiple strips form a
stripe.
⚫ Stripe: A stripe consists of strips of the same location or ID on multiple disks in the same
array.

Disk 1 Disk 2 Disk 3

D6 D7 D8 Stripe 2 Stripe depth

D3 D4 D5 Stripe 1

D0 D1 D2 Stripe 0

Data strips Data strips Data strips

in a disk in a disk in a disk

28 Huawei Confidential

• Stripe width

▫ Indicates the number of disks in an array for striping. For example, if a disk
array consists of three member disks, the stripe width is 3.

• Stripe depth

▫ Indicates the size of a stripe.

 RAID Data Protection
⚫ 1. Mirroring: Data copies are stored on another redundant disk.
⚫ 2. Parity check algorithm (XOR)
 XOR is widely used in digital electronics and computer science.
 XOR is a logical operation that outputs true only when inputs differ (one is true, the other
is false).
◼ 0 ⊕ 0 = 0, 0 ⊕ 1 = 1, 1 ⊕ 0= 1, 1 ⊕ 1 = 0

Physical disk 1 Physical disk 2 Parity disk

1 1 0
0 1 1
0 0 0

XOR for redundant backup

29 Huawei Confidential

• RAID generally protects data by the following methods:

▫ Stores data copies on a redundant disk to improve reliability and read

performance.

▫ Uses the parity check algorithm. Parity data is additional information

calculated using user data. For a RAID array that uses parity, an additional
parity disk is required. The XOR (symbol: ⊕) algorithm is used for parity.
 RAID Hot Spare and Reconstruction
⚫ Hot spare
 If a disk in a redundant RAID group is faulty, a functional backup disk in the RAID group can automatically replace the faulty
one to ensure RAID system redundancy.

⚫ Hot spare can be classified into the following types:

 Global: The spare disk is shared by all RAID groups in the system.
 Dedicated: The spare disk is used only by a specific RAID group in the system.

A0 Reconstruction XOR

Fault

Reconstruction
A0 A1 A2 P
Replacement
Data disk Data disk Data disk Parity disk

Hot spare
disk

30 Huawei Confidential

• Data parity: Redundant data is used to detect and rectify data errors. The
redundant data is usually calculated through Hamming check or XOR operations.
Data parity can greatly improve the reliability, performance, and error tolerance
of the disk arrays. However, the system needs to read data from multiple
locations, calculate, and compare data during the parity process, which affects
system performance.

• Generally, RAID cannot be used as an alternative to data backup. It cannot

prevent data loss caused by non-disk faults, such as viruses, man-made damages,
and accidental deletion. Data loss here refers to the loss of operating system, file
system, volume manager, or application system data, not the RAID data loss.
Therefore, data protection measures, such as data backup and disaster recovery,
are necessary. They are complementary to RAID, and can ensure data security
and prevent data loss at different layers.
 Common RAID Levels
RAID 0 RAID 1 RAID 5 RAID 10

D3 D2 D3 D3

D2 Logical D1 Logical D2 Logical D2

Logical
disk disk disk D1
D1 D0 D1 disk
D0 D0 D0
RAID 0

D2 D2
D2 D3 D1 D1 D2 P1 D3 D0 D1 D2 D3
D0 D1 D0 D1 P0 RAID 1 RAID 1 RAID 1 RAID 1
D0 D0

Physical Physical Physical Physical Physical Physical Physical

disk 1 disk 2 disk 1 disk 2 disk 1 disk 2 disk 3 D0' D0' D1' D1 ’ D2 ’ D2' D3 ’ D3'
D3'
Physical Physical Physical Physical
Data striping Mirroring Data parity disk 1 disk 2
…… disk 7 disk 8

Three key
technologies
31 Huawei Confidential

• RAID 0 consists of striping without fault tolerance. Data of the RAID group is
evenly distributed on all disks in stripe form.

• RAID 1, also called mirroring, can simultaneously write data into the primary disk
and mirror disk.

• RAID 3 consists of striping with dedicated parity. Data is striped on data disks,
and parity data is stored on a dedicated parity disk.

• RAID 5 is similar to RAID 3 except that parity information is evenly distributed

among data disks. RAID member disks store both the data and the parity
information, and data blocks and corresponding parity information are stored on
different disks. RAID 5 is one of the most commonly used RAID levels.

• RAID 10 combines mirroring and striping. The first level is RAID 1 mirrored pairs,
and the second level is RAID 0. RAID 10 is also a widely used RAID level.
 Working Principles of RAID 6 DP
⚫ Double parity (DP): In addition to the horizontal XOR parity disk used in RAID 4, it adds another disk to
store diagonal XOR parity data.
⚫ P0 to P3 on the horizontal parity disk are the parity information of horizontal data on all data disks.
 For example, P 0 = D 0 XOR D 1 XOR D 2 XOR D 3

⚫ DP 0 to DP 3 in the diagonal parity disk represent the diagonal parity data for respective data disks
and the horizontal parity disk.
 For example, DP 0 = D 0 XOR D 5 XOR D 10 XOR D 15

Physical Physical Physical Physical Horizontal Diagonal

disk 1 disk 2 disk 3 disk 4 parity disk parity disk
D0 D1 D2 D3 P0 DP0 Stripe 0
D4 D5 D6 D7 P1 DP1 Stripe 1

D8 D9 D10 D11 P2 DP2 Stripe 2

D12 D13 D14 D15 P3 DP3 Stripe 3

32 Huawei Confidential

• RAID 6 DP has two independent parity data blocks: horizontal parity data and
diagonal parity data.

• Parity values in the horizontal parity disk are also called parity check values,
which are obtained by performing the XOR operation on user data in the same
stripe.As shown in the following figure,P0 is obtained by performing an XOR
operation on D0, D1, D2, and D3 on a stripe 0, and P1 is obtained by performing
an XOR operation on D4, D5, D6, and D7 on a stripe 1. Therefore, P0 = D0 ⊕D1⊕
D2⊕D3, P1 = D4⊕D5⊕D6⊕D7, and so on.

• The diagonal parity uses the diagonal XOR operation to obtain the row-diagonal
parity data block. A process of selecting a data block is relatively complex. DP0 is
obtained by performing an exclusive OR operation on D0 on a stripe 0 of a hard
disk 1, D5 on a stripe 1 of a hard disk 2, D10 on a stripe 2 of a hard disk 3, and
D15 on a stripe 3 of a hard disk 4. DP1 is obtained by performing an exclusive OR
operation on D1 on a stripe 0 of a hard disk 2, D6 on a stripe 1 of a hard disk 3,
D11 on a stripe 2 of a hard disk 4, and P3 on a stripe 3 of a first parity hard disk.
DP2 is obtained by performing an exclusive OR operation on D2 on a stripe 0 of a
hard disk 3, D7 on a stripe 1 of a hard disk 4, P2 on a stripe 2 of an odd even
hard disk, and D12 on a stripe 3 of a hard disk 1. Therefore, DP0 =
D0⊕D5⊕D10⊕D15, DP1 = D1⊕D6⊕D11⊕P3, and so on.
 Working Principles of RAID 6 DP
⚫ Double parity (DP): In addition to the horizontal XOR parity disk used in RAID 4, it adds another disk to
store diagonal XOR parity data.
⚫ P0 to P3 on the horizontal parity disk are the parity information of horizontal data on all data disks.
 For example, P 0 = D 0 XOR D 1 XOR D 2 XOR D 3

⚫ DP 0 to DP 3 in the diagonal parity disk represent the diagonal parity data for respective data disks
and the horizontal parity disk.
 For example, DP 0 = D 0 XOR D 5 XOR D 10 XOR D 15

Physical Physical Physical Physical Horizontal Diagonal

disk 1 disk 2 disk 3 disk 4 parity disk parity disk
D0 D1 D2 D3 P0 DP0 Stripe 0
D4 D5 D6 D7 P1 DP1 Stripe 1

D8 D9 D10 D11 P2 DP2 Stripe 2

D12 D13 D14 D15 P3 DP3 Stripe 3

33 Huawei Confidential

• A RAID 6 array tolerates failures of up to two disks.

• Performance of a RAID 6 group: Dual-disk verification is used, and the

performance is relatively slow. Therefore, RAID 6 applies to the following two
scenarios:

▫ Data is critical and should be consistently online and available.

▫ The disk capacity is large (usually greater than 2 TB). The reconstruction of
a large-capacity disk takes a long time. Data will be inaccessible for a long
time if two disks fail at the same time. A RAID 6 array tolerates failure of
another disk during the reconstruction of one disk. Some enterprises want
to use a dual-redundancy RAID array for their large-capacity disks.
 Introduction to RAID 2.0
⚫ RAID 2.0
 RAID 2.0 is an enhanced RAID technology that effectively resolves the following problems: prolonged
reconstruction of an HDD, and data loss if a disk is faulty during the long reconstruction of a traditional RAID
group.
⚫ RAID 2.0+
 RAID 2.0+ provides smaller resource granularities (tens of KB) than RAID 2.0 to serve as the units of standard
allocation and reclamation of storage resources, similar to VMs in computing virtualization. This technology is
called virtual block technology.
⚫ Huawei RAID 2.0+
 Huawei RAID 2.0+ is a new RAID technology that overcomes traditional RAID issues. Huawei RAID 2.0+ evolves
in line with the storage architecture virtualization to implement two-layer virtualized management instead of
the traditional fixed management. Based on the underlying disk management that employs block
virtualization (Virtual for Disk), RAID 2.0+ uses Smart-series efficiency improvement software to implement
efficient resource management that features upper-layer virtualization (Virtual for Pool).

34 Huawei Confidential

• Block virtualization is to divide disks into multiple contiguous storage spaces of a

fixed size called a chunk (CK).
 RAID 2.0+ Block Virtualization
⚫ If data is not evenly stored on SSDs, some heavily loaded SSDs may become the
system bottleneck.
⚫ The storage system uses RAID 2.0+ for fine-grained division of SSDs to evenly
distribute data to all LUNs on each SSD and balance loads.

35 Huawei Confidential

1. Multiple SSDs form a storage pool.

2. Each SSD is then divided into CKs of a fixed size (typically 4 MB) for logical space
management.

3. CKs from different SSDs form chunk groups (CKGs) based on the RAID policy
specified on DeviceManager.

4. CKGs are further divided into grains (typically 8 KB). Grains are mapped to LUNs
for refined management of storage resources.

• RAID 2.0+ has the following advantages over traditional RAID:

• Balanced service loads for zero hotspots. Data is evenly distributed to all
SSDs in a storage resource pool, ensuring no SSD becomes a hotpot,
thereby lowering the SSD failure rate.

• Quick reconstruction for a lowered data loss risk. Faulty SSDs trigger data
reconstruction on all the other SSDs in the storage pool. This many-to-
many reconstruction is rapid and significantly reduces data vulnerability.

• All SSDs in a storage resource pool participate in reconstruction, and each

SSD only needs to reconstruct a small amount of data. Therefore, the
reconstruction process does not affect upper-layer applications.
 Contents

1. Storage Basics

2. Key Storage Technologies

 RAID Technologies
◼ Storage Protocol

36 Huawei Confidential
 SCSI
⚫ Small Computer System Interface (SCSI) is an interface technology developed for midrange
computers and used for connecting between hosts and peripheral devices.

I/O request
SCSI bus
Data/Address bus Host Disk

Host
Adapter C/S

SCSI ID 7 Control signal SCSI application layer SCSI application layer

Command/Data
SCSI array SCSI transport layer SCSI transport layer
SCSI array
ID 0 ID 5
Bus connection
LUN 0 LUN 1 LUN 0 SCSI interconnection SCSI interconnection
LUN 2 LUN 1 layer layer

37 Huawei Confidential

• Computers communicate with storage systems through buses. The bus is a path
through which data is transferred from the source device to the target device. To
put it simple, the high-speed cache of the controller functions as the source
device and transfers data to target disks, which serve as the target devices. The
controller sends a signal to the bus processor requesting to use the bus. After the
request is accepted, the controller's high-speed cache sends data. During this
process, the bus is occupied by the controller and other devices connected to the
same bus cannot use it. However, the bus processor can interrupt the data
transfer at any time and allow other devices to use the bus for operations of a
higher priority.

• A computer has numerous buses, which are like high-speed channels used for
transferring information and power from one place to another. For example, the
universal serial bus (USB) port is used to connect an MP3 player or digital
camera to a computer. The USB port is competent to the data transfer and
charging of portable electronic devices that store pictures and music. However,
the USB bus is incapable of supporting computers, servers, and many other
devices.
 SCSI
⚫ Small Computer System Interface (SCSI) is an interface technology developed for midrange
computers and used for connecting between hosts and peripheral devices.

I/O request
SCSI bus
Data/Address bus Host Disk

Host
Adapter C/S

SCSI ID 7 Control signal SCSI application layer SCSI application layer

Command/Data
SCSI array SCSI transport layer SCSI transport layer
SCSI array
ID 0 ID 5
Bus connection
LUN 0 LUN 1 LUN 0 SCSI interconnection SCSI interconnection
LUN 2 LUN 1 layer layer

38 Huawei Confidential

• In this case, SCSI buses are applicable. SCSI, short for Small Computer System
Interface, is an interface used to connect between hosts and peripheral devices
including disk drives, tape drives, CD-ROM drives, and scanners. Data operations
are implemented by SCSI controllers. Like a small CPU, the SCSI controller has its
own command set and cache. The special SCSI bus architecture can dynamically
allocate resources to tasks run by multiple devices in a computer. In this way,
multiple tasks can be processed at the same time.
 iSCSI
⚫ iSCSI encapsulates SCSI commands and block data into TCP packets and transmits the
packets over an IP network. iSCSI uses mature IP network technologies to implement and
extend SANs.
SCSI applications (such as file systems and databases)
SCSI block instruction SCSI flow instruction Other SCSI instructions

SCSI instruction, data and status

iSCSI

TCP

IP

Physical layer

39 Huawei Confidential

• The SCSI controller card is used to connect to multiple devices to form a network.
The devices can communicate with each other on the network but cannot be
shared on the Ethernet. If devices form a network through SCSI and the network
can be mounted to an Ethernet, the devices can interconnect and share with
other devices as network nodes. As a result, the iSCSI protocol evolved from SCSI.
The IP SAN using iSCSI converts user requests into SCSI codes and encapsulates
data into IP packets for transmission over the Ethernet.

• The iSCSI scheme was initiated by Cisco and IBM and then advocated by Adaptec,
Cisco, HP, IBM, Quantum, and other companies. iSCSI offers a way of transferring
data through TCP and saving data on SCSI devices. The iSCSI standard was
drafted in 2001 and submitted to IETF in 2002 after numerous arguments and
modifications. In Feb. 2003, the iSCSI standard was officially released. The iSCSI
technology inherits advantages of traditional technologies and develops based on
them. On one hand, SCSI technology is a storage standard widely applied by
storage devices including disks and tapes. It has been keeping a fast development
pace since 1986. On the other hand, TCP/IP is the most universal network
protocol and IP network infrastructure is mature. The two points provide a solid
foundation for iSCSI development.

• Prevalent IP networks allow data to be transferred over LANs, WANs, or the

Internet using new IP storage protocols. The iSCSI protocol is developed by this
philosophy. iSCSI adopts IP technical standards and converges SCSI and TCP/IP
protocols. Ethernet users can conveniently transfer and manage data with a small
investment.
 iSCSI Initiator and Target
⚫ Initiator
iSCSI
 The SCSI layer generates command descriptor blocks
Initiator Target
(CDBs) and transfers them to the iSCSI layer.
SCSI SCSI
 The iSCSI layer generates iSCSI protocol data units
(PDUs) and sends them to the target over an IP
iSCSI iSCSI
network.

⚫ Target TCP TCP

 The iSCSI layer receives PDUs and sends CDBs to the
SCSI layer. IP IP

 The SCSI layer interprets CDBs and gives responses

Link Link
when necessary.

40 Huawei Confidential

• The iSCSI communication system inherits some of SCSI's features. The iSCSI
communication involves an initiator that sends I/O requests and a target that
responds to the I/O requests and executes I/O operations. After a connection is
set up between the initiator and target, the target controls the entire process as
the primary device.

• There are three types of iSCSI initiators: software-based initiator driver,

hardware-based TCP offload engine (TOE) NIC, and iSCSI HBA. Their
performance increases in that order.

• iSCSI targets include iSCSI disk arrays and iSCSI tape libraries.

• The iSCSI protocol defines a set of naming and addressing methods for iSCSI
initiators and targets. All iSCSI nodes are identified by their iSCSI names. This
method distinguishes iSCSI names from host names.

• iSCSI uses iSCSI names to identify initiators and targets. Addresses change with
the relocation of initiator or target devices, but their names remain unchanged.
When setting up a connection, an initiator sends a request. After the target
receives the request, it checks whether the iSCSI name contained in the request is
consistent with that bound with the target. If the iSCSI names are consistent, the
connection is set up. Each iSCSI node has a unique iSCSI name. One iSCSI name
can be used in the connections from one initiator to multiple targets. Multiple
iSCSI names can be used in the connections from one target to multiple initiators.
Discussion:
⚫ We have learned the FC SAN and IP SAN. Now assume that two sites use different
networks FC SAN and TCP/IP. How can storage devices at the two sites
communicate with each other?

➢ To converge Fibre Channel and TCP?

41 Huawei Confidential
 Convergence of Fibre Channel and TCP
⚫ Ethernet technologies and Fibre Channel technologies are both developing fast. Therefore, it
is inevitable that IP SAN and FC SAN that are complementary coexist for a long time.
⚫ Fibre Channel over a TCP/IP network:
◼ iFCP
◼ FCoE

42 Huawei Confidential

• Fibre Channel over IP (FCIP) is an IETF proposed standard that defines the Fibre
Channel architecture over TCP/IP links. FCIP uses the current IP protocol and
facilities to connect the tunnels of two Fibre Channel SANs at different places.

• Internet Fibre Channel Protocol (iFCP) is a gateway-to-gateway protocol that

provides Fibre Channel communication services for optical devices on TCP/IP
networks. iFCP delivers congestion control, error detection, and recovery functions
through TCP. The purpose of iFCP is to enable current Fibre Channel devices to
interconnect and network at the line rate over an IP network. The frame address
conversion method defined in this protocol allows Fibre Channel storage devices
to be added to the IP-based network through transparent gateways.

• Fibre Channel over Ethernet (FCoE) transmits Fibre Channel signals over an
Ethernet, so that Fibre Channel data can be transmitted at the backbone layer of
a 10 Gbit/s Ethernet using the Fibre Channel protocol.

• IP over Fiber Channel (IPFC) uses the Fibre Channel connections between two
servers as IP data exchange media. To do this, IPFC defines how to transmit IP
packets over a Fibre Channel network. Like all other application protocols, IPFC is
implemented by a device driver in an operating system. The ifconfig or ipconfig
command is executed for local IP connections. Then the IPFC driver addresses the
Fibre Channel HBA. After that, IP packets can be transmitted through Fibre
Channel.
 iFCP
⚫ Internet Fibre Channel Protocol (iFCP) is a gateway-to-gateway protocol that provides Fibre
Channel communication services for optical devices on TCP/IP networks to implement end-
to-end IP connection.
iFCP gateway iFCP gateway

TCP/IP

Fibre Channel storage system

Fibre Channel storage system
43 Huawei Confidential

• iFCP is a gateway-to-gateway protocol that provides Fibre Channel

communication services for optical devices on TCP/IP networks to implement
end-to-end IP connection. Fibre Channel storage devices, HBAs, and switches can
directly connect to iFCP gateways. iFCP provides traffic control, error detection,
and error recovery through TCP. It enables Fibre Channel devices to interconnect
and network at the line rate over an IP network.

• The frame address conversion method defined in the iFCP protocol allows Fibre
Channel storage devices to be added to the TCP/IP-based network through
transparent gateways. iFCP can replace Fibre Channel to connect to and group
Fibre Channel devices using iFCP devices. However, iFCP does not support the
merge of independent SANs, and therefore a logical SAN cannot be formed. iFCP
outstands in supporting SAN interconnection as well as gateway zoning, allowing
fault isolation and breaking the limitations of point-to-point tunnels. In addition,
iFCP enables end-to-end connection between Fibre Channel devices. As a result,
the interruption of TCP connection affects only a communication pair. SANs that
adopt iFCP support fault isolation and security management, and deliver higher
reliability than SANs that adopt FCIP.
 iFCP Protocol Stack
⚫ iFCP is between Fibre Channel and TCP/IP, which means that iFCP can interwork with either
Fibre Channel or TCP/IP.

SCSI applications (such as file systems and databases)

SCSI block instruction SCSI flow instruction Other SCSI instructions

SCSI instruction, data and status

Fibre Channel

iFCP

TCP

IP

Physical layer

44 Huawei Confidential

• The main function of the iFCP protocol layer is to transport Fibre Channel frame
images between locally and remotely attached N_Ports. When transporting
frames to a remote N_Port, the iFCP layer encapsulates and routes the Fibre
Channel frame comprising each Fibre Channel information unit, and transmits
the frame via a predetermined TCP connection over the IP network.

• In the IP SAN that uses iFCP, iFCP devices take the place of Fibre Channel
switches, which means that iFCP switches can also function as Internet Storage
Name Servers (iSNSs) to provide the name discovery service for terminal nodes.
The iFCP switch allocates a 4-byte IP address to each Fibre Channel terminal
node. When a Fibre Channel device sends an SNS name query request, the
request is intercepted by the iFCP switch and interpreted by the iSNS server.
 FCoE
⚫ Fibre Channel over Ethernet (FCoE) allows the transmission of LAN and FC SAN data on the
same Ethernet link. This reduces the number of devices, cables, and network nodes in a data
center, as well as power consumption and cooling loads, simplifying management.
⚫ FCoE encapsulates FC data frames into Ethernet frames and allows service traffic on a LAN
and SAN to be transmitted over the same Ethernet.
Ethernet data link layer frame

Service flow IP address

Block storage FCoE

VoIP call

Video stream VoIP

45 Huawei Confidential

• FCoE offers standard Fibre Channel services, including discovery, global naming,
and zoning. These services run in the same way as the original Fibre Channel
services with low latency and high performance.

• From the perspective of Fibre Channel, FCoE enables Fibre Channel to be carried
by the Ethernet Layer 2 link. From the perspective of the Ethernet, FCoE is an
upper-layer protocol that the Ethernet carries, like IP or IPX.
 FCoE Protocol Encapsulation
⚫ FCoE encapsulates contents in the FC-2 and above layers into Ethernet packets for
transmission.

FC-4 FC-4
FC-3 FC-3 Fibre Channel protocol layers
FC-2 FC-2
FC-1 FCoE
FC-0 MAC
Ethernet protocol layers
PHY

46 Huawei Confidential

• The Fibre Channel protocol stack has five layers. FC-0 defines the medium type,
FC-1 defines the frame coding and decoding mode, FC-2 defines the frame
division protocol and flow control mechanism, FC-3 defines general services, and
FC-4 defines the mapping from upper-layer protocols to Fibre Channel.
Discussion:
⚫ What are the application scenarios of FCoE?
⚫ What are the application scenarios of iFCP?

47 Huawei Confidential
 Quiz

1. Which of the following statements about FC SAN are true?

A. Fibre Channel switches are required.

B. Ethernet switches are required.

C. Fibre Channel links cannot be used between storage devices.

D. Data packets comply with the Fibre Channel protocol stack.

2. The performance of SATA disks is better than that of SAS disks.

A. True

B. False

48 Huawei Confidential

Answers:

▫ AD

▫ B
 Summary

In this course, we covered:

⚫ Mainstream data storage modes and network topologies
⚫ RAID and Huawei RAID 2.0+ block virtualization technologies
⚫ Differences and relationships between centralized storage and distributed
storage
⚫ Storage protocols and their application scenarios

In the next course, we will learn the network technologies.

49 Huawei Confidential
 Recommendations

⚫ Huawei iLearning
 https://e.huawei.com/en/talent/#/
⚫ Huawei Support Case Library
 https://support.huawei.com/enterprise/en/knowledge?lang=en

50 Huawei Confidential
 Acronyms and Abbreviations
FC: Fibre Channel

FCIP: Fibre Channel over IP

FCoE: Fibre Channel over Ethernet

iFCP: Internet Fibre Channel Protocol

iSCSI: Internet Small Computer System Interface

IPFC: IP over Fiber Channel

51 Huawei Confidential
 Acronyms and Abbreviations
IOPS: Input/Output per second

MTBF: Mean Time Between Failure

NAS: Network Attached Storage

RAID: Redundant Array of Independent Disks

SAN: Storage Area Network

SCSI: Small Computer System Interface

52 Huawei Confidential
Thank you. 把数字世界带入每个人、每个家庭、
每个组织,构建万物互联的智能世界.
Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright© 2021 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.