6891 HowSDN CG 20180720 Web2
6891 HowSDN CG 20180720 Web2
in OT Networks
Colin Gray
Schweitzer Engineering Laboratories, Inc.
Presented at
IEEE ROC&C 2018/2019
Acapulco, Mexico
March 6–8, 2019
Colin Gray
Schweitzer Engineering Laboratories, Inc.
3 Hampstead Way
Hamilton, New Zealand
Email: papers@selinc.com
Abstract—With the business need for interconnectivity between information technology (IT) and
operational technology (OT) networks becoming more prevalent, cybersecurity is now a critical aspect of
how that interconnectivity is achieved. This paper discusses the differences between implementing
cybersecurity measures in IT and OT networks, introduces software-defined networking (SDN) technology,
and describes how SDN can be used to build a robust and secure OT network. This paper is intended for
engineers interested in secure network design strategies and provides a platform for discussion about what
SDN is and how it can be implemented in an OT network to meet cybersecurity, performance, and
management needs.
I. INTRODUCTION
In the last decade, significant advancements have been made in how industrial control systems (ICSs) monitor,
control, and manage an organization’s infrastructure. Implementation of substation-hardened Ethernet local-area
networks (LANs) has paved the way for more precise and sophisticated tools for retrieving and analyzing critical
real-time system information. The advanced capabilities of today’s modern digital measurement and control
intelligent electronic devices (IEDs), coupled with the implementation of Ethernet access directly from the device,
has opened the door for Transmission Control Protocol/Internet Protocol (TCP/IP) services to access an
unprecedented amount of data from these devices.
IEC 61850 provides high-speed peer-to-peer communications between IEDs, enabling faster and more accurate
control and restoration in the event of network disturbances. IEEE C37.118 synchrophasors and IEC 61850-9-2
Sampled Values provide high-speed sampling of real-time data directly from the secondary source with
millisecond accuracy.
While this information has traditionally been firmly in the domain of the operational technology (OT) network,
a wider audience has seen the need to bridge the gap between OT and information technology (IT) networks.
While IEC 61850 and other Ethernet-based supervisory control and data acquisition (SCADA) protocols operate
at the machine-to-machine (M2M) level, a large portion of network activity now involves person-to-machine
(P2M) interconnectivity [1].
The demand for access to these data is skyrocketing as more organizational departments use an array of services
for a variety of purposes, such as for analyzing post-event disturbance records, accessing metering data, and
gaining secure access directly to a device for diagnostics and troubleshooting to ensure that the highest possible
service availability is maintained. This, of course, inevitably raises the specter of cybersecurity and how the
traditional Ethernet network deals with securing critical infrastructure information. The fifth version of the North
American Electric Reliability Corporation Critical Infrastructure Protection (NERC CIP) reliability standards
(known as CIP Version 5) outlines a set of requirements for securing the traditional LAN from unwanted intrusions
and attacks. The impact of such attacks has been well documented in the past.
Recent research has shown that software-defined networking (SDN) can provide far better packet delivery
performance, greater cybersecurity, and greater centralized seamless control than can a traditional Ethernet LAN.
SDN has revolutionized the way IT system managers program their networks to keep up with a vastly diverse
range of users and applications. This paper introduces SDN and how it can be implemented into an OT network
to provide a greater level of security and performance.
Router
Manag ed Manag ed Ope nFlow Data P lane
Etherne t Switch Etherne t Switch
SDN S witch SDN S witch
Manag ed Manag ed
Etherne t Switch Etherne t Switch SDN S witch SDN S witch
SDN, on the other hand, implements a logical separation of the control plane and data plane, as shown in
Fig. 1(b). It does this by removing the control plane logic from the switch and introducing a network controller,
which determines the data flow criteria and defines a set of control plane flow rules that are sent to the SDN
switch. The SDN switch then simply executes the flow rule as predetermined by the controller [3]. SDN gives
system administrators much greater control over how specific data are forwarded around the network and allows
network configuration to be designed to suit the data exchange needs of the application. An SDN network,
therefore, does not need to rely on STAs and RSTP to determine a best-path connection between two devices or
wait for an RSTP reconvergence decision after a failure in the network before re-establishing a new data
connection. This means that SDN can program more than one active connection to an IED at the same time and
create redundant data paths in the same network.
SDN uses the OpenFlow protocol as the interface between the network action required for data packets (in the
control plane) and the hardware that carries out that action (in the data plane). OpenFlow passes flow rules from
an OpenFlow controller to OpenFlow-enabled SDN hardware. The traditional network switch must make its own
decision on how data flows based on network conditions. Because flow rules have been predetermined in the data
plane OpenFlow-enabled hardware, there is no need for the data plane hardware to make these decisions. The
network simply follows its flow table match rules to decide what to do next.
Purpose-engineered network control has significant advantages over IT traffic reactive control. Generally, an
IT middlebox will be plug-and-play, which is great for ease of installation because the system manager does not
have to configure many parameters. However, it makes network changes more complex because the ultimate
decision about which path a packet will take is determined by the STA and RTSP. Plug-and-play networks always
attempt to deliver a packet to its intended destination. In many cases, such as when using Address Resolution
Protocol (ARP), the networks also attempt to resolve the packet query by broadcasting to all hosts in the subnet.
This practice has obvious security ramifications.
Purpose engineering gives the system engineer complete control over the desired path a packet should take
during normal and abnormal network conditions. While some argue that purpose engineering is inherently more
difficult to configure because of the deny-by-default nature of the network, it is the only way we can be sure the
critical data in networks can be delivered to the intended destinations no matter what state the network is in.
Enterpr ise
Networ k
IT
OT
LEV EL 6
M H H LEV EL 5
Firewall/VPN
Perimeter
People
Switch
LEV EL 4
H H L
SCADA HMI PC
LEV EL 3
H M L Acces s Firewall
TDM
TDM Device Spa ns
LEV EL 2 Levels 1–4
L H M Aut om ation Switch
Switch Spa ns
Relay Levels 2–3 Comm Pro c
LEV EL 0
Physical
NA NA H 52
Cyber attackers are always looking to exploit vulnerabilities in an STA network. Media access control (MAC)
table flooding attacks can be used to force a switch to refresh its MAC table, which causes a significant portion
of the incoming frames to be flooded out all ports, thus creating an opportunity to capture data that reveals
legitimate MAC addresses. ARP spoofing (in which attackers attempt to redirect traffic to their IP address by
sending ARP messages to the network) or BPDU spoofing (in which unauthorized BPDU frames are sent into a
network to alter how packets are forwarded through the network) are both common techniques of intercepting
packets for the launching of denial-of-service (DOS) or man-in-the-middle attacks [3].
In the case of the MAC table flooding attack, mitigation is achieved by MAC-locked port technology. ARP
spoofing can be mitigated with static ARP tables so that hosts do not need to reply to ARP requests. However, in
both instances, maintaining a network that periodically requires new or replacement hardware requires additional
resources and funds to unlock and relock MAC addresses and manually update ARP tables. BPDU spoofing
requires port monitoring and disabling and BPDU filtering so that any BPDU packets arriving at the port are
dropped.
By default, STA switches forward all packets when more information is required so that security relies on the
quality of packet filtering. This is required for every individual switch in a network, resulting in more
administrative overhead to continually monitor, patch, and maintain the highest level of security.
The deny-by-default architecture of SDN technology ensures any unauthorized packets are dropped and only
packets that match proactive traffic-engineered flows are forwarded. MAC table flooding and BPDU spoofing
will not work in SDN because SDN switches do not have MAC tables or use BPDU packets. ARP spoofing is
overcome by engineering ARP requests and responses between devices.
The same concept of security levels can be implemented in SDN in a much simpler fashion. Fig. 3 shows the
implementation of security zones based on the collection of devices required to interact within that zone. This
keeps each zone or region isolated with trust established between all the devices within that zone. SDN flow rules
to apply simple techniques (such as MAC and IP address masking for traffic external to the security zone and
blocking of all multicast and broadcast traffic outside the security zone) provide a powerful tool to manage the
security needs of an ICS network.
Security Zone 1 Security Zone 2
SCADA
HMI RTU Master
Zone Zone
Ingress Ingress
SDN S witch SDN S witch
Encrypted Tu nnel
SDN S witch SDN S witch
IED IED
Historia n
With available OT SDN cybersecurity software, the security state of the network can be managed through
policy implementations that can be changed as the security state of the network changes. This provides security
orchestration for the flow controller through complete network visibility, programmable security zones, situational
awareness, and security policy management through whitelisting all devices on the network. Fig. 4 details the
architecture of third-party applications that are being developed to provide SDN-based cybersecurity policies and
measures at the enterprise level.
Reconnaissance
(network map ping;
OS fingerp rinting) Deliver Control Maintain
With SDN flow matches and conditions, we can be more selective with the packets needing further inspection
in a switch and redirect them to the appropriate analysis device. A DPI application requires all packets to be
rerouted to the DPI analysis device and then possibly return to the originating switch. The application may only
be interested in HTTP traffic, for instance, and lets all other traffic traverse the network without additional
scrutiny. SDN allows packet identifier tags to be added so that the packet can be traced back to the switch where
it originally entered the network, which is valuable to know when the packet is rerouted to the DPI device and is
required if the DPI return route is needed.
SDN switches that allow the network engineer to add or remove VLAN IDs determine the switch by which a
packet first entered the network. When packets already have a VLAN ID, bit masking can be used in the VLAN
field to identify the packets that have been previously marked for inspection and need to be routed to the analysis
device. This combination of bit masking and VLAN ID assigning ensures that any packets that were previously
sniffed and tagged in a downstream switch are forwarded, will remain isolated from other traffic, and will be
ensured a unique VLAN ID as they enter each switch [5]. This also ensures that if the packet is not dropped by
the DPI analysis device, it can be forwarded to the original destination if required.
An SDN system assumes that all packets have a pre-engineered match in a flow table and therefore any packet
that causes a table miss is either not required or may be an intruder attempting to access the network. For an IDS
and port-mirroring analysis, the packet is simply duplicated and sent to its original destination and to a predefined
switch port for forwarding to the DPI analysis device.
When SDN packet sniffing is used in conjunction with analytical tools that can respond to the detection of an
attacker (who is attempting to map the network through ARP scanning or to identify the operating system (OS)
of a network through OS fingerprinting techniques), it becomes very difficult for the attacker to move to the
weaponize and deliver phase. These responses could be in the form of false IP addresses in ARP replies or OS
information preventing the attacker from mapping a network or identifying the OS.
VIII. CONCLUSION
Through control plane abstraction, SDN technology removes the restrictions of STA, both in physical network
design and operational performance. ICS purpose-engineered networks are redefining the criteria for the creation
of new best-known methods of delivering information between services and applications. They are doing this
while still improving system performance in technical, policy, and procedural ways without breaking the existing
interoperability with Ethernet.
SDN allows network engineers to support even the most demanding applications used to control, operate, and
monitor critical infrastructure in an ICS network. Centralized monitoring and change control services reduce the
risk of network disruption. The traditional extended wait times for an STA network to converge have become a
thing of the past since SDN switches are preprogramed to forward packets in such events.
Although not a completely lossless technology, as packets could still be lost in transit if they are on a link or
in a switch that fails or if a port buffer is overrun, OT SDN is effectively a next ingress packet-healing technology.
SDN clearly provides significant advantages in cybersecurity over STA, with its core architecture complementing
the ICS core attributes and predictable whitelisting technologies. NERC CIP compliance can be achieved with
near real-time centralized reporting and monitoring of port activity, traffic volumes, and application status without
the need to manually deploy engineers to gather and analyze these data.
SDN challenges the best-known practices for STA network architectures, router design, combined control
plane and data plane switch design, and the services they use. The proactive engineering design of SDN provides
DPI and an IDS at every switch while simultaneously eliminating the services that increase the attack surface in
the STA network. MAC tables and BPDU packets are not used in SDN, and ARP spoofing through broadcast and
multicast blocking can be eliminated. In fact, the whole concept of what a subnet is and how IP addresses are
grouped can be challenged with flow rules that manage packet flow based on any layer of packet contents.
The exposure to attack and the recovery from a detected threat or intrusion is vastly reduced with the evolution
of third-party OT-focused SDN management platforms. Coupled with SDN multilayer capabilities, management
platforms that provide corporate-level threat models, security zones, and policies allow network operators to have
preplanned contingencies for mitigation of a threat detection. As the future possibilities for OT SDN development
and design continue to evolve and a greater understanding of the performance and security aspects of SDN are
further understood by asset owners, SDN will become the go-to architecture for making exciting changes in ICS
networks.
IX. REFERENCES
[1] D. Dolezilek, C. Gordon, and D. Anderson, “Applying New Technology to Improve the Performance of Teleprotection
Communications,” proceedings of the 13th International Conference on Developments in Power System Protection, Edinburgh,
United Kingdom, March 2016.
[2] R. Hill and R. Smith, “Purpose-Engineered, Active-Defense Cybersecurity for Industrial Control Systems,” August 2017. Available:
https://www.selinc.com.
[3] M. Hadley, D. Nicol, and R. Smith, “Software-Defined Networking Redefines Performance for Ethernet Control Systems,” proceedings
of the Power and Energy Automation Conference, Spokane, WA, March 2017.
[4] C. Gray, “An Exercise in Trust: Examining the Future of Cryptographic Technology in Electrical Control Systems,” proceedings of the
South East Asia Protection, Automation and Control Conference, Melbourne, Australia, March 2017.
[5] J. Dearien, “Implementing Packet Sniffing (IDS, DPI, Port Mirroring, etc.) in an SDN Network,” SEL Application Guide (AG2017-18),
2017. Available: https://www.selinc.com.