OPENFLOW WITH MPLS IN SMART GRIDS

Aditee Patil

Mohammad Zohaib Yunus

Shweta Daheeval

Varun Sharma

Group 15

Advisor: Professor Conwell Dickey

University of Colorado at Boulder

TLEN 5710: Capstone

Professor David Reed

April 25, 2014

2

Abstract

In an era of technological advances, the demand from customers is intensifying in regards

to high bandwidth, better services, faster speed, more reliability, and lower costs. A rapid migration

from the access infrastructures to the Next Generation Networks (NGN) is required in order to

cope with the proliferating and escalating requirements. This research paper proposes a new smart

grid communications network using OpenFlow with Multiprotocol Label Switching (MPLS) that

provides a centralized control in Smart Grid Supervisory Control and Data Acquisition (SCADA)

systems, in addition to being cost-effective, more efficient, and interoperable with existing Time

Division Multiplexing (TDM) protocols and legacy applications. This research will investigate the

ways to evolve the smart grid communications network from TDM over Synchronous Digital

Hierarchy (SDH) or Synchronous Optical Network (SONET)-centric network to a smarter

OpenFlow based architecture that will provide benefits of reduced operational costs, increased

power quality, higher security, and greater reliability.

3

Table of Contents

1. Introduction Page

1.1. Statement of the Problem…………………………………………4

1.2. Research Question……………………………………………….. 4

1.2.1 Sub-Questions………………………………………….4

2. Literature Review………………………………………………………….5

3. Research Methodology………………………………………………….…8

4. Research Results…………………………………………………………..10

4.1. Resolution to Sub-Question No. 1 ……………………………….10

4.2. Resolution to Sub-Question No. 2 ……………………………….12

4.3. Resolution to Sub-Question No. 3 ……………………………….14

4.4. Resolution to Sub-Question No. 4 ……………………………….20

5. Discussion of Results………………………………………………………22

6. Conclusions and Future Research………………………………………...23

7. References…………………………………………………………………..24

4

1. Introduction

1.1 Statement of the Problem

Utility companies have been using decade old technologies in smart grid networks and are

clearly in need of new communication techniques. Most companies are still relying on point-to-

point radio wave links and leased lines for communication between the Supervisory Control and

Data Acquisition (SCADA) master control station and the remote substation. These technologies

do not provide adequate performance, security, and cost-effectiveness for the time critical control

signals from the substation. Utilities are clearly in need of new and advanced communication

networks that are not only more secure and offer better performance, but are also interoperable

with legacy devices and protocols in addition to being more cost-effective.

1.2 Research Question

Investigate the feasibility of OpenFlow being deployed with Multiprotocol Label

Switching (MPLS) in smart grid communication networks in order to reduce operational costs,

increase network performance, and promote innovation by adding infrastructure flexibility.

1.2.1 Sub-Questions

1. Explore the problems associated with the current smart grid network architecture and determine

how OpenFlow can provide a feasible solution. What are the current technologies being used in a

smart grid network for communication between the control station and substation? This research

will address the security and performance issues that plague the current infrastructure and research

on how OpenFlow can provide better performance, security, lower costs, and still be interoperable

with legacy protocols and devices?

2. Implement an MPLS based smart grid network and evaluate the performance between substation

and control station. Determine how the MPLS network will perform during a link failure and

5

evaluate the number of dropped packets when a link has failed and the packets are forced to take

a different route.

3. Can OpenFlow with MPLS support the existing applications and be interoperable with legacy

devices at the remote substations, in addition to providing better features and performance?

Implement a lab setup with conventional smart grid substation equipment and run DNP3 protocol

to check interoperability with OpenFlow.

4. How will OpenFlow prove to be a cost-efficient technology that will pave the way for new

network services to be added without service interruptions to utilities? What are the costs involved

in today’s smart grid network and how operational costs can be reduced using OpenFlow based

devices? Conduct an analysis and comparison of costs incurred in an OpenFlow based network

and a pure MPLS network for smart grids.

2. Literature Review

Smart grids could be pictured as a fusion of power and communication infrastructure.

Taking into account all key aspects to the functioning of smart grids, communication plays the

most important role. All these years, SCADA has evolved as a power communication system that

uses a multitude of transmission mediums such as wired (twisted pair, coaxial, and fiber optic) and

wireless (UHF, satellite, and microwave)[1]. SCADA system comprises of several Remote

Terminal Units (RTUs). These units gather information from field devices and transmit data to the

master station. The acquired data is then processed by the master station and appropriate control

signals are sent to the field devices to perform certain control tasks [2]. Distributed Network

Protocol (DNP3) is the telecommunications standard that defines communication between the

master stations and Remote Terminal Units (RTUs) [3].

6

In a research by Mak and Holland [6], the authors reviewed and recommended TCP/IP

over ethernet as a networking protocol specific to the electric utility employing SCADA systems.

The authors submitted a technical report explaining details of their proposal for “SCADA system

migration” to TCP/IP and Ethernet [6]. Leading power utility company ABB, attempted to

introduce a new design for smart grid networking known as the Smart Grid Transmission Protocol

(SGTP) as an alternative to Transport Control Protocol (TCP) [28]. The paper explored some of

the issues with the existing transport protocols that do not meet the demands of the smart grid

application. The authors discussed the new ‘SGTP’ protocol that could accomplish lower latency

along with maintaining reliability and inbuilt security mechanisms [28]. In another research, Cao

and Andonovic [7] investigated the selection criteria for wide area backbone communications and

proposed an MPLS Virtual Private Network (VPN) to achieve real-time data communication in

smart grid networks by implementing comprehensive computer simulation using a network

simulation tool called ‘OPNET’ to compute the performance in a changing load setup environment

[7]. MPLS based layer 3 technology was examined using OPNET, which offers a ‘graphic edit

interface’ and supports ‘object-oriented technology [7].’ It was found that MPLS VPN has added

functionalities such as Traffic Engineering (TE) and fast re-route mechanisms imparting lower

latency of the control signal, which is critical in most of the real time applications in smart grids

[7]. In a research by Qin [5], an “adversary model” for smart grids was discussed, where the author

analyzed attacks in an MPLS network due to corruption of the Label Distribution Protocol (LDP)

messages with the motive of impacting the Quality of Service (QoS) for certain real-time traffic in

SCADA systems [5]. Sydney et al. [20] demonstrated in their research how MPLS possesses

techniques for attaining efficient “overlay technologies” along with means to improving security

in smart grid networks [20]. The research examines how innovations are limited using MPLS

7

routing and switching technologies due to “features enclosed in the box” and states that there are

no empirical methods that have been developed for utilities to experiment and test new alternatives

to IP and MPLS [20].

Our MPLS based OpenFlow architecture for smart grids extends state of the art research

by providing utility operators the flexibility to add new services and applications at lower costs

without any impact to existing services. Our research aims to investigate the feasibility of

implementing OpenFlow with all the features of MPLS and show how it can perform better than

MPLS alone, thus being an excellent alternative to MPLS in smart grids.

In a classical scenario with routers and switches in the network, the control plane, which

makes the routing decisions and the data plane, which is responsible for data forwarding, reside

on the same device [8]. With OpenFlow, these functions are separated into different devices. The

data forwarding still takes place on the switch, but the control plane that makes all the routing

decisions resides on the controller, which can be a Linux based server or a computer. The

OpenFlow architecture is comprised of three main parts – (1) A flow table, which has a list of

different flows with their respective action to be taken by the switch, (2) A secure channel, which

is the interface between the switch and the controller that serves as a channel for exchange of

messages between the two devices, and (3) the OpenFlow protocol, an open standard protocol for

communication between controller and switch [8].

8

Fig. 2.3– OpenFlow architecture [8]

3. Research Methodology

A qualitative research methodology coupled with grounded theory was the directorial tool

towards approaching the research sub-problems. A variety of data sources, including quantitative

data, descriptive examination, review of records, lab experiments, observations, facts, and surveys

were used in order to aggregate and organize this paper.

The major challenge for us was to gauge how easily OpenFlow with MPLS could be

incorporated into the smart grid networks system over the existing architecture in order to avoid

having a complete overhaul of the communications system. We explored problems associated with

the current network design in smart grids by conducting a survey with one of the biggest utility

companies, ‘Xcel energy’. The parameters of the survey included the communication model,

technology, security issues, and current devices used in the company. Through our interaction, we

acquired knowledge about the current operations to gain first-hand experience.

We set up a lab to emulate a smart grid network and conducted performance tests. The lab

set-up was built using Cisco 3640 routers as the core MPLS cloud and the SEL 2411 Programmable

Logic Controller (PLC) [9] that served as a substation device sending control signals across the

9

network to a computer acting as a host in a control station. We did the setup at the University of

Colorado, Boulder’s state of the art telecommunications and energy communications laboratory.

Data was gathered based on pings with incrementing intervals from the host to the PLC and

observed the number of dropped packets when a link was failed and the packets were forced to

take an alternate route.

In order to examine if OpenFlow can support the existing legacy applications and co-exist

with MPLS in a smart grid network, we implemented a network using the SEL 2411 PLC and

made it poll to a host using DNP3 protocol via the OpenFlow enabled switch, which was a 48-port

Pica 8 P-3290 Open Virtual switch (OVS) [10]. We used an OpenFlow controller called ‘POX’

[11] on a computer running on the Ubuntu operating system. The OVS was configured to connect

to the controller and its ports were enabled for communication with the PLC. Upon successfully

testing the interoperability of legacy protocols and equipment with OpenFlow protocol, we further

decided to implement an OpenFlow with MPLS based solution for smart grids. Another OpenFlow

controller called ‘NOX’ [12] was used to run this network. For this task, we required four open

virtual switches, which would emulate a provider MPLS core. There was a constraint here as we

did not have four such switches available to experiment upon. We, therefore decided to use the

OVS emulator called ‘Mininet’ [13].

Operational and capital costs are very important to the utility company, and through our

research, we analyzed these costs and provided definitive reasons for utilities to upgrade to an

OpenFlow architecture. We found solutions for transmuting the existing network model so that an

OpenFlow with MPLS based network can be deployed on the existing smart grid communication

network to enhance the operational aspects of distribution and automation system. A well-

structured and highly organized team effort, along with impeccable designing and deep

10

understanding of the research question was required to achieve the results. All results are based on

different scenarios, taking into consideration all the facts, observations, actualities, annotations,

clarifications, interpretations, and opinions of all the team members.

4. Results

4.1 Explore the problems associated with the current smart grid network architecture and

determine how OpenFlow can provide a feasible solution.

Our team conducted a survey with distribution communication engineers, David Houston

and Richard Huck, at Xcel Energy to know about the current industry practices. This survey was

intended to explore issues with the current communication technologies used in smart grid

networks. According to the survey results, one of the issues encountered was an undefined network

infrastructure that would work in all scenarios and support real time critical applications. The

survey results show that microwave point to point radio system is being used by some utilities

creating the ‘line of sight’ problem. The survey also suggested that DNP3 is currently the standard

protocol used in electric utilities for communication between devices in the field substation and

control station. DNP3 was modeled to enhance the transmission of data and control information

between the control station and substation [3]. There is no single network infrastructure design

that works for everyone. Depending on the type of area: rural, urban, or terrain, the form of

communication varies. Devices out in the field such as breakers, reclosers, capacitors, banks, and

regulators [14] can be fitted with radios that allow them to connect to the backhaul network

allowing them to be controlled.

As per the survey results, utilities widely rely on MPLS infrastructure owned by a cloud

provider such as Alcatel Lucent [15]. Support for existing TDM services and provision for features

such as predictability and high availability are the attributes that utilities look for [16]. An MPLS

11

enabled router consists of both, the control plane as well as the data plane. Control plane is

responsible for making control decisions and the data plane is responsible for forwarding packets.

Holding both these functionalities in the same device complicates the functioning as it needs to

make routing decisions along with propagating routing and MPLS label information [17]. On the

contrary, OpenFlow technology can provide the same services with the MPLS data plane and a

separate control plane.

Fig. 4.1 - IP/MPLS versus MPLS/OpenFlow [14]

There are several reasons why we have chosen OpenFlow with MPLS over IP/MPLS as a proposed

technology for smart grids:

1. OpenFlow decouples the control and data plane, thus replacing the processor and memory

hogging MPLS routers with simple Open Virtual Switches (OVS) in the substation environment.

This would eliminate the need of specific high-end routers and also eradicate demands of a perfect

cooling system and cages in the smart grid atmosphere.

2. Frequent instability in the network also causes MPLS routers to recalculate paths and generate

packet drops and congestion. This in turn, may lead to drops in hello packets resulting in large

convergence time and routing loops [17].

12

3. Incorporating new services would be easier as there would not arise a need to upgrade any boxes

or tie new protocols with existing control plane protocols such as Resource Reservation Protocol

(RSVP) and (Open Shortest Path First) OSPF. Smart grid networks support many of the mission

critical applications and thus, loosing data or control signals is not acceptable. OpenFlow

overcomes these problems by connecting the MPLS data plane to the logical centralized controller.

4. Smart grid networks are required to offer the predictability, reliability, and high availability of

TDM networks that MPLS fails to impart in smart grids. OpenFlow accomplishes this by

distributing the centralized controller over several servers.

5. Lastly, high end MPLS enabled routers that are needed to be installed in the data path increase

the total cost of ownership. OpenFlow, on the contrary, uses simplified switches, thus making the

whole setup cost effective [17].

4.2 Implement an MPLS based smart grid network and evaluate the performance between

substation and control station.

We implemented a lab scenario where an MPLS cloud consisting of four Cisco 3640

routers was set up connecting to the host at the control station, which was a personal computer (IP:

10.10.10.2) and substation (IP: 10.10.10.3), which was a SEL 2411 PLC as shown in figure 4.2.

Fig. 4.2: MPLS Network configuration with 2411 Smart Grid Controller

13

The host machine was made to poll to the SEL 2411 PLC using DNP3 protocol and we

found that the DNP3 packet was encapsulated in the MPLS header as the packet traverses the

network. Figure 4.3 shows the Wireshark packet capture demonstrating the success of

implementing DNP3 in an MPLS based network.

Fig. 4.3: Wireshark packet capture demonstrating DNP3 network running between the

Control station (SEL 2411) and Sub-station (Workstation)

In order to evaluate the performance of MPLS in the smart grid communications network,

we generated Internet Control Message Protocol (ICMP) ping traffic from host (10.10.10.3) to the

PLC (10.10.10.2). For the initial phase, pings were transmitted at an interval of 1ms and the link

was failed between R1 and R3. We recorded the maximum number of packets dropped and iterated

the test over different ping intervals till 50ms.

14

Figure 4.4: MPLS fast re-route graph as ICMP ping packets are transmitted.

From this experiment, we successfully concluded that with MPLS, the number of packet drops is

considerably high and it could cause disruption in real time critical control signals that are

transmitted from the substation to the control station.

4.3 Can OpenFlow with MPLS support the existing applications and be interoperable with

legacy devices at the remote substations, in addition to providing better features and

performance?

In order to prove the working of DNP3 protocol over an OpenFlow based network, we

implemented a lab setup that emulated the smart grid network environment using an OpenFlow

(OF) controller and Open Virtual Switch (OVS). The controller used for this simulation is called

the ‘POX’ controller developed by Stanford University [11]. This is an open source controller

available to everyone. We used a Pica8 Open vSwitch that was provided to us by the University

of Colorado, Boulder. Our lab setup is shown in the figure below:

Fig 4.5. OpenFlow Network Topology

15

The OVS was configured according to our requirement. The following describes the configuration

steps:

1. Configure the switch to connect to the Open Flow (OF) controller, which has an IP address of

10.0.0.4 and communicates over port 6633.

2. Create a bridge on the switch and add ports ge1/1/4 and ge1/1/36 to the bridge. The port ge1/1/4

connects to the host computer, which acts as the control station in a smart grid network. The port

ge1/1/36 connects to the SEL 2411 Programmable Logic Controller (PLC).

We downloaded and installed the POX controller virtual machine in our computer. The

controller comes with a number of in-built modules that can provide several switching and routing

features. In this lab setup, we used the “forwarding.l2_learning” module that makes the OVS act

as a type of L2 learning switch [11]. The OF controller uses the OpenFlow protocol to install new

flows in the flow table, which come in from the PLC at the substation. A flow is defined as a

stream of packets with identical headers [4]. As flows arrive at the switch, they are checked against

the list of existing rules in the flow table. If a packet does not match any rule in the flow table, it

is transmitted over to the OF controller that evaluates the packet and installs a new rule for it that

specifies the action for the switch if it sees the same type of packet again.

As seen in the captures below taken from the controller, we see how new flows are being

installed as the controller learns them:

16

The figure 4.6 shows the communication between the substation PLC and the control station host.

The IOServer is a tool that polls between the PLC and the host every second and shows the live

communication. Here, we can see that there are packets transmitted and received through the OVS.

17

Fig. 4.6 – Capture of transmission from PLC to Host

We also captured the packets from the PLC to the host and OpenFlow messages between the

controller and the switch through the Wireshark Packet Capture software [18]. Figure 4.7 shows

the capture on the controller showing the exchange of OpenFlow messages between controller and

switch. Figure 4.8 shows the exchange of DNP3 messages between the PLC and host machine.

Fig 4.7 - OpenFlow packet capture

18

Fig. 4.8 – DNP3 packet capture

The above lab setup thus proved that DNP3 protocol can work over an OpenFlow based network.

To demonstrate an OpenFlow with MPLS based network, we created a new network topology with

four open virtual switches. Switches Sw1 and Sw4 act as provider core, while switches Sw2 and

Sw3 act as provider edge devices.

Fig. 4.9 - MPLS/ OpenFlow Network Topology

For demonstrating MPLS with OpenFlow, we use another controller called the ‘NOX’,

which is also developed by Stanford University [12]. Since we did not have four Open vSwitches

readily available to us, we decided to emulate the network topology on an open flow switch

emulator called Mininet [13]. We created a python script for our network topology that would run

19

on Mininet, which exists on the same Ubuntu virtual machine (VM) that the controller is installed

on. The OF controller along with Mininet would simulate the entire network.

We ran into an issue here, due to which we were not able to simulate the network

successfully. We were not able to bridge the PLC connected on the ethernet port of our computer

to the Mininet, which was running on the VM. Upon further analysis, we found that the SEL 2411

PLC devices do not interact with Mininet, which is isolated from any physical connections on the

computer.

With the above OpenFlow with MPLS network topology, we aimed to prove how the

OpenFlow implementation of MPLS can significantly lower the number of packets dropped during

a link failure where the MPLS fast reroute process comes into effect to find an alternate path to

the destination. However, upon further analysis, we can conclude that OpenFlow can significantly

reduce the packet drops in the network. In a normal MPLS-TE (MPLS with Traffic Engineering)

scenario, the Resource Reservation Protocol (RSVP) is used to establish multiple paths for LSPs

and resources are accordingly allocated for preferred paths [4]. As seen in our first experiment

with MPLS, we observed that 40 packets were dropped when 10ms pings were sent from the host

machine to the PLC. This high number of packet drops is due to the reason that the core routers

maintain a large number of Label Switches Paths (LSP) and any link failure can take significant

amount of re-signaling as the core routers calculate the next preferred path in RSVP [19].

However, in an OpenFlow scenario, the controller has already learnt every flow between

the source and destination. When a link fails and MPLS fast reroute process occurs, the controller

simply refers to the flow table to find an alternate path to the destination. The OVS does not have

to waste any processor memory to calculate a new route, and simply sends the packet out through

a different port. This would result in very few packets dropped across the network. We estimated

20

that it could be as low as 5 packet drops when 10ms pings are sent. This means that packet drops

are reduced by up to 80% in an OpenFlow based architecture. In smart grid SCADA systems, it is

important that the data and control traffic between substation and control station should not be

impacted by large packet loss because there are time critical applications that cannot tolerate any

loss of control signals during a link failure. Thus, OpenFlow with MPLS ensures reliable

communication of packets between the substation and control station.

4.4. How will OpenFlow prove to be a cost-efficient technology that will pave the way for new

network services to be added without service interruptions?

In the smart grid domain, conventional Wide Area Networks (WAN) prove to be an

expensive affair for utilities as well as service providers and they are unable to cope with the

rapidly growing requirements of cloud technologies and services along with advancements in other

technical areas. The complexity of networks has increased, and it has become increasingly difficult

for utilities to plan, manage, and operate services across all layers of the WAN. OpenFlow, in a

way, personifies what SDN (Software Defined Networking) promised to the world of networking,

that is, programmable, scalable, and efficient deployment and management of networks. With

traditional networks, integrating new services and technologies has become highly expensive and

is close to approaching a breaking point where traditional networks cannot cope with the demands

of such services and utility companies are not finding it economically feasible, thus resulting in

declining service margins. This will ultimately result in smart grid utility companies looking for

more economically viable options to make their services available to the consumers. OpenFlow

can address these issues and provide solutions to multiple challenges that are being faced by

utilities across various levels. It provides them with a plethora of options to deploy services that

have high bandwidth requirements and variable traffic densities [4, 20].

21

OpenFlow provides the following advantages for utility companies:

● The operational complexity is reduced which naturally results in lower operational costs.

As OpenFlow based SDN provides centralized programmable control to network services,

the cost otherwise spent on several hours required to configure multiple devices on

traditional networks is reduced considerably, thus enabling a reduction in network-

management costs.

● It provides better utilization of network resources that would eventually lead to more

economical delivery of services. Bandwidth and other resources allotted to services can be

easily and dynamically controlled for changing demands and requirements at various times

of the day. Thus, excess network resources are not allotted to services that do not require

them [4].

OpenFlow switches are cost effective when compared with traditional switches and routers

deployed in conventional networks. These switches are simple devices that forward data as per the

instructions fed by the controller. In contrast, traditional routers and switches have all the software

and hardware embedded in the device itself such as the CAM (Content Addressable Memory)

which they use to lookup forwarding tables. This amalgamation of software and hardware control

in a single device makes it a lot more expensive than OpenFlow switches. Such routers and

switches, naturally, also require more energy and power to fuel the running of multiple processing

components that are present in them, while OpenFlow switches consume a lot less power to operate

[4, 21].The longevity of regular routers and switches is limited to a period of three to four years

after which, they need to be replaced as they reach end of life and support.

22

5. Discussion of Results

This project explored the capabilities of OpenFlow technology to support the smart grid

network infrastructure. We investigated the problems associated with a MPLS design for smart

grids and found that the design does not offer good flexibility and predictability. In addition, we

concluded that the MPLS featuring special routers require a high use of energy resources and come

with expensive upgrade costs. We implemented a setup to run DNP3 over a MPLS network and

the link failure test showed the high drop in packets during a fast reroute process. The data gathered

was used to conclude that a pure MPLS network for smart grids is not a good choice for time

critical smart grid applications.

We further deployed DNP3 to run over an OpenFlow based network and could successful

poll between the PLC (substation) and the host computer (control station). However, when we

tried to deploy OpenFlow with MPLS, we faced hardware constraints due to which we could not

test the reroute feature in an OpenFlow based network. Nevertheless, we were successful in

proving that OpenFlow can be deployed in smart grid utilities, resulting in a highly scalable and

interoperable network. Lastly, we studied the economic benefits of deploying OpenFlow in smart

grids and concluded that with OpenFlow, we aim to reduce capital costs and complexity by

providing programmable control and solutions to most services. It also leads to lower energy

utilization thus resulting in reduced operational costs over the long run.

6. Conclusion and Future Research

Our capstone project successfully investigated the feasibility of deploying OpenFlow in

smart grids. We concluded that the current network for smart grid utilities are relying on old

network technologies that are not reliable and secure, thus requiring a new and advanced network

23

architecture that can help reduce costs for utilities and also have a highly available network. Our

experiment with MPLS in the lab showed the high drop in number of packets during a link failure

that could prove fatal for a utility company. We deployed OpenFlow in the network to prove that

it is interoperable with legacy protocols such as DNP3 and it is also supported on legacy PLC

devices such as SEL 2411. OpenFlow with MPLS can be deployed to provide all the features of

MPLS but with the advantage of a separate control plane so that the load on the router processor

is vastly reduced. This would also allow a centralized network management through the OpenFlow

controller and would not impact any existing services, thus providing an interruption free network.

We hope that our research will be the guiding light for further research that could be

conducted in the smart grids domain. Future researchers could build on the OpenFlow controller

program and add additional features such as Quality of Service (QoS) for differentiated traffic and

also include comprehensive service level agreements (SLAs) to ensure maximum availability. It

would be interesting for researchers to design a more efficient code that could take comparatively

less time to process and thus, decisions made by the controller with respect to routing and

congestion avoidance could be quicker, avoiding packet loss and drop in performance. In addition,

further research can be done on evaluating the performance of a larger network topology with

multiple devices and flood the network with heavy traffic to study the performance characteristics

of OpenFlow networks.

24

References

[1] W. Jewell, V. Namboodiri, V. Aravinthan, and B. Karimi, “Communication

Requirements and Integration Options for Smart Grid Deployment,” Texas A&M

University, Texas, Rep. PSERC Publication 12-03, Apr. 2004.

[2] G. Clarke and D. Reynders, “Practical Modern SCADA Protocols” Great Britain,

Newnes Publications, 2004.

[3] K. Curtis (2004, May 20). A DNP3 Protocol Primer [Online]. Available:

http://www.dnp.org/aboutus/dnp3%20primer%20rev%20a.pdf

[4] A. Sydney, “The Evaluation of Software Defined Networking for Communication and

Control of Cyber Physical Systems,” Ph.D. dissertation, Department of Electrical and

Computer Engineering, Kansas State University, Kansas, USA, 2013.

[5] Z. Qin. (2013, Dec. 20). A Survey of Networking Issues in Smart Grid [Online].

Available: http://www1.cse.wustl.edu/~jain/cse570-13/ftp/smrtgrid.pdf

[6] K. H. Mak and B. L. Holland, “Migrating electrical power network SCADA systems to

TCP/IP and Ethernet networking,” Power Eng. J., vol. 16, no. 6, pp. 305–311, Dec. 2002.

[7] D. Cao and I. Andonovic, “Research on Backbone Communication Network in

Smart Grid by using OPNET,” IEEE, 2011.

[8] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S.

Shenker, and J. Turner. (2008, March 14). OpenFlow: Enabling Innovation in Campus

Networks. [Online]. Available: http://archive.openflow.org/documents/openflow-wp-

latest.pdf

[9] (2014, April 24). SEL-2411 Programmable Automation Controller. [Online]. Available:

https://www.selinc.com/SEL-2411/

25

[10] (2013). Pica8 Open Networking. [Online]. Available:

http://www.pica8.com/documents/pica8-datasheet-48x1gbe-p3290-p3295.pdf

[11] (2014, April 20). POX Wiki - Open Networking Lab - Confluence. [Online]. Available:

https://openflow.stanford.edu/display/ONL/POX+Wiki

[12] (2014, April 20). MPLS-TE Demo - OpenFlow Wiki. [Online]. Available:

http://archive.openflow.org/wk/index.php/MPLS-TE_Demo.

[13] (2014, April 20). Mininet Overview - Mininet. [Online]. Available:

http://mininet.org/overview/

[14] H. L. Willis and R. R. Schrieber, Aging Power Delivery Infrastructures, Second ed.,

Dekker, April 16, 2007.

[15] Deploying IP/MPLS Communications Networks for Smart Grids. N.d.

http://www.tmcnet.com/tmc/whitepapers/documents/whitepapers/2013/7585-alcatel-

lucent-deploying-ipmpls-communications-networks-smart-grids.pdf, accessed December

1, 2013.

[16] Alcatel-Lucent Why an IP/MPLS Network Makes Sense for Smart Grids | TechZine |

N.d. http://www2.alcatel-lucent.com/techzine/why-an-ipmpls-network-makes-sense-for-

smart-grids/, accessed December 1, 2013

[17] S. Das, A. R. Sharafat, G. Parulkar, and N. McKeown, “MPLS with a simple OPEN

control plane,” in Optical Fiber Communication Conference and Exposition and National

Fiber Optic Engineers Conference (OFC/NFOEC 2011), Technical Digest (CD) (Optical

Society of America, 2011), paper OWP2.

[18] (2014, April 20). Wireshark. [Online]. Available: http://www.wireshark.org/

26

[19] D. Narula, M. Rojasmartine, and V. Rayipati. (May 2010). “Evaluating performance on

an ISP MPLS network,” MS Capstone, Interdisciplinary Telecommunications Program,

University of Colorado, Boulder, USA.

[20] A. Sydney, J. Nutaro, C. Scoglio, D. Gruenbacher, and N. Schulz, “Simulative

Comparison of Multiprotocol Label Switching and OpenFlow Network Technologies for

Transmission Operations,” Vol. 4, No. 2 ed. , IEEE Transactions on Smart Grid, June

2013.

[21] Open Networking Foundation (ONF), “OpenFlow-Enabled Cloud Backbone Networks

Create Global Provider Data Centers,” ONF Solution Brief ed., Open Networking

Foundation, November 14, 2012.

[22] S. Tanterdtid and A. Pao-on, "IP/MPLS-based data communication network for

power utility,” Proc. on IEEE Power Tech Conference, 2003 Bologna, Italy vol 3, pp:5,

June 2003.

[23] Quality of Service Networking – DocWiki. [Online]. Available:

http://docwiki.cisco.com/wiki/Quality_of_Service_Networking.[N.d.]

[24] Benefits of MPLS - The Top 4 MPLS Benefits. Available:

http://www.rcrwireless.com/mobile-backhaul/mpls-benefits.html [N.d]

[25] Industry Solutions - Cisco Systems Utilities/Smart Grid. [Online]. Available:

Internet: http://www.cisco.com/web/strategy/energy/external_utilities.html [N.d.]

[26] (2003, May). Multi-Protocol Label Switching (MPLS) Using MPLS to Build an

Application-Centric Network. [Online]. Available:

http://www.fourdtech.com/downloads/mpls_indepth_analysis.pdf

27

[27] (2014, 20 April). Next Generation Networks. [Online]. Available:

http://www.etsi.org/technologies-clusters/technologies/next-generation-networks

[28] (2014, 20 April). SCADA over IP-based LAN-WAN connections Application. [Online].

Available:

http://www05.abb.com/global/scot/scot221.nsf/veritydisplay/09e2909e92d2ce8ac125786

3004e7da0/$file/scada%20application%20flyer_small.pdf