IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO.

5, SEPTEMBER 2014 2539

Energy-Storage-Based Low-Frequency Oscillation

Damping Control Using Particle Swarm Optimization
and Heuristic Dynamic Programming
Xianchao Sui, Yufei Tang, Student Member, IEEE, Haibo He, Senior Member, IEEE, and Jinyu Wen, Member, IEEE

Abstract—Low-frequency oscillation is one of the main barriers broader perspective, such power outage events underscore the
limiting power transmission between two connected power sys- complex issues associated with the generation and use of elec-
tems. Although power system stabilizers (PSSs) have been proved tricity: the reliability of the grid, the increased deployment of re-
to be effective in damping inner-area oscillation, inter-area oscil-
lation still remains a critical challenge in today’s power systems. newable energy, and the development of electric vehicles (EVs)
Since the low-frequency oscillation between two connected power to decrease dependence on traditional resources [1]. Among the
systems is active power oscillation, power modulation through efforts to address these problems, recent development of en-
energy storage devices (ESDs) can be an efficient and effective ergy storage devices (ESDs) offers a well-established approach
way to maintain such power system stability. In this paper, we to improve grid reliability and utilization. While the transmis-
investigate the integration of a new goal representation heuristic
dynamic programming (GrHDP) algorithm to adaptively control sion and distribution systems are responsible for moving elec-
ESD to damp inter-area oscillation. A particle swarm opti- tricity over distances to end users, an ESD system involves a
mization (PSO)-based power oscillation damper (POD) has also time dimension to provide electricity when it is needed and in-
been proposed for comparison. Various simulation studies with crease the power system operation and control margin. A re-
residue-based POD controller design, the proposed PSO optimized cent EPRI study identified a number of high-value opportunities
controller design, and the GrHDP-based controller design over
a four-machine-two-area benchmark power system with energy for energy storage, including wholesale energy services, inte-
storage device have been conducted. Simulation results have gration of renewables, commercial and industrial power quality
demonstrated the efficiency and effectiveness of the GrHDP-based and reliability, transportable systems for transmission and distri-
approach for inter-area oscillation damping in a wide range of bution, and grid management [2]. In this paper, we focus on the
system operating conditions. ESD-based controller design for power system damping control.
Index Terms—Energy storage device (ESD), goal representation In traditional power system stability controller design, such
heuristic dynamic programming (GrHDP), particle swarm opti- as power system stabilizers (PSSs), a linearized power system
mization (PSO), power oscillation damper (POD), power system model near the operating point is used [3]. However, we need to
stability.
relax this assumption as modern power systems become more
and more nonlinear, time-variant, and uncertain with the con-
I. INTRODUCTION tinuously increased deployment of flexible alternating current
transmission system (FACTS), renewable energy, and EVs. As

T HE August 2003 blackout in the northeast United States system state parameters and operating conditions are changing,
and the July 2012 India blackout that affected over 620 power system modeling becomes a very complex and time-
million people are two of the widely publicized examples in consuming task for the electrical engineers and operators. In
which power outages affected many millions of users. From a such a situation, two major drawbacks of the traditional con-
trol methods are the lack of robustness and online learning ca-
pability. Meanwhile, as an inherent phenomenon, inter-area os-
Manuscript received October 10, 2013; revised January 31, 2014; accepted
February 03, 2014. Date of publication March 03, 2014; date of current ver- cillation in connected power systems is mainly due to the dy-
sion August 15, 2014. This work was supported in part by the National Sci- namic power imbalance between synchronous machines caused
ence Foundation (NSF) under Grant ECCS 1053717, the Army Research Of-
by disturbances, and, for most of the cases, this imbalance be-
fice under Grant W911NF-12-1-0378, NSF-DFG Collaborative Research on
“Autonomous Learning” (a supplement grant to CNS 1117314), and the Na- haves as low-frequency oscillation (0.1 Hz to 0.8 Hz). In recent
tional Natural Science Foundation of China under Grant 51228701. Paper no. years, high-voltage direct current (HVDC) devices and FACTS
TPWRS-01299-2013.
have been adopted for inter-area oscillation damping control [4].
X. Sui is with Dalian Power Supply Company, Dalian 116033, China. (e-mail:
xianchaosui@gmail.com). However, PSSs are still the first choice for the suppression of
Y. Tang and H. He are with the Department of Electrical, Computer and low-frequency oscillation. A PSS provides supplementary con-
Biomedical Engineering, University of Rhode Island, Kingston, RI 02881 USA
trol signal to an excitation system of synchronous machines,
(e-mail: ytang@ele.uri.edu; he@ele.uri.edu).
J. Wen is with the College of Electrical, Electronic and Engineering, and this supplementary control signal is generated using local
Huazhong University of Science and Technology, Wuhan 430074, China. measurements, which limits its effectiveness for system-wide
(eemail: jinyu.wen@hust.edu.cn).
damping control [5].
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. Meanwhile, ESDs hold the advantage of providing flexible
Digital Object Identifier 10.1109/TPWRS.2014.2305977 active or reactive power to the power grid to compensate for

0885-8950 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2540 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

Fig. 1. Four-machine–two-area benchmark system with energy storage devices.

the power imbalance caused by disturbances, which could be a ESD can interact with the benchmark system and learn to
powerful tool in power system stability control [6]–[8]. Previous adaptively adjust its active power output to damp system
studies have shown that the flywheel energy storage system oscillations.
using independent active and reactive power decoupling con- • Comparative studies of GrHDP, PSO, and residue method
trol strategy can effective suppress the low-frequency oscilla- have been performed under three different scenarios. Other
tion in the system [9]. In [10], an energy-storage-based damping issues, such as real-time data acquisition using phase mea-
controller (ESDC) considering anti-windup to improve the sat- surement units (PMUs), impact of signal delay, and coor-
uration-dependent stability has been proposed. The anti-windup dination with PSSs, have also been discussed for practical
feedback loop is augmented to the ESDC, along with model considerations.
reduction technique and linear matrix inequality (LMI) tech- The remainder of this paper is organized as follows. Section II
nique design, to improve the system damping under both normal briefly describes the benchmark system, the POD model, and
and saturation operating conditions. Many other ESDs applica- the ESD model used in this paper. Section III presents the de-
tions, such as transient stability enhancement by fuzzy logic- tailed oscillation damping controller design using the aforemen-
controlled super-conducting magnetic energy storage (SMES) tioned three methods. In particular, Section III-A provides the
[11], inter-area oscillation damping by unified power flow con- residue-based controller design, Section III-B provides the PSO
trollers using ultra-capacitors [12], and wind farm fluctuations algorithm-based controller design, and Section III-C provides
mitigation by a battery energy storage system [13] have been the GrHDP-based controller design. Section IV illustrates the
intensively investigated in the society. effectiveness of the GrHDP-based control for improving the sta-
Inspired by the aforementioned discussions, in this paper, a bility of the benchmark system by comparing against conven-
real-time wide-area control framework using the reinforcement tional method-based control and PSO algorithm-based control.
learning (RL) technique has been used to design an ESD-based Section V concludes this paper and provides some discussions
damping controller, which can provide effective control to in- on practical application in real power systems.
crease the power system stability margin. The main contribu-
tions of this paper are summarized as follows.
II. BENCHMARK SYSTEM AND POD MODULE
• A particle swarm optimization (PSO)-based power oscilla-
tion damper (POD) for ESD control has been investigated Fig. 1 demonstrates the structure of Kundur’s four-ma-
in this paper. The PSO algorithm has been employed to chine–two-area benchmark system [3], which includes two
tune the control parameters in POD using a time-domain areas and four synchronous machines. The ESD is applied to
simulation mechanism. this benchmark power system to damp inter-area oscillations.
• A new heuristic dynamic programming (HDP) algo- From the system’s controllability and observability perspective
rithm, namely goal representation heuristic dynamic [3], the optimal location of ESD is different from the reactive
programming (GrHDP), has been introduced in this paper power compensators. As indicated in [8], ESD has better
to adaptively control the ESD in a real-time manner. performance to damp the inter-area oscillation when its located
Different from the classical HDP algorithm with two at the end-side of the tie-line rather than at the middle of the
networks (i.e., action network and critic network), the tie-line. Thus, in this paper, the ESD is placed at bus 7 to
GrHDP introduced an additional network (i.e., goal net- inject/absorb active power to/from the system. The capacity
work) to provided adaptive internal goal signal to facilitate of the ESD is limited to 40 MW, which is about 10% of the
the learning ability. Under this GrHDP framework, the transmission power from area one to area two.
SUI et al.: ENERGY-STORAGE-BASED LOW-FREQUENCY OSCILLATION DAMPING CONTROL USING PSO AND HEURISTIC DYNAMIC PROGRAMMING 2541

TABLE I
MODE ANALYSIS OF THE BENCHMARK SYSTEM

Fig. 2. The schematic diagram of POD controller.

The controllable current resource is used to represent the ESD

as follows: here. The other is using an optimization algorithm to search the
control parameters in the solution space [20], which will be de-
(1) scribed in the next section.
where the real and imaginary part of the current is calculated by The benchmark system is linearized around a nominal op-
the equation as follows: erating point, and small-signal analysis is shown in Table I. It
is shown that mode 1 is the inter-area oscillation mode, which
needs to be damped with the POD controller. Using the equa-
(2) tions in [18], and can be obtained. The
POD controller gain should be carefully selected to in-
In simulation, the current is transferred to a polar coordinate crease the inter-area damping, while not deteriorating the other
presentation, and the magnitude and angle are the orders given inner-area modes. In this paper, the controller gain is set
to the controlled current source. The model of ESD can repre- to 5, as suggested in [19].
sent different kinds of energy storage devices in real power sys-
tems, including super capacitors energy storage (SCES), super- B. PSO Algorithm-Based Design
conducting magnetic energy storage (SMES), flywheels energy Here, PSO algorithm is used to search the optimal , ,
storage (FES), and advanced batteries energy storage (ABES) and parameters in a POD controller.
[14], [15], among others. PSO was first brought forward by Kennedy and Eberhart
A classical POD controller is shown in Fig. 2 [16]. Part 1 is a by representing the movement organisms in a bird flock or a
measurement unit, where the measured transmission-line active fish school in 1995 [21]. It is an evolution algorithm based
power is compared with the steady-state value to generate active on the swarm’s behavior. The main idea, through constructing
power deviation. Part 2 is a POD amplifier/gain unit. Part 3 is a a number of swarm particles and setting the fitness function,
direct current (dc) blocking unit to filter out the smooth compo- is to make a judgment of the adaptability of each particle in
nent in input signal. Parts 4 and 5 are two time constants of the each generation. Then, the fitness value of each particle in
lead-lag blocks to provide necessary phase compensation. Part each generation is compared to obtain the global best particle
6 simulates time lag of energy storage devices. Each phase com- and the local best particle. Finally, based on the information
pensation block is recommended to compensate less than 60 . sharing among particles, direction and velocity of each particle
The value of is set as 5–10. The number of phase compensa- is updated. The effectiveness and searching capability of this
tion block is , then the phase compensation of the POD is . method are related to the group size, generation number, and
The parameters of the lead-leg block are calculated as follows: fitness function design. The procedure can be generalized in
the following four steps.
1) Step One: Initialization: The position of each particle in
(3)
the swarm contains three dimensions, corresponding to the three
control parameters in the POD controller. The original speed
POD controllers are effective in contribution to the damping value of the PSO method is formed by experience. The par-
of poorly damped inter-area modes, while PSSs are an efficient ticle number and iteration times are both set to 20, the searching
tool for damping the local modes. These two kinds of controllers range of gain is set as , where the searching
could be properly coordinated to ensure that the power system range of time constants and are set as . The ve-
operated with adequate damping over a wide range of operating locities for the position updating are initialized as follows:
conditions and system configurations. In this paper, we focus on
inter-area oscillation damping, where only the POD controller (4)
parameters , , and are optimized.
where and are the upper and lower bounds of par-
III. OSCILLATION DAMPING CONTROLLER DESIGN ticle . The initial velocities are generated randomly between
, where the other parameters are initialized as
A. Residue-Based Design follows:
In general, there are two kinds of methods to tune the POD
controller parameters. One is residue method based on tradi- (5)
tional control theory. The design procedure is similar to that
of the FACTS-based damping controller design and POD de- where and are the accelerating constants, and and
sign in [8], [10], and [17]–[19], which will be briefly introduced are the initial and final weights.
2542 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

2) Step Two: Updating Individual Best and Global Best:

After initialization, the position of each particle is sent to the
POD controller as parameters values and run the simulation to
obtain the fitness value using the following function:

(6)

where the optimization goal is to minimize the fitness function.

is the time range of oscillation, and is the instant re-
inforcement signal. The later time’s oscillation magnitude can
better reflect the effect of decay and therefore it is more impor-
tant. The individual and global best particles will be determined
according to the fitness of each particle.
3) Step Three: Position Updating: The velocity and position
updating of each particle is formulated as follows:

(7)
where and are uniformly distributed numbers in , and
and are the individual and global best solution in the
current generation.
4) Step Four: Determining Whether to Finish the Proce-
dure: If one of the following termination criteria is satisfied,
the process of optimization will be finished.
1) The iteration number has reached the maximum generation
number.
2) The fitness value of global best solution is smaller than the
set value, which is called iteration converge. Fig. 3. Flow chart of the proposed PSO-based parameters tuning.
If neither of the two situations is satisfied, then the procedure
will jump to step two.
The optimization goal/fitness function of PSO is to minimize goal network, a critic network, and an action network. The
the reinforcement signal as follows [22]: critic network learns to approximate the cost-to-go function
in Bellman’s equation, the action network learns to generate
(8) the control policy that minimizes the cost-to-go approximated
by the critic network, while the goal network provides an
where , and are rotor speed de- adaptive internal reinforcement signal in addition to the pri-
viations corresponding to different oscillation modes as follows: mary reinforcement signal to the critic network for improved
generalization and learning capability [23]–[25]. Specifically,
the cost-to-go function is defined as follows:
(9)
(10)
where , , 2, 3, 4 is the rotor speed of the th gener-
ator. By adjusting the weights , 1, 2, 3, the most possible
where is the state vector of the system, is the con-
destabilizing oscillation mode will be suppressed. From the en-
trol action, is the utility function, and is a discount factor.
ergy point of view, there are several oscillation modes after a
In this paper, all three networks are implemented in neural net-
system fault, and is viewed as an index of the kinetic energy
works of a three-layer nonlinear architecture with one hidden
of the entire system oscillation. The flow chart of the proposed
layer. However, the learning principles can also be generalized
PSO-based parameters tuning is shown in Fig. 3.
to any arbitrary function approximator by properly applying the
backpropagation rule. The comparison between different imple-
C. HDP-Based Design
mentations is beyond the scope of this paper.
Before presenting the HDP-based damping controller design, 1) Goal Network Training: As indicated in (10), the system
we will first briefly introduce the goal representation heuristic performance cost is expressed in a compact form. The objective
dynamic programming (GrHDP) algorithm. GrHDP is a new of dynamic programming is to choose the control sequence
reinforcement learning mechanism from the family of adaptive so the cost function is minimized as follows:
dynamic programming (ADP) designs in recent years [23],
[24]. It requires three function approximation networks: a (11)
SUI et al.: ENERGY-STORAGE-BASED LOW-FREQUENCY OSCILLATION DAMPING CONTROL USING PSO AND HEURISTIC DYNAMIC PROGRAMMING 2543

This is the foundation for implementation dynamic program- success. Therefore, the error function to adjust the parameters
ming by working backward in time. In this structure, can be of the action network is
estimated by minimizing the following error over time:
(21)
(12)
Since the action network is connected with both goal network
When for all , (12) indicates and critic network, the backpropagation path will formed in two
parts as follows:
(13)

Putting one time step backward, we can obtain (22)

(14)

From (13) and (14), the objective function to be minimized in and the weight adjustments for the hidden to the output layer
the goal network is [26] and for the input to hidden layer in the action network are as
follows:
(15)

(23)
and the high-level conceptual backpropagation path is

(16)
4) GrHDP-Based Damping Controller: The configuration of
the GrHDP-based controller with the power plant is shown in
Since the three-layer neural network is used in this paper, the
Fig. 4. The utility function is set equal to zero to represent
weight adjustments for the hidden to the output layer and for
success. Since contains the information
the input to hidden layer are as follows:
of inter-area and inner-area oscillation, they are chosen as the
inputs of the GrHDP controller. The output of the GrHDP con-
troller is the injected active power by the ESD and limited to 40
(17) MW. The reinforcement signal is the same as in the PSO
fitness function and is rewritten as follows:

2) Critic Network Training: Once the goal network outputs (24)

the signal, it will be used as an input to the critic network
and also be used to define the error function to adjust the param- The controller works in the following procedures.
eters of the critic network as follows: • The action network receives the measured plant state
and uses it to generate the control signal to the ESD.
• The goal network uses the external reinforcement signal
(18)
and plant state to generate the internal reinforce-
ment signal .
and the backpropagation path is • The critic network uses the internal reinforcement signal
, plant state , and control signal to estimate the
(19) cost function .
• The goal network will update its weights according to
The weight adjustments for the hidden to the output layer and (15)–(17) until the stop criterion is satisfied.
for the input to hidden layer in the critic network are as follows: • The critic network will update its weights according to
(18)–(20) until the stop criterion is satisfied.
• The action network will update its weights according to
(20) (21)–(23) until the stop criterion is satisfied.
• These steps are repeated at each simulation time step.

IV. SIMULATION RESULTS

3) Action Network Training: The procedure of adapting the
action network in this architecture is similar to the classic ADP Here, we focus on comparison of the residue-based POD
approach to implicity backpropagate the error between the de- controller, the PSO optimized controller, and the GrHDP-based
sired ultimate object and the approximate function from controller to damp system oscillation resulting from a variety of
the critic network [27]. is in accordance with the signal of the disturbances applied at different system operating conditions.
reinforcement when the state conducted by the action implies a The parameter setting in the GrHDP controller is shown in
2544 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

Fig. 4. Proposed configuration of the GrHDP-based controller with power plant.

TABLE II
PARAMETER USED IN GRHDP CONTROLLER

Table II. The weights of the neural networks in GrHDP are

randomly initialized only in the first trial. The controller does
Fig. 5. Comparison between line active power and energy storage output in the
not know the proper control strategy before training. It is well first trial.
known that, in neural network, the initial weights contribute
significantly to the performance of the controller. Therefore, we
should save the weights of the controller and carry them on as and 7. Specifically, Fig. 6 shows the ESD output active power
the initial weights for the next trial, regardless of the simulation with conventional POD control, PSO optimized control, and
performance. This trial-and-error methodology [28]–[30] is GrHDP control, and Fig. 7 shows the active power of the trans-
used in the following three different scenarios. mission line. From both figures, we can see that, with all three
approaches, the system can become stable after about 6 s. Using
A. Case 1: Disturbance With Original Benchmark System as an index of the inter-area oscillation mode, Fig. 8
In case 1, the structure of the benchmark power system is shows the comparison of the three controllers for inter-area os-
illustrated in Fig. 1. The excitation of synchronous machine cillation mode damping. It can be observed that, without ESD,
G3 experiences a 0.2-s-long, 5% step disturbance at time 1 s. the system will become unstable after the disturbance. The per-
Without an ESD, the system would have lost stability after this formance of residue-based POD controller and PSO optimized
small disturbance. Then, the GrHDP controller is activated in controller are similar to that of the GrHDP-based controller in
the benchmark power system to control ESD, and the simula- this case.
tion result of the first trial is shown in Fig. 5. Because of the
random initial weights, we can observe that the GrHDP con- B. Case 2: Disturbance With Line Cut-Off
troller does not generate proper control signal during the early In case 2, the benchmark power system configuration is
stage of the simulation (1–5 s) in the first trial. After about 10 changed. In addition to the same disturbance in case 1, we also
s, the GrHDP controller learned to damp the line active power assume that one transmission line between buses 7 and 8 is
swing by adapting the weights of the neural networks. out of service. Under a new operating condition, the GrHDP
The weights in the first trial are saved as the initial weights in controller keeps adjusting the weights in the neural network to
the second trial. Results of the second trial are shown in Figs. 6 obtain an optimal control performance. Figs. 9–11 show the
SUI et al.: ENERGY-STORAGE-BASED LOW-FREQUENCY OSCILLATION DAMPING CONTROL USING PSO AND HEURISTIC DYNAMIC PROGRAMMING 2545

Fig. 6. Comparison of ESD output with POD, PSO, and GrHDP controllers in
case 1.

Fig. 9. Comparison of ESD output with POD, PSO, and GrHDP controllers in
case 2.

Fig. 7. Comparison of line active power with POD, PSO, and GrHDP con-
trollers in case 1.

Fig. 10. Comparison of line active power with POD, PSO, and GrHDP con-
trollers in case 2.

Fig. 8. Comparison of inter-area oscillation with POD, PSO, and GrHDP con-
trollers in case 1.

results of ESD output active power, transmission-line active

power, and inter-area rotor speed deviation with residue-based
POD controller, PSO optimized controller, and GrHDP con-
troller. We can observe that the PSO method and the residue Fig. 11. Comparison of inter-area oscillation with POD, PSO, and GrHDP con-
method have similar performances under this operating condi- trollers in case 2.
tion. The simulation results also indicate that, under a different
operating point, the conventional method-based POD controller
and the PSO optimized controller can no longer maintain its condition. However, the GrHDP-based method still has better
desired performance. We should notice from Fig. 10 that, robustness and optimization capability as it obtains the best
with the GrHDP controller, the first swing of the transmitted damping performance in the post-fault period. The reason for
active power has deteriorated. This is because the weights this is that the learning ability of the GrHDP controller keeps
in the neural network are adjusted for the changed operating driving the controller to the optimal control point.
2546 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

residue-based POD controller, PSO optimized controller, and

GrHDP controller. In order to focus on the comparison of these
three methods, we did not show the system dynamics without
ESD in these results. The simulation results indicate that, under
this new operating point, the POD controller design based on
the conventional residue method has the worst performance.
Meanwhile, with the continue learning ability, the GrHDP con-
troller performs slightly better than the PSO optimized con-
troller. These results also demonstrate that damping enhance-
ment can be achieved over a wide range of operating points with
the proposed GrHDP method.

V. CONCLUSION AND DISCUSSIONS

This paper analyzed the power system low-frequency
oscillation supplementary damping control using heuristic
dynamic programming. A classical four-machine–two-area
system with ESDs has been applied for the comparative study
Fig. 12. Comparison of ESD output with POD, PSO, and GrHDP controllers of residue-based POD control design, PSO-based control de-
in case 3. sign, and the GrHDP control design. The simulation results
under different operation conditions and system configurations
demonstrated the effectiveness of the GrHDP method over
the other two methods. From this study, we can see that the
GrHDP controller has the potential of more robust performance
than the conventional POD design and the PSO optimal design
over a wide range of system conditions. Also, we should
notice that the benchmark power system used in this paper
could easily be replaced by a large power system for a more
comprehensive study of the proposed controller. Considering
the adaptive ability of damping power system oscillation of
GrHDP controller, it may be interesting to apply it to other
power system oscillation problems, such as subsynchronize
oscillation problems in the future studies.
Fig. 13. Comparison of line active power with POD, PSO, and GrHDP con-
trollers in case 3.
The proposed ESD damping controller design based on HDP
can be utilized for system-wide damping control or local mode
enhancement. We should notice that the reinforcement signal
in (8) and (9) requires real-time rotor speed signals in re-
mote areas. However, this is no longer a hurdle in modern power
systems because of large-scale installation of PMUs [31], [32].
Since the proposed real-time HDP damping controller is based
on instant interactions and learning between the power system,
the signal transmission delay in a real power system will impact
its performance. However, it has been shown that the neural net-
work-based control can successfully compensate for communi-
cation delays [33], [34]. In practice, the HDP-based controller
could service as a supplementary control for the traditional PID
controllers. Since PSSs have been widely used in power system
damping control, especially for local oscillation mode, the pro-
Fig. 14. Comparison of inter-area oscillation with POD, PSO, and GrHDP con-
trollers in case 3.
posed controller could be coordinated with the local PSSs to
achieve a better system operating stability.

C. Case 3: Disturbance With Load Changing REFERENCES

[1] B. Dunn, H. Kamath, and J.-M. Tarascon, “Electrical energy storage
In case 3, we modify the benchmark power system configura- for the grid: A battery of choices,” Science, vol. 334, no. 6058, pp.
tion with load 2 active power decreased from 1767 to 1567 MW. 928–935, 2011.
Under this system operating point, the GrHDP controller can [2] Electr. Power Res. Inst., “Electrical energy storage technology
options,” Tech. Rep., Dec. 2010 [Online]. Available: http://www.
still adapts to this new situation. Figs. 12–14 show the results of
epri.com
ESD output active power, transmission-line active power, and [3] P. Kundur, Power System Stability and Control. New York, NY,
inter-area oscillation represented by rotor speed deviation with USA: Mc Graw-Hill, 1994.
SUI et al.: ENERGY-STORAGE-BASED LOW-FREQUENCY OSCILLATION DAMPING CONTROL USING PSO AND HEURISTIC DYNAMIC PROGRAMMING 2547

[4] S. P. Azad, R. Iravani, and J. Tate, “Damping inter-area oscillations [26] Z. Ni, H. He, J. Wen, and X. Xu, “Goal representation heuristic dy-
based on a model predictive control (MPC) HVDC supplementary con- namic programming on maze navigation,” IEEE Trans. Neural Netw.
troller,” IEEE Trans. Power Syst., vol. 28, no. 3, pp. 3174–3183, Aug. Learning Syst., vol. PP, no. 99, pp. 1–1, 2013.
2013. [27] J. Si and Y.-T. Wang, “Online learning control by association and re-
[5] D. Molina, G. Venayagamoorthy, J. Liang, and R. Harley, “Intelligent inforcement,” IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264–276,
local area signals based damping of power system oscillations using Mar. 2001.
virtual generators and approximate dynamic programming,” IEEE [28] Y. Tang, H. He, and J. Wen, “Comparative study between HDP and
Trans. Smart Grid, vol. 4, no. 1, pp. 498–508, 2013. PSS on DFIG damping control,” in Proc. IEEE Symp. Computational
[6] B. Pal, A. Coonick, and D. C. Macdonald, “Robust damping con- Intell. Applicat. in Smart Grid, 2013, pp. 59–65.
troller design in power systems with superconducting magnetic energy [29] Y. Tang, H. He, Z. Ni, J. Wen, and X. Sui, “Reactive power control of
storage devices,” IEEE Trans. Power Syst., vol. 15, no. 1, pp. 320–325, grid-connected wind farm based on adaptive dynamic programming,”
Feb. 2000. Neurocomputing, vol. 125, no. 1, pp. 125–133, 2014.
[7] S. Cheng, G. Li, H. Sun, and J. Wen, “Energy storage application in [30] Y. Tang, H. He, and J. Wen, “Adaptive control for an HVDC trans-
power system and its prospects,” Grid and Clean Energy, no. 2, pp. mission link with FACTS and a wind farm,” in Proc. IEEE Innovative
1–8, 2009. Smart Grid Technol. Conf., Washington, DC, USA, Feb. 2013, pp. 1–6.
[8] J. Wu, J. Wen, H. Sun, and S. Cheng, “Feasibility study of segmenting [31] I. Kamwa, R. Grondin, and Y. Hebert, “Wide-area measurement based
large power system interconnections with ac link using energy storage stabilizing control of large power systems-A decentralized/hierarchical
technology,” IEEE Trans. Power Syst., vol. 27, no. 3, pp. 1245–1252, approach,” IEEE Trans. Power Syst., vol. 16, no. 1, pp. 136–153, Feb.
Aug. 2012. 2001.
[9] S. Cheng, W. Yu, J. Wen, H. Sun, and H. Wang, “Energy storage and [32] B. Chaudhuri, R. Majumder, and B. Pal, “Wide-area measurement-
its application in power system stability enhancement,” Power Syst. based stabilizing control of power system considering signal transmis-
Technol., vol. 31, no. 20, pp. 98–108, Oct. 2007. sion delay,” IEEE Trans. Power Syst., vol. 19, no. 4, pp. 1971–1979,
[10] J. Fang, W. Yao, Z. Chen, J. Wen, and S. Cheng, “Design of anti- Nov. 2004.
windup compensator for energy storage-based damping controller to [33] W. Qiao, G. K. Venayagamoorthy, and R. G. Harley, “Optimal wide-
enhance power system stability,” IEEE Trans. Power Syst., vol. PP, no. area monitoring and nonlinear adaptive coordinating neurocontrol of a
99, pp. 1–11, 2013. power system with wind power integration and multiple facts devices,”
[11] M. Ali, T. Murata, and J. Tamura, “Transient stability enhancement by Neural Netw., vol. 21, no. 2, pp. 466–475, 2008.
fuzzy logic-controlled SMES considering coordination with optimal [34] N. Chaudhuri, B. Chaudhuri, S. Ray, and R. Majumder, “Wide-area
reclosing of circuit breakers,” IEEE Trans. Power Syst., vol. 23, no. 2, phasor power oscillation damping controller: A new approach to han-
pp. 631–640, May 2008. dling time-varying signal latency,” IET Gener., Transm. Distrib., vol.
[12] M. Zarghami, M. Crow, J. Sarangapani, Y. Liu, and S. Atcitty, “A novel 4, no. 5, pp. 620–630, 2010.
approach to interarea oscillation damping by unified power flow con-
trollers utilizing ultracapacitors,” IEEE Trans. Power Syst., vol. 25, no.
1, pp. 404–412, Feb. 2010.
[13] Q. Jiang, Y. Gong, and H. Wang, “A battery energy storage system Xianchao Sui received the B.S. and M.S. degrees in
dual-layer control strategy for mitigating wind farm fluctuations,” electrical engineering from Huazhong University of
IEEE Trans. Power Syst., vol. 28, no. 3, pp. 3263–3273, Aug. 2013. Science and Technology (HUST), Wuhan, China, in
[14] P. Ribeiro, B. Johnson, M. Crow, A. Arsoy, and Y. Liu, “Energy storage 2008 and 2011, respectively.
systems for advanced power applications,” Proc. IEEE, vol. 89, no. 12, He is currently with Liaoning Dalian Power
pp. 1744–1756, Dec. 2001. Supply Company, Dalian, China. His research
[15] M. G. Molina, “Dynamic modelling and control design of advanced interests include smart grids, renewable energy, and
energy storage for power system applications,” Universidad Nacional power system control.
de San Juan, Argentina.
[16] R. Sadikovic, “Use of FACTS Devices for power flow control and
damping of oscillations in power systems,” Ph.D. dissertation, Dept.
Inf. Technol. Electr. Eng., Swiss Federal Inst. of Technol., Zurich,
Switzerland, 2006.
[17] P. S. Dolan, J. Smith, and W. Mittelstadt, “A study of TCSC optimal Yufei Tang (S’13) received the B.Eng. and M.Eng.
damping control parameters for different operating conditions,” IEEE degrees in electrical engineering from Hohai Univer-
Trans. Power Syst., vol. 10, no. 4, pp. 1972–1978, Nov. 1995. sity, Nanjing, China, in 2008 and 2011, respectively.
[18] N. Yang, Q. Liu, and J. McCalley, “TCSC controller design for He is currently working toward the Ph.D. degree
damping interarea oscillations,” IEEE Trans. Power Syst., vol. 13, no. at the Department of Electrical, Computer, and
4, pp. 1304–1310, Nov. 1998. Biomedical Engineering, University of Rhode
[19] J. Sun, X. Zhao, D. Li, M. Li, X. Li, W. Lin, and J. Wen, “Study on Island, Kingston, RI, USA.
energy storage in damping tie line power oscillations in power system,” His research interests include power system
Power Syst. Protection and Control, no. 17, pp. 1–8, 2013. modeling, power system stability control, wind
energy generation and integration, smart grids,
[20] Y. Tang, P. Ju, H. He, C. Qin, and F. Wu, “Optimized control of DFIG-
power system cyber security, and the application of
based wind generation using sensitivity analysis and particle swarm
computational intelligence in power systems.
optimization,” IEEE Trans. Smart Grid, vol. 4, no. 1, pp. 509–520,
2013.
[21] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proc.
IEEE Int. Conf. Neural Networks, 1995, vol. 4, pp. 1942–1948.
Haibo He (SM’11) received the B.S. and M.S.
[22] C. Lu, J. Si, and X. Xie, “Direct heuristic dynamic programming for
degrees from Huazhong University of Science and
damping oscillations in a large power system,” IEEE Trans. Syst., Man,
Technology, Wuhan, China, in 1999 and 2002,
Cybern. B, Cybern., vol. 38, no. 4, pp. 1008–1013, Aug. 2008.
respectively, and the Ph.D. degree from Ohio Uni-
[23] H. He, Self-Adaptive Systems for Machine Intelligence. Hoboken, versity in 2006, all in electrical engineering.
NJ, USA: Wiley, 2011. He is currently the Robert Haas Endowed Pro-
[24] H. He, Z. Ni, and J. Fu, “A three-network architecture for on-line fessor in Electrical Engineering with the University
learning and optimization based on adaptive dynamic programming,” of Rhode Island, Kingston, RI, USA. From 2006 to
Neurocomputing, vol. 78, no. 1, pp. 3–13, 2012. 2009, he was an Assistant Professor with the De-
[25] Z. Ni, H. He, and J. Wen, “Adaptive learning in tracking control based partment of Electrical and Computer Engineering,
on the dual critic network design,” IEEE Trans. Neural Netw. Learning Stevens Institute of Technology, Hoboken, NJ,
Syst., vol. 24, no. 6, pp. 913–928, Jun. 2013. USA. He has published one research book, edited one research book and six
2548 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 29, NO. 5, SEPTEMBER 2014

conference proceedings, and authored and coauthored over 130 peer-reviewed Jinyu Wen (M’10) received the B.Eng. and Ph.D. de-
journal and conference papers. His research has been covered by national grees in electrical engineering from Huazhong Uni-
and international media. His research interests include smart grids, renewable versity of Science and Technology, Wuhan, China, in
energy, power system cyber security, cyber-physical systems, computational 1992 and 1998, respectively.
intelligence, machine learning, data mining, and various application fields. He was a Visiting Student from 1996 to 1997 and a
Prof. He is an associate editor of the IEEE TRANSACTIONS ON NEURAL Research Fellow from 2002 to 2003 at the University
NETWORKS AND LEARNING SYSTEMS and the IEEE TRANSACTIONS ON SMART of Liverpool, Liverpool, U.K., and a Senior Visiting
GRID. He was a recipient of the IEEE Computational Intelligence Society Researcher with the University of Texas at Arlington,
Outstanding Early Career Award (2014), the National Science Foundation TX, USA, in 2010. From 1998 to 2002, he was a Di-
CAREER Award (2011), and the Providence Business News Rising Star rector Engineer with XJ Electric Company Ltd. in
Innovator Award (2011). China. In 2003, he joined Huazhong University of
Science and Technology, Wuhan, China, where he is now a Full Professor. His
current research interests include renewable energy integration, power system
control, energy storage application, multi-terminal HVDC, and power system
operation and control.