Argonne title page 1 [ANL/DIS-11-8]

Use of Wind Power Forecasting

in Operational Decisions
by
Audun Botterud, Zhi Zhou, Jianhui Wang
Ricardo J. Bessa, Hrvoje Keko, Joana Mendes, Jean Sumaili, Vladimiro Miranda

September 30, 2011

CONTENTS
Acknowledgments.......................................................................................................................... ix
Acronyms ........................................................................................................................................ x
Summary ...................................................................................................................................... xiii
1 Introduction ............................................................................................................................... 1
1.1 Wind Power Forecasting .................................................................................................. 1
1.1.1 Wind Power Point Forecasting ............................................................................ 2
1.1.2 Wind Power Uncertainty Forecasting .................................................................. 2
1.2 Wind Power Forecasting in Operational Decisions ......................................................... 3
1.2.1 System Operators ................................................................................................. 4
1.2.2 Wind Power Producers ........................................................................................ 4
1.3 Report Outline .................................................................................................................. 5
2 Good or Bad Wind Power Forecasting: A Relative Concept ................................................... 7
2.1 Introduction ...................................................................................................................... 7
2.2 The Forecast Consumer Paradigm ............................................................................... 9
2.2.1 Wind Generation Companies Viewpoint............................................................ 9
2.2.2 System Operators Viewpoint ............................................................................ 11
2.2.3 The Compromise Viewpoint, or the Search for Utopia ..................................... 13
2.3 Training W2P Models .................................................................................................... 14
2.3.1 Aspects of Training W2P Models ...................................................................... 14
2.3.2 Training Criteria................................................................................................. 15
2.4 Case Studies ................................................................................................................... 17
2.4.1 Data Characteristics ........................................................................................... 17
2.4.2 Training Characteristics ..................................................................................... 18
2.4.3 Testing Methodology ......................................................................................... 19
2.4.4 Analysis of the Results....................................................................................... 19
2.5 Conclusions .................................................................................................................... 21
3 Wind Power Forecasting in U.S. Electricity Markets ............................................................. 23
3.1 Introduction .................................................................................................................... 23
3.2 Wind Power Forecasting Models ................................................................................... 23
3.3 Use of Forecasting in Market Operations ...................................................................... 24
3.3.1 Operating Reserve Requirements ...................................................................... 25
3.3.2 Day-Ahead Operations....................................................................................... 29
3.3.3 Reliability Assessment Commitment................................................................. 30
3.3.4 Real-time Operations ......................................................................................... 31
3.4 Summary and Conclusions ............................................................................................ 32

iii

CONTENTS (CONT.)
4 Wind Power Forecasting Uncertainty and Unit Commitment ................................................ 35
4.1 Introduction .................................................................................................................... 36
4.2 Forecasting and Scenario Generation Approaches ........................................................ 38
4.3 Unit Commitment and Dispatch Formulations .............................................................. 39
4.3.1 Objective Function ............................................................................................. 39
4.3.2 Thermal Unit Constraints ................................................................................... 40
4.3.3 Deterministic Formulation ................................................................................. 42
4.3.4 Economic Dispatch ............................................................................................ 42
4.4 Case Study ..................................................................................................................... 43
4.4.1 Assumptions....................................................................................................... 43
4.4.2 Wind Power and Forecast Characteristics ......................................................... 44
4.4.3 Simulated Cases ................................................................................................. 45
4.4.4 Results ................................................................................................................ 46
4.4.5 Results for One Day ........................................................................................... 46
4.4.6 Results for a Three-month Period ...................................................................... 47
4.4.7 Use of Different WPFs....................................................................................... 50
4.4.8 Reserve Requirements ....................................................................................... 51
4.4.9 Deterministic vs. Stochastic UC ........................................................................ 51
4.5 Conclusions .................................................................................................................... 52
5 Representative Wind Power Scenarios and their Probabilities for Stochastic Models ........... 55
5.1 Introduction .................................................................................................................... 55
5.2 Scenario Generation and the Need for Scenario Reduction........................................... 57
5.3 Wind Power Scenario Clustering and Modes Finding................................................... 57
5.3.1 Application of Classical Clustering Techniques to Wind Power Scenarios ...... 57
5.3.2 Modes Finding using the ITL Mean Shift Algorithm ........................................ 58
5.4 New Methodology ......................................................................................................... 60
5.5 Results ............................................................................................................................ 62
5.6 Conclusion ..................................................................................................................... 65
6 Unit Commitment and Operating Reserves with Probabilistic Wind Power Forecasts.......... 67
6.1
6.2
6.3
6.4
6.5

Introduction .................................................................................................................... 67
Scheduling of Energy and Operating Reserves ............................................................. 68
Probabilistic Wind Power Forecasts .............................................................................. 69
Scenario Generation and Reduction............................................................................... 70
Case Study ..................................................................................................................... 71
6.5.1 Probabilistic Wind Power Forecasts .................................................................. 71
6.5.2 Overview of Simulated Cases ............................................................................ 75
6.6 UC and ED Results ........................................................................................................ 76
6.7 Current Work: IL Case Study ........................................................................................ 78
6.8 Conclusion ..................................................................................................................... 79

iv

CONTENTS (CONT.)
7 Wind Power Trading Under Uncertainty in Electricity Markets ............................................ 81
7.1
7.2
7.3
7.4
7.5

Introduction .................................................................................................................... 82
Literature Review........................................................................................................... 82
Wind Power in LMP Markets ........................................................................................ 83
Probabilistic Wind Power Forecasting........................................................................... 87
A Model for Wind Power Trading under Uncertainty ................................................... 89
7.5.1 Representation of Uncertainty in DA and RT LMPs ......................................... 90
7.5.2 Mathematical Formulation of the DA Bidding Problem ................................... 90
7.6 Case Study: Hypothetical Wind Farm in Illinois ........................................................... 92
7.6.1 Assumptions....................................................................................................... 92
7.6.2 Results ................................................................................................................ 93
7.7 Conclusion ................................................................................................................... 106
8 Conclusions ........................................................................................................................... 107
References ................................................................................................................................... 111
LIST OF FIGURES
Fig. 1-1 Illustration of WPF system................................................................................................ 2
Fig. 2-1 Basic arrangement of a W2P identifying its three main modules. .................................. 15
Fig. 3-1 Uncertainty representation in wind power forecasts: (a) probabilistically by a set
of intervals or quantiles (left), and (b) scenarios of wind power generation
representing the temporal dependency of forecast errors (right). .................................. 24
Fig. 3-2 Market operations timeline for MISO, indicating where WPF could play
an important role. ........................................................................................................... 26
Fig. 3-3 Traditional operating reserve categories as defined by NERC
and a proposed new category to address increased uncertainty and variability
from renewable generation. ........................................................................................... 28
Fig. 3-4 Role of WPF in electricity market operations. ................................................................ 33
Fig. 4-1 Wind power forecast (deterministic point forecast and 10 stochastic scenarios)
and realized wind generation for day 87. ....................................................................... 46
Fig. 4-2 Number of on-line units for day 87. ................................................................................ 48
Fig. 4-3 Available operating reserves in RT dispatch................................................................... 48
Fig. 4-4 Total hours of commitment for thermal units in all cases. .............................................. 48
Fig. 5-1 Day 1 wind power scenarios. .......................................................................................... 58
Fig. 5-2 Day 2 wind power scenarios. .......................................................................................... 59
Fig. 5-3 Day 3 wind power scenarios. .......................................................................................... 59
Fig. 5-4 Day 3 Mode finding results for the three-day set of scenarios thick lines
represent the modes discovered, coherent with the merged sets. .................................. 60
Fig. 5-5 Day 1 scenario reduction results with = 0.22. .............................................................. 62

LIST OF FIGURES (CONT.)

Fig. 5-6 Day 2 scenario reduction results with = 0.18. .............................................................. 62
Fig. 5-7 Day 3 scenario reduction results with tolerance = 0.18................................................ 63
Fig. 5-8 Day 1 distribution of the maximum deviation between each pair of 2 scenarios. .......... 64
Fig. 5-9 Day 2 distribution of the maximum deviation between each pair of 2 scenarios. .......... 64
Fig. 5-10 Day 3 distribution of the maximum deviation between each pair of 2 scenarios. ........ 64
Fig. 6-1 Probabilistic forecasts of wind power quantiles (5%, 10%, ... 95%) based on
QR and realized wind power generation (dotted line) for day 1.................................... 72
Fig. 6-2 Calibration diagram for QR and NW methods, Oct.Dec. 2006. ................................... 72
Fig. 6-3 Averages of 10 scenarios with reduction methods SR1, SR2, and SR3
as compared to the average of the original 1,000 scenarios, QR,
and days 1 and 2. ............................................................................................................ 73
Fig. 6-4 Variances for 10 scenarios with reduction methods SR1, SR2, and SR3
as compared to the average of the original 1,000 scenarios, QR,
and days 1 and 2. ............................................................................................................ 74
Fig. 6-5 Variances for 100 scenarios with reduction methods SR1, SR2, and SR3
as compared to the average of the original 1000 scenarios, QR,
and days 1 and 2. ............................................................................................................ 74
Fig. 6-6 Overview of simulated total costs for deterministic cases. ............................................. 76
Fig. 6-7 Overview of simulated total costs for stochastic cases. .................................................. 76
Fig. 6-8 Simulation of two settlement electricity markets in Illinois case study. ......................... 78
Fig. 7-1 Examples of hourly financial settlements as function of deviation
between RT delivery and DA schedule with penalties: penDA = 0 (left),
and penDA = 5 $/MWh (right), and LMP differences:
,
,
= 3, -3, 0 $/MWh (upper, middle, lower). ............................................................... 87
Fig. 7-2 Probabilistic representation of wind power forecast as intervals or quantiles. ............... 88
Fig. 7-3 Forecast scenarios of wind power generation. ................................................................ 88
Fig. 7-4 Excerpts from Excel interface. ........................................................................................ 91
Fig. 7-5 Wind power forecast: Deterministic point forecast (solid thick line)
and 10 forecast scenarios for selected day (Oct. 9, 2006). ........................................... 92
Fig. 7-6 Estimated DA and RT price parameters for selected day (Oct. 9, 2006). ....................... 93
Fig. 7-7 30 scenarios for DA (upper) and RT (lower) prices
for selected day (Oct. 9, 2006). ...................................................................................... 93
Fig. 7-8 Expected profit (E) and CVAR criteria (C) as function of DA bid quantity,
no deviation penalty (hour 5, Oct. 9, 2006). .................................................................. 95
Fig. 7-9 Expected utility for the risk-averse ( = -3) and risk-prone ( = 3)
decision maker, no deviation penalty............................................................................. 95
Fig. 7-10 Variance and conditional value at risk, no deviation penalty
(hour 5, Oct. 9, 2006). *Without wind power curtailment
during negative prices. ................................................................................................... 95
Fig. 7-11 Illustration of the trade-off between expected profit and conditional
value at risk, no deviation penalty, (hour 5, Oct. 9, 2006). .......................................... 96
Fig. 7-12 Illustration of the trade-off between expected profit and conditional
value at risk, $5/MWh deviation penalty, (hour 5, Oct. 9, 2006)................................. 96

vi

LIST OF FIGURES (CONT.)

Fig. 7-13 Optimal DA bidding under different decision criteria for 24 hours
(Oct. 9, 2006). No deviation penalty. pf is the point forecast. ..................................... 98
Fig. 7-14 Optimal DA bidding under different decision criteria for 24 hours
(Oct. 9, 2006). US$5/MWh penalty. pf is the point forecast. ...................................... 98
Fig. 7-15 Optimal DA bidding under different decision criteria for 24 hours
(Oct. 9, 2006). US$10/MWh penalty. pf is the point forecast. .................................... 99
Fig. 7-16 Optimal DA bidding with risk prone decision criteria ( = 3) for different
deviation penalties (0, 5, 10 $/MWh). pf is the point forecast. (Oct. 9, 2006)............. 99
Fig. 7-17 Illustration of the trade-off between expected daily profit and conditional
value at risk, no deviation penalty (Oct. 9, 2006). ....................................................... 99
Fig. 7-18 Illustration of the trade-off between expected daily profit and conditional
value at risk, $5/MWh deviation penalty (Oct. 9, 2006). ........................................... 100
Fig. 7-19 Total October profit vs. hourly conditional value at risk, no deviation penalty. ........ 101
Fig. 7-20 Total October profit vs. average absolute deviation from schedule,
no deviation penalty. .................................................................................................. 102
Fig. 7-21 Total monthly profit vs. hourly conditional value at risk,
$5/MWh deviation penalty. ........................................................................................ 103
Fig. 7-22 October profit vs. average absolute deviation from schedule,
$5/MWh deviation penalty. ........................................................................................ 103
Fig. 7-23 Monthly profit for different quantile forecasts compared to bidding
based on point forecast and based on the expected profit criterion,
no penalty (left), $5/MWh deviation penalty (right). ................................................. 104
Fig. 7-24 Average hourly profit and CVaR for different quantile forecasts,
no penalty (left), $5/MWh deviation penalty (right). ................................................. 104
Fig. 7-25 Evaluation of probabilistic forecasts for different methods: calibration (left)
and sharpness (right). Training period: JanuaryJuly, 2006;
test period: AugustDecember 2006. ......................................................................... 105
LIST OF TABLES
Table 1-1 Different types of uncertainty representation in WPF. ................................................... 3
Table 2-1 Training, validation, and testing sets. ........................................................................... 17
Table 2-2 Annual mean market prices. ......................................................................................... 18
Table 2-3 Simulation of wind farm As participation in the Iberian market. ............................... 19
Table 2-4 Simulation of wind farm Bs participation in the Iberian market. ............................... 19
Table 2-5 Simulation of wind farm Cs participation in the Iberian market. ............................... 20
Table 3-1 Overview of WPF and market operation in five U.S. electricity markets.
Based on information from the Utility Wind Integration Group (UWIG)
and ISO/RTOs. ............................................................................................................ 27
Table 4-1 Generator data. ............................................................................................................. 44
Table 4-2 Simulated cases ............................................................................................................ 45
Table 4-3 Average dispatch for thermal units [MW].................................................................... 49

vii

LIST OF TABLES (CONT.)

Table 4-4 Summary of operating costs ......................................................................................... 50
Table 4-5 Summary of other results.............................................................................................. 50
Table 6-1 Summary statistics for scenarios (QR) with different reduction methods:
average (AVG), mean absolute error (MAE), Oct.Dec. ........................................... 74
Table 6-2 Simulated cases with deterministic UC ........................................................................ 75
Table 6-3 Simulated cases with stochastic UC ............................................................................. 76
Table 6-4 Summary of other results: deterministic cases ............................................................. 77
Table 6-5 Summary of other results: stochastic cases .................................................................. 77
Table 7-1 Overview of published research/models for wind power trading. ............................... 84
Table 7-2 Price parameters and wind power point forecast (pf) for hour 5, Oct. 9, 2006. ........... 94
Table 7-3 Summary of results at optimum for different decision criteria
(hour 5, Oct. 9, 2006), no deviation penalty. .............................................................. 97
Table 7-4 Summary of results at optimum for different decision criteria
(hour 5, Oct. 9, 2006), $5/MWh deviation penalty. .................................................... 97
Table 7-5 Summary statistics for wind power and DA/RT prices, October. .............................. 100
Table 7-6 Summary of results for October with different decision criteria,
no deviation penalty. ................................................................................................. 102
Table 7-7 Results summary for October with different decision criteria,
$5/MWh deviation penalty. ...................................................................................... 103
Table 7-8 Summary of results for October with the C*(w=0.1) strategy
for different wind power uncertainty forecasts,
$5/MWh deviation penalty. ...................................................................................... 106

viii

ACKNOWLEDGMENTS

This report has been prepared by Argonne National Laboratory in collaboration with INESC
Porto, Portugal. The authors acknowledge the U.S. Department of Energy, Office of Energy
Efficiency and Renewable Energy through its Wind & Water Power Program for funding the
research presented in this report under contract DE-AC02-06CH11357.
Argonne National Laboratory, September 30, 2011.

ix

ACRONYMS
AGC

automatic generation control

CAISO
CTE
CVAR

California Independent System Operator

conditional tail expectation
conditional value at risk

DA
DOE

day-ahead
U.S. Department of Energy

ED
ERCOT
EWITS

economic dispatch
Electric Reliability Council of Texas
Eastern Wind Integration and Transmission Study

GHG
GW

greenhouse gas
gigawatt(s)

ISO
ITL

independent system operator

information theoretic learning

KDE
KDF

kernel density estimation

kernel density forecast

LMP

locational marginal prices

MAE
MCC
MEE
MIBEL
MISO
MLP
MPC
MPCC
MSE
MW

mean absolute error

Maximum Correntropy Criterion
Minimum Error Entropy
name of the Iberian electricity market
Midwest Independent System Operator
multilayer perceptron
Minimum Penalty Costs
Maximum Parametric Correntropy Criterion
Minimum Square Error
megawatt

NERC
NW
NWP
NYISO

North American Electricity Reliability Corporation

Nadaraya-Watson
numerical weather prediction
New York Independent System Operator

pdf
pmf
PPA

probability density function

probability mass function
power purchasing agreement

QC
QR

quantile-copula
quantile regression

RAC
RT
RTO

reliability assessment commitment

SCADA
SCED
SCUC
SO
Std

supervisory control and data acquisition

security-constrained economic dispatch
security constrained unit commitment
system operator
standard deviation

UC

unit commitment

W2P
WGENCO
WPF

wind-to-power
wind generation company
wind power forecasting

real time
regional transmission organization

xi

This page intentionally blank.

xii

SUMMARY
The rapid expansion of wind power gives rise to a number of challenges for power system
operators and electricity market participants. The key operational challenge is to efficiently
handle the uncertainty and variability of wind power when balancing supply and demand in ths
system. In this report, we analyze how wind power forecasting can serve as an efficient tool
toward this end. We discuss the current status of wind power forecasting in U.S. electricity
markets and develop several methodologies and modeling tools for the use of wind power
forecasting in operational decisions, from the perspectives of the system operator as well as the
wind power producer. In particular, we focus on the use of probabilistic forecasts in operational
decisions. The main findings from each of the chapters are summarized below.
Statistical algorithms based on computational learning are typically used to convert weather
forecasts and observational data to wind power point forecasts. In Chapter 2, we test different
training criteria for a neural network applied to wind power point forecasting. We argue that the
best choice of training criterion depends on the forecast user and the intended application of the
forecast. In a case study with data from the Iberian electricity market, we illustrate the
conflicting objectives between wind power producers and system operators, and how this
conflict may influence the choice of training criteria for wind power forecasting. In the Iberian
market, there is an assymmetry in the imbalance prices, and this gives the wind power producers
an incentive to base their market bids on a biased forecast. However, such a strategy would give
rise to larger deviations between scheduled and delivered energy, which is not in the interest of
the system operator. By testing the traditional minimum square error criterion against several
entropy-based criteria from information theoretic learning, we find that the maximum parametric
correntropy criterion in this case gives results that are a compromise between the objectives of
the system operator and wind power producer. We conclude that it is important for system
operators to have their own unbiased forecast for reliability purposes.
We analyze the current status of wind power forecasting in U.S. electricity markets in Chapter 3.
Most independent system operators (ISOs)/regional transmission operators (RTOs) in the
United States have relatively limited experience with wind power forecasting. Still, wind power
forecasting is already used for several important applications with a focus on maintaining
reliability. As the wind power capacity is rapidly increasing, we find that there is a need to better
integrate wind power forecasting into different parts of electricity market operations. Wind
power forecasting can play an important role in the determination of operating reserve
requirements, day-ahead scheduling and market clearing, reliability commitments, and real-time
dispatch. At the same time, from a regulatory perspective it is also important to design electricity
markets that give market participants the opportunity and right incentives to provide their
unbiased forecast information through their scheduling and bidding decision. ISO/RTOs and
market participants will need to develop new tools and procedures to make efficient use of the
forecast information with the overall goal to make better operational decisions under the
increased uncertainty and variability from wind power and other sources of renewable
generation. A key challenge is therefore to improve decision making under uncertainty and the
understanding of the impact of uncertainty on operational decisions. At the same time, it is
important to continue the improvements of wind power forecasting models and to better tune

xiii

them to the specific needs of the forecast users. From the perspective of system/market operators,
it is particularly important to improve predictions of forecast uncertainty and ramping events.
We investigate the representation of wind power forecasting uncertainty in the unit commitment
problem in Chapter 4. Traditional deterministic unit commitment models use a point forecast for
wind power output. In contrast, we propose a stochastic alternative that captures forecast
uncertainty by using scenarios that capture cross-temporal dependencies in the predicted wind
power. We conduct a comparison of a diversity of unit commitment strategies on a test power
system with ten thermal units and 20% wind power penetration. The results indicate that
representing wind power forecasting uncertainty with wind power scenarios combined with
stochastic unit commitment has advantages over the traditional deterministic approaches.
Moreover, the stochastic model provides a rational and adaptive way to provide adequate
spinning reserves at every hour, as opposed to increasing reserves to predefined, fixed margins
that cannot account for the systems costs or assumed risks. However, the results also indicate
that the scenarios do not capture the full range of wind power forecast uncertainty, and that it is
prudent to impose a certain level of additional reserves also with stochastic unit commitment.
A set of scenarios is a convenient way to represent wind power uncertainty in decision problems
with inter-temporal constraints. However, an extremely large set of elementary scenarios is
needed to obtain an accurate representation of the probability density function covering the
complete forecasting space. The size of this set is usually incompatible with complex algorithms
such as the stochastic unit commitment problem, which is time demanding and requires high
computational effort. In Chapter 5, we propose a methodology that can substitute a large scenario
set by a smaller set of clusters. Each cluster is replaced by a representative scenario associated
with the probability of the scenarios it represents. The reduction capability depends on the
admitted tolerance, the distance metric used to define the clusters, and the dispersion of the
original scenarios. The choice of the metric and the admissible tolerance should be defined in
accordance with the decision problem, and the level of accuracy needed for the results. The
method not only produces a set of focal or representative scenarios, but also orders them by
probability value. This approach allows retaining a selected number of scenarios and
understanding the risk of missing an adequate representation due to truncating the original
scenario set. Furthermore, the necessary number of scenarios to satisfy a maximum level of risk
can be chosen. This work, therefore, is a contribution to facilitating the adoption of stochastic
optimization models in operational planning, for instance, to use as input to the stochastic unit
commitment algorithms such as the one presented in Chapter 4.
In Chapter 6, we investigate in more detail the potential use of probabilistic wind power
forecasting in the commitment of energy and operating reserves. Such forecasts can provide
scenarios as input to a stochastic unit commitment or, alternatively, can be used to derive
dynamic operating reserve requirements for use in deterministic unit commitment. One
advantage of dynamic operating reserves is that this approach is better aligned with current
operating procedures. Furthermore, the deterministic unit commitment model has a much lower
computational burden. However, dynamic reserves do not capture the effect of inter-temporal
ramping events. Moreover, this approach does not consider uncertainty and its cost in the
objective function. In contrast, stochastic unit commitment does address inter-temporal
variability through the scenario representation of uncertainty. Furthermore, the total cost,
including the expected cost of scarcity, is explicitly taken into account in the objective function.
However, the switch to stochastic unit commitment involves a more radical departure from
xiv

current practice, it may run into computational constraints in large systems, and the benefits in
terms of cost savings may be limited. In fact, our numerical results from the 10-unit test case,
and also from a more realistic case study of the power system in Illinois, show that dynamic
operating reserves and stochastic unit commitment give similar results in terms of cost and
reliability although there may be distinct differences in commitment and dispatch decisions.
Furthermore, we find that scenario reduction algorithms, such as the one presented in Chapter 5,
tend to lower the variance in the reduced scenario set. In turn, this result reduces the amount of
hedging built into the stochastic unit commitment solutions giving rise to a need for additional
reserves. Finally, when comparing the use of different probabilistic forecasts, we find that
forecasts based on kernel density estimation tend to give better operational results than forecasts
from quantile regression, possibly due to better forecast calibration.
Future work is likely to improve the quality of probabilistic wind power forecasts, both in terms
of the estimation of probability distributions and in the generation of adequate scenarios.
However, there will always be inaccuracy in probabilistic forecasts, just as in point forecasts.
This fact calls into question the overall risk paradigm under which the system operators
scheduling decisions are made. In stochastic unit commitment, where the objective is to
minimize the expected cost, one can adjust the risk level by changing the costs of unserved
energy and reserves. However, other approaches based on different decision paradigms such
as utility theory, value at risk, robust optimization, or regret should also be considered.
Ultimately, the optimal decision strategy does not only depend on the risk preferences of the
system operator, but also on the quality of the probabilistic forecast.
Finally, in Chapter 7 we investigate the potential use of probabilistic forecasting for wind power
producers. We describe a model for optimal trading of wind power in day-ahead electricity
markets under uncertainty in wind power and prices. The model considers settlement
mechanisms in markets with locational marginal prices, where wind power is not necessarily
penalized from deviations between day-ahead schedule and real-time dispatch. We use utility
theory and conditional value at risk to represent the risk preferences of the wind power producer.
The model is tested on data from a hypothetical wind farm located in the state of Illinois
combined with historical price data. The results demonstrate that probabilistic wind power
forecasts can play an important role in guiding wind power trading decisions in the day-ahead
market. The results also indicate that price forecasts are important in order to derive good trading
strategies, particularly under market designs without penalties for deviations between day-ahead
schedule and real-time delivery. Furthermore, risk preferences clearly influence the optimal
trading decisions, with risk averse owners bidding less energy into the day-ahead market to
reduce their exposure to high real-time prices. The decisions are also highly dependent on market
design, and the results show that a deviation penalty drives the optimal day-ahead bid closer to
the point forecast for all strategies. This is in the interest of the system operator, because it makes
the task of balancing supply and demand in the system an easier one. However, deviation
penalties also add a significant financial burden on wind power producers and therefore reduce
the profitability of wind energy. Revising market rules to give efficient operational incentives
without adding an undue burden on wind power and other renewables will clearly be a challenge
as we move forward. Overall, we conclude that stochastic models will be important tools to
control the trade-off between risk and return for wind power producers participating in electricity
markets. Improved scheduling decisions and better risk management will increase the viability of

xv

wind power in the long run, and our analysis shows that wind power forecasting can serve as an
important tool to achieve this goal.

xvi

INTRODUCTION

Driven by increasing prices for fossil fuels and concerns about greenhouse gas (GHG) emissions,
wind power, as a renewable and clean source of energy, is rapidly being introduced into the
existing electricity supply portfolio in many parts of the world. The U.S. Department of Energy
(DOE) has analyzed a scenario in which wind power meets 20% of the U.S. electricity demand
by 2030, which means that the U.S. wind power capacity would have to reach more than
300 gigawatts (GW) [1]. The European Union is pursuing a target of 20/20/20, which aims to
reduce greenhouse gas (GHG) emissions by 20%, increase the amount of renewable energy to
20% of the energy supply, and improve energy efficiency by 20% by 2020 as compared to 1990
[2]. Meanwhile, China is the leading country in terms of installed wind capacity, and had 45 GW
of installed wind power capacity out of about 200 GW on a global level at the end of 2010 [3].
The rapid increase in the penetration of wind power into power systems introduces more
variability and uncertainty in the electricity generation portfolio, and these factors are the key
challenges when it comes to integrating wind power into the electric power grid. Wind power
forecasting (WPF) is an important tool to help efficiently address this challenge, and significant
efforts have been invested in developing more accurate wind power forecasts. In this report, we
document our work on the use of wind power forecasting in operational decisions.
1.1 Wind Power Forecasting
A WPF system, as illustrated in Fig. 1-1, uses input data from different sources, including results
from numerical weather prediction (NWP) models, local meteorological measurements,
supervisory control and data acquisition (SCADA) data describing the real-time (RT) state of the
wind power plants, and additional information about the characteristics of the wind power plants
and the nearby terrain and topography. WPF models are typically based on a combination of
physical and statistical models. The physical approach describes the physical relationship
between atmospheric conditions, local topography, wind speed, and the output from the wind
power plant. In contrast, the statistical approach estimates a statistical relationship between all
the relevant input data and the resulting wind power generation without considering the physics
of the system. The performance of wind power forecasts and the forecast accuracy depends on
the availability of good NWP forecasts, the complexity of the terrain, and the availability of RT
weather and wind power plant data. Hence, there can be large differences in forecasting errors
between wind power plants at different locations. WPF systems typically produce forecasts for a
time horizon up to 23 days ahead in time. In general, the forecasting error increases with the
forecast horizon.
There are two common forms of delivering the wind power forecast: point forecast and
uncertainty forecast. Through the course of this project, we have developed and tested improved
statistical algorithms for both types of forecasts, as briefly outlined below. We provide an
extensive overview of the state-of-the-art in WPF in [4]. A complete documentation of our work
on improved statistical WPF algorithms is available in [5].

Wind Power Forecast

NWP Data
Physical Models
Weather Data

Wind Power Plant Data

Statistical Models

Fig. 1-1 Illustration of WPF system.

1.1.1

Wind Power Point Forecasting

A point forecast represents a single value for each look-ahead time horizon, which contains the
conditional expectation at each time step. The point forecast estimation is typically based on
statistical training criteria (e.g. by minimizing the square error [MSE]). We have analyzed the
impacts of adopting different statistical training criteria for point forecasting models, focusing on
the comparison of different forecasting error metrics. Our study, documented in [5], showed a
significant degree of dependence on the training criterion applied, also pointing out the
importance of online training. Specifically, the application of information theoretic learning
(ITL) training criteria in wind power point forecasting, using neural networks, was tested for two
wind farms located in the U.S. Midwest. The advantage of using such metrics over the MSE
criterion was confirmed in these experiments. However, a point/deterministic forecast cannot
provide information about the dispersion of observations around a predicted value. Moreover, an
expectation cannot help decision makers in calculating risks or the conditional values at risk.
Therefore, it is essential to generate, together with (or as an alternative to) point forecasts, a
representation of the wind power uncertainty by a description of the probability density function
(pdf).
1.1.2

Wind Power Uncertainty Forecasting

Uncertainty forecasts estimate the marginal distributions for each time step, for example, in the
form of intervals, quantiles, or scenarios, which represent information about the uncertainty.
Such forecasts can help forecast users in their decision-making processes (e.g., enabling
derivation of advanced strategies for market participation). Wind power uncertainty comprises
three key features: model chain, uncertainty representation, and time-adaptive models. A
traditional model chain consists of using the forecast errors of point forecasts from a
deterministic WPF model as input to the uncertainty estimation. The wind power uncertainty
representation is determined by the end-users requests and the nature of the decision-making
problem. It can take the form of probabilistic forecasts, risk indices, or scenarios for short-term
wind power generation (Table 1-1). In general, one can only talk about more or less adequate
uncertainty representations, rather than better or worse ones. However, the pdf representation is
the most complete and flexible, enabling a range of different decision-making problems in the
power system domain. By using a time-adaptive model, the forecasting system is able to cope
with non-stationary data and, therefore, to adapt to the new situation without requiring additional
offline training of the model.

Table 1-1 Different types of uncertainty representation in WPF.

Uncertainty Representation
Quantiles
IntervalForecasts
ProbabilityMassFunction
ProbabilityDensityFunction
MeteoRiskIndex
PredictionRiskIndex
ScenarioswithTemporalDependency
ScenarioswithSpatial/Temporal
Dependency

Probabilistic
RiskIndices
ScenariosofGeneration

Our uncertainty forecasting studies, documented in [5], tested two new kernel density forecast
(KDF) methods applied to the wind power uncertainty problem in both offline and time-adaptive
modes: the Nadaraya-Watson (NW) estimator and quantile-copula (QC) estimator. Both the
Eastern Wind Integration and Transmission Study datasets, as well as two Midwest wind farms,
were used in different case studies and were compared to linear and splines quantile regression
(QR) methods, which is the current state of the art. The results show that the KDF and QR
methods have similar performance in terms of sharpness, resolution, and skill score, although the
former lead to a better calibration. In addition, we found that time-adaptive approaches for both
NW and QC change the bias of the probabilistic forecasts (i.e., calibration), while only slightly
changing the sharpness and resolution. Moreover, the time-adaptive mode improves the skill
score when compared to the offline approach.
An alternative way of presenting the wind power uncertainty is by using a discrete set of
scenarios, sampled according to a probability density function associated with the forecasts, and
therefore able to characterize the uncertainty according to the historical error distribution.
Another advantage of a scenario set is that it can include the temporal dependency information of
the errors. However, in order to capture the entire characteristic of the wind power forecasted
during the prediction period accurately, a decision maker may need to evaluate a high number of
scenarios. This is a time-consuming and computationally demanding task and the
computational burden usually is not compatible with stochastic programming algorithms run in
useful time. Therefore, there is typically a need to select only a small number of scenarios that
can represent the whole scenario set at best. We explore scenario reduction algorithms and their
application in the power system domain in more detail in this report.
1.2 Wind Power Forecasting in Operational Decisions
Different agents in the electricity market may have conflicting interests as they interact in the
market environment. Wind power producers typically aim at minimizing their forecasting error
over a longer period of time, in order to better schedule their generation. However, their main
concern is in maximizing income. System operators, on the other hand, are concerned with
guaranteeing the security of the system. System operators are interested in minimizing
operational costs while maintaining a high level of reliability. This implies that the choice of a
WPF model is not neutral and, in some cases, the definition of what constitutes a good wind
power forecast might differ between forecast users. For a long time, a good forecast has been
considered to fit predictions closest to the observed values, in what is known as the forecasters

paradigm. This idea has given place to the forecast consumers paradigm, which states that
forecasts should be tuned toward the users interests. With this (new) relative concept, we can
assess the forecast either in terms of consistency (correspondence between forecasters
judgment), quality (correspondence between forecasts and observations), or value (incremental
benefits due to implementation into decision-making process). We focus on system operators and
wind power producers as two distinct and different WPF users in this report.
1.2.1

System Operators

From the perspective of a system operator, its concern in guaranteeing the security of the system
is closely related to WPF uncertainty. In fact, large unexpected changes in wind power output
over a short period of time is one of the major issues in power system operations with wind
power. In the presence of such ramping events, system operators have to develop operational
procedures in order to satisfy the load and maximize both the economical and environmental
benefits. The longer the time ahead of such events that they can be predicted, the more options a
system operator has to balance the system. However, forecast uncertainty generally increases
with the forecast horizon. A key challenge for the system operator is therefore to handle the
uncertainty and variability from wind power in a cost-efficient manner without compromising
system reliability. System operators will need to adopt new methods to better handle wind power
and other renewable resources in their scheduling and dispatch decisions. In this study, we tackle
this problem from two ways. One way is to extend the traditional operating reserve strategies,
that is, to schedule appropriate amounts of reserve capacity based on information from wind
power uncertainty forecasting. Another way is to rethink the formulations of the power system
scheduling and market clearing problems. We explore a stochastic formulation of the unit
commitment (UC) problem and analyze to what extent stochastic programming approaches can
deal with the wind power uncertainty.
1.2.2

Wind Power Producers

From the perspective of a wind power producer, it is becoming increasingly important to manage
the risk of participating in the electricity market. A majority of the wind power in the
United States is sold under long-term power purchasing agreements (PPAs), which lock in the
price of delivered electricity and thereby stabilize the revenue stream. However, a significant
share of wind power is being sold directly to the electricity market. It is obviously important for
merchant wind generation to apply scheduling and trading strategies that reflect the uncertainty
in wind power and prices and the corresponding trade-off between risk and return. Buyers of
PPAs also have to schedule the contracted wind power into the electricity market and must
consider how wind power influences the scheduling strategy of the rest of their portfolio.
Trading of wind power in electricity markets is still a relatively new problem. In this report, we
investigate the scheduling and trading of wind power focusing on electricity markets in the
United States with locational marginal prices (LMPs), including settlement procedures and their
influence on trading decisions. Moreover, we develop an optimization model for wind power
trading under uncertainty. The model can find the optimal wind power bid under uncertainty in
forecasts for wind power and prices (day-ahead and real-time, considering the risk preferences of
the wind power producer and the impact of potential deviation penalties.

1.3 Report Outline

This report documents our research on the application of wind power forecasting for both system
operators and wind power producers. The report is organized as follows: Chapter 2 reports a
study of WPF training criteria and the goodness of a forecast, focusing on the electricity
market in Spain and Portugal; in Chapter 3, we discuss the current use of WPF in
U.S. independent system operator (ISO)/regional transmission operator (RTO) markets;
Chapter 4 presents our implementation of WPF uncertainty in a stochastic formulation of the UC
problem, as well as a comparison among different UC strategies; Chapter 5 presents a new
clustering-based scenario reduction method for generation of adequate WPF scenarios for use in
stochastic programming models; in Chapter 5, we present a comparison between different
probabilistic forecasting and scenario reduction methods and test the resulting forecasts on the
stochastic UC model from Chapter 4; Chapter 7 describes a new model for optimal trading of
wind power in day-ahead electricity markets under uncertainty in wind power and prices,
considering settlement mechanisms in the electricity markets with LMPs; finally, Chapter 8
summarizes the overall conclusions from our work on the use of WPF in operational decisions.
Note that for many of the chapters, the content has been published in conference proceedings or
journal papers. References to underlying publications are provided in footnotes at the beginning
of each chapter.

This page intentionally blank.

GOOD OR BAD WIND POWER FORECASTING: A RELATIVE CONCEPT 1

This chapter reports a study on the importance of the training criteria for WPF and calls into
question the generally assumed neutrality of the goodness of particular forecasts. The study,
focused on the Spanish electricity market as a representative example, combines different
training criteria and different users of the forecasts to compare them in terms of the benefits
obtained. In addition to more classical criteria, an Information Theoretic Learning (ITL) training
criterion, called parametric correntropy, is introduced as a means to correct problems detected in
other criteria and to achieve more satisfactory compromises among conflicting criteria, namely
forecasting value and quality. We show that the interests of wind farm owners may lead to a
preference for biased forecasts, which may be in conflict with the larger needs of secure
operating policies. The ideas and conclusions are supported by results from three real wind
farms.
2.1 Introduction
The demand for more accurate, short-term WPF models has led to solid and impressive
development in recent years [4][6]. Increasing the value of wind generation by improving WPF
systems performance and furthering its integration into operational management is one of
the priorities in current research [7]. On many past occasions, energy market participants have
taken for granted that a good forecast would be something that turns out to be as close to the
actual values as possible. This view is called the forecaster paradigm, where the concern is to fit
the prediction to the observed values in the signal processing sense. A good example of a WPF
evaluation protocol in agreement with this paradigm is [8]: the main concern is minimizing the
forecasting error, and no importance is given to the nature and consequences of the errors.
However, it may be argued that the definition of the goodness of a forecast should be strongly
related to the use given to such a forecast. In the forecasting pilot project by the Alberta Electric
System Operator [9], an important idea is that a forecast should be oriented by the users
interests. Forecasting errors may be seen through many perspectives (e.g., mean, bias, maximum
deviation, phase error, etc.), and the nature of the errors may be too broad one particular
forecast is not likely to be considered optimal simultaneously in distinct applications, such as
market participation or power system operation.
This situation has been recognized in the domain of weather forecast. Several publications
discuss what a good or bad forecast is. Murphy [10] distinguished three different types of
goodness in forecasts: (1) the correspondence between forecasts and forecasters judgments
(consistency); (2) correspondence between forecasts and observations (quality); and (3) the
incremental benefits (economic/or other) when employed by users as an input into their decisionmaking processes (value). The factor of consistency in WPF is not discussed in this chapter: it is
assumed to be included by the forecaster in the model during the development phase. In order to
assess the quality of WPF, the standard approach is based on statistical error measures
(e.g., mean absolute error) [8]. However, this approach does not guarantee that the forecast with
the best score is the one with best quality. An alternative approach, called verification
framework, was presented by Murphy and Winkler [11]. This approach is based on the joint
distribution of forecasts and observations and factorization of this distribution into conditional
1

This chapter is reproduced from Wind Energy, Vol. 14, No. 5, R.J. Bessa, V. Miranda, A. Botterud, J. Wang,
Good or Bad Wind Power Forecasts: A Relative Concept, pp. 625636, 2011, with permission from Wiley.

and marginal distributions. Pinson [12] used this framework, named the distribution-oriented
approach, and applied it to WPF. His objective was to study the main characteristics of WPF
uncertainty.
The third type of forecast goodness, forecast value, is often very difficult to quantify, as in some
cases it is related to non-economic factors. However, some decision makers may give a high
relevance to these criteria [13]. Some researchers have reported incremental economic benefit
from the use of WPF in the participation of wind power in electricity markets. Usaola et al. [14]
quantified the economic advantages of using an advanced forecasting approach that takes as
input numerical weather predictions (NWPs) instead of persistence. Barthelmie et al. [15]
estimated the wind farm size for which WPF outweighs the cost of installing or implementing
short-term forecasts. The authors conclude that the economic benefit of using WPF depends on
the accuracy and cost of purchasing the forecast. Angarita-Mrqueza et al. [16] estimated the
income obtained by a wind farm in the Spanish and British markets by using an advanced
forecasting system (Siprelico) compared to persistence. In the British market, the Siprelico
system produced results similar to persistence because the period between gate closure and
actual delivery was very short. For the Spanish market, because the period between gate closure
and delivery was longer, Siprelico improved the income amount(s) when compared to
persistence.
Pinson et al. [17] studied the use of WPF probabilistic forecasts to support the participation of
wind generation in the electricity market. The authors showed that with the information about
uncertainty, it is possible to derive advanced strategies for market participation leading to a
significant increase in income when compared to persistence and point forecasts obtained with
fuzzy-neural networks. Fabbri et al. [18] proposed a probabilistic methodology for estimating
the WPF error costs in the electricity market. Other related works can be found in [19][21].
One crucial aspect is that the relationship between quality and value is nonlinear and differs from
problem to problem, as well as from user to user [22]. Hence, it is important to recognize that
there are different users for WPF, the two most important being wind generation companies
(WGENCOs) that bid their wind generation into the electricity market and system operators
(SOs) that must guarantee the security of the system.
The output of a forecasting process or model can therefore be evaluated according to two
paradigms, which we will call the forecaster paradigm and the forecast consumer paradigm. The
former is mainly concerned with the time series in itself, in the signal processing sense, and is
therefore related with quality; the latter focuses on the use given to the forecasts and is therefore
related to value.
The main idea driving this chapter is to promote the recognition (many times forgotten) that
distinct user groups may have conflicting interests in a restructured market environment, and
therefore the choice of a model is not neutral. For instance, SOs are interested in minimizing
operational costs while maintaining a high level of reliability. In contrast, WGENCOs are mainly
interested in maximizing their income levels in the electricity market. Hence, in some conditions,
the definition of a good forecast varies with different forecast users.

This chapter contributes to the discussion of this topic by comparing the forecast errors and
economic consequences of adopting several criteria for the statistical training of WPF models.
To illustrate the relevance of the concepts discussed, the chapter presents a case study of the
Iberian day-ahead (DA) electricity market (MIBEL). The study could be reproduced for other
markets; however, the objective here is not to study and compare markets but to discuss models,
and the example presented is sufficiently representative. Our main objective is not to provide an
alternative to the approaches based on probabilistic forecasts [17][21] but rather to highlight a
new view of the relative importance of the forecast error in a multi-criteria and multi-perspective
paradigm, in which biased point forecasts could be produced to increase the market income, and
to stress the conflicting objectives that may exist between different users (or stakeholders). To
compensate for some of the problems identified and analyzed, we introduce parametric
correntropy, which is a new training criterion for WPF that aims to arrive at a satisfactory
compromise between forecast value and quality.
We first discuss the WGENCOs and SOs perspectives on WPF. Next, on the basis of actual DA
market data from MIBEL, we illustrate the conflict between the viewpoints and also the
applications of different training criteria. Finally, we analyze the consequences of the different
training criteria for the two WPF users.
2.2

The Forecast Consumer Paradigm

2.2.1

Wind Generation Companies Viewpoint

In a restructured market environment, forecast errors may have an associated cost or penalty, as
the wind power producer usually ends up supplying an amount of energy that differs from the
market bid. The market remuneration scheme adopted in this chapter is for operating in the DA
market and is analogous to the one formulated in [17][21].
In this mechanism, wind power producers offer energy quantity bids (Eb) at day D (typically till
12:00 noon) for every hour of day D+1 in the DA market, and they are paid at the market
clearing price (ps). Then, if the actual wind generation is above or below their bid, they will be
subject to a penalty as follows:
-

If the actual generation (Ep) is above the market bid, the excess is paid at a discounted
price (psurplus). This price can also be negative, which represents a cost.

If the actual generation remains below the market bid, a penalty is applied according to
the price (pshortage) of the generation that the SO has to purchase to compensate for the
insufficient generation.

Each market specifies its own rules for the calculation of deviation costs, and therefore, no
general conclusion can be offered. However, it seems obvious and this chapter confirms this
finding that if the penalties are asymmetric, then an opportunity arises for a gambler to benefit
from using a biased model instead of a neutral model.
A wind farms income (I) from selling for a given look-ahead time t+k can be formulated as:

(
(

)
)

p s E b + ptsurplus
Etp+k Etb+k , Etp+k > Etb+k
+k
I t +k = st +k bt +k
shortage
Etb+k Etp+k , Etp+k < Etb+k
pt +k Et +k pt +k

(2-1)

This formulation can be rearranged by considering the balancing prices, cdown_reg=ps-psurplus and
cup_reg = pshortage-ps, as:

_ reg
p s E p ctdown
Etp+ k Etb+ k , Etp+ k > Etb+ k
+k
I t + k = t +sk t +pk
up _ reg
b
p
p
b
p t + k E t + k ct + k E t + k E t + k , E t + k < E t + k

(2-2)

It is important to emphasize that in this market formulation, the WGENCOs are appointed
balance responsibility; hence, they have a financial responsibility for any imbalance.
In some markets, if the deviation helps the SO balancing measure, then the market agent is not
penalized. Note that the intra-day markets are not taken into account but may reduce the forecast
error and consequently the deviations considerably.
From the WGENCO viewpoint, the value of the WPF in the market is translated by the
additional income provided by such forecasts over the income obtained with an alternative
method. The first component of (2-1) is constant and corresponds to the income obtained with
perfect forecasts. Hence, maximizing the market income is analogous to minimizing the expected
costs of balancing (second component in [(2-2)]). Therefore, in order to meet its goal, the
WGENCO will follow a particular bidding strategy depending on the market balancing costs.
Because of the uncertainty in prices and wind generation, resolving this matter is a decision
problem under uncertainty. Several decision rules can be used for finding the preferred bid. The
decision rule used most often is to find the bid Et+kb that minimizes the expected balancing costs
(i.e., to maximize expected income). This rule was first introduced by Bremnes [23] and has been
used by authors in [17] and [21]. This decision rule describes a risk-neutral attitude (i.e., a linear
utility function). Bourry et al. [20] presented an alternative strategy based on portfolio theory
that balances between the expected value and the risk of the balancing costs distribution.
Botterud et al. [24] present a methodology to derive optimal DA bids for a wind power producer
under uncertainty and compare the results of different bidding strategies (e.g., maximizing the
utility). The authors discussed how the optimal bids depend on the electricity market design.
Under the market assumptions outlined above, the optimal bid of the expected value decision
rule possesses an analytical solution; for a proof see [23]. The bid that maximizes the expected
income is the cdown _ reg (cdown _ reg + cup _ reg )1 quantile of the forecasted probability distribution of wind
generation. It is found that the optimal bid depends only on the quantile proportion computed
from the balancing prices. For instance, if cdown_reg> cup_reg, the optimal quantile is above 0.5
(median), and this result constitutes overestimation; conversely, if cdown_reg< cup_reg, the optimal
quantile is below the median and the result is underestimation.
Based on this idea and as carried out by Pinson et al. [17], the forecast with higher economic
value is not necessarily the one with lower forecast error (in accordance with the forecasters
paradigm). The results presented by the authors support this idea: a point forecast with 40.55%

10

of imbalances (with respect to produced energy) achieved an income of 1.15 M over one year,
whereas an advanced strategy based on a probabilistic forecast reached a higher 55.46% of
imbalances but also an income of 1.21 M.
As mentioned before, if the deviation is in favor of the system, then the WGENCO does not pay
balancing costs under some markets rules. In [21], the bidding strategy proposed by Bremnes
[23] was extended to include this possibility, assuming that the system balance is unrelated to the
wind power imbalance or that all wind power producers follow the same bidding strategy.
Finding the decision rule and the best bidding strategy is clearly a complex problem that requires
wind power and price forecasting information. One alternative is to develop economic-oriented
training criterion for training wind-to-power (W2P) models. Ravn [25] analyzed a training
criterion that penalizes according to the balancing costs and the sign of the forecast errors:
min

J down

down _ reg

Pt +k Pt +k + wup _ reg Pt +k Pt +k

(2-3)

J up

The expression is divided into terms of overestimation, Jup ( Pt+k > Pt+k ), and underestimation,
Jdown ( Pt +k < Pt +k ); wdown_reg and wup_reg are penalization factors related to the market balancing
costs. By applying this training criterion, an economic bias is introduced, which can increase the
forecast error, but also increases the income from the market. Therefore, the economic value of a
wind power forecast can be increased by information provided by either probabilistic forecasts or
economic-oriented deterministic (point) forecasts.
2.2.2

System Operators Viewpoint

In our discussion, the viewpoint of the power SO is not linked to how the SO should make the
forecasts. In Botterud et al. [26], the current use of WPF in U.S. ISO/RTO markets is discussed,
and recommendations for how to make efficient use of the information in state-of-the-art
forecasts are offered.
In this section, the discussion is related instead to the impact on the operational policy from the
economic-oriented market bids provided by the WGENCOs. As stated in the previous section, a
WPF system that provides the maximization of income to a WGENCO should not be trusted or
used by SOs.
The SO needs to manage the combination of generation and consumption variability considering
unit outages. Therefore, in a restructured electricity market environment, the SO acquires all of
the reserve needed for the control area, in order to maintain a minimum reliability level.
In the daily market, the SO at day D is in charge of defining and contracting the operating
reserve (i.e., the reserve related to loss of load or generation surplus attributable to forecast errors
both in load and wind and unit outages (e.g., spinning and non-spinning reserve) needed
for the next day (day D+1) [27]. These operating reserve levels are generally settled by
deterministic criteria, such as the capacity of the largest unit plus an empirical formula, such as
the Union for the Coordination of Transmission of Electricity (UCTE, now ENTSO-E) rule [28].

11

However, in the literature, probabilistic approaches are proposed by several authors; see, for
example, [29] and [30].
The integration of large shares of wind generation requires an increase in the amount of
operating reserve that is needed to balance generation and load in order to maintain an acceptable
level of reliability. Results from [29] and [31] support this idea. Despite reducing wind power
variability as a result of the aggregation of several wind farms, the variability and uncertainty of
wind generation tend to be a source of stress for the operations personnel.
Because the load and generation are dispatched by the market mechanism, and assuming that
WGENCOs participate with bids in the market, the strategic bidding followed by WGENCOs
may influence significantly the needs for operating reserves and also the overall cost of
balancing the system. As mentioned in the previous section, a WGENCOs income does not
necessarily increase by reducing forecast deviations. Therefore, the forecast error should be
considered to be of secondary concern for the SO, because the relevant error in this case is
introduced by the market bids.
For instance, if the market bids overestimate the wind generation, then the load will be met by
untrue generation in the DA market. In this case, the SO with its own neutral WPF system can
forecast the wind generation uncertainty and by using, for instance, an approach that is similar to
that used in [30] to detect a high risk of not meeting the load. Hence, the SO will need to commit
more generating units (e.g., using the reliability assessment commitment to commit more units
[26]) or to contract more upward reserve, which increases the operational costs.
The need for upward reserves is the most critical situation, even for planning the power system
capacity, as it requires that traditional power station capacities must be available as operating
reserves. This situation may limit the ability of wind power plants to replace the conventional
power station capacities.
Of course, this system balancing cost depends on the generation mix (e.g., the reserve provided
by gas turbines or hydro is less expensive) and on the operating strategy [32]. For instance, in
cases where the SO defines the reserve requirements on the basis of deterministic criteria, the
awareness of this overestimation may lead to a highly conservative attitude and an adoption of
high safety margins to minimize the risk and will therefore heavily increase the operational
costs. In addition, as mentioned in [33], imbalances may aggravate potentially large out-of-merit
automatic generation control (AGC) costs. All of these operational costs will be part of the tariff
paid by all customers.
Nevertheless, the SO may have to shed load if it has offline units because of the expected wind
power availability (as bid into the market). It may also happen that the deviation between load
and generation cannot be met by all of the available operating reserve because of transmission
constraints.
The same behavior is valid for the case where the wind power bids are underestimated. This
problem is of less concern for the SO, because it does not need to shed load or have additional
power stations available to cover the energy deficit. However, the SO also needs to contract

12

downward reserve (e.g., reserve to deal with generation surplus due to forecast errors,
e.g., pumped storage) and may have to curtail wind generation. This situation leads to what is
called an over-commitment of conventional generation, which will reduce the economic and
environmental welfare of the power system.
At this point, the key question from an SO viewpoint is: what is a good forecast (or bid) for the
market made by WGENCOs? The UCTE NetWork of Experts on Wind Power, composed of
experts from several SOs such as Transpower (Germany) and REE (Spain), stated in [34] that the
crucial issue is the expected maximum forecast deviation, and not the mean forecast error. The
groups finding is because the extreme deviations may lead to load shedding. Consequently, for
an SO, a good criterion will be to minimize the expected maximum forecast error. However, this
training criterion may be difficult to translate to a function, so the minimum mean square error
(MSE) could be an acceptable criterion because it weights the large deviations more heavily.
However, this criterion must be viewed from the perspective of maintaining a desired reliability
standard based on energy offered in the market by the market participants, including
WGENCOs. In this case, a good WGENCO forecast for an SO should be the one that leads to:
(1) less use of reserves, in particular upward reserves; (2) a lower expected maximum forecast
error; and (3) deviations in favor of the system.
The first two criteria may lead to a conflict with the WGENCOs viewpoint. With respect to the
third item, it is important to point out that if all WGENCOs use a bidding strategy based on the
system deviation, it is probable that their deviations will be against the system deviation. Even in
a situation where forecast errors help the system deviation, the SO must (in the day before)
contract reserve and schedule the generation according to a probabilistic algorithm or
deterministic rule without knowing this information a priori.
2.2.3

The Compromise Viewpoint, or the Search for Utopia

The forecasting and market bidding analysis must be performed carefully, because we are
dealing with two generally conflicting viewpoints, as outlined above. In this case, there is no
direct application of the trade-off concept, because there is no single agent that evaluates its
gains in one criterion for giving up some value in another criterion with disjoint or partially
disjoint sets of criteria, a loss in one criterion is felt by an agent by just a loss. This analysis
pertains to the field of multiple agent decision making, where it is recognized that the Utopia
solution (i.e., one that optimizes all criteria for all agents, even those with conflicting interests
and who may not even share the same criteria) is generally an infeasible point. Utopia
corresponds to the highest level of aspiration for all agents. In contrast, the ideal solution refers
to the maximum aspiration of a single agent and is therefore an individualistic concept.
Market participants therefore seek solutions that may be seen as an acceptable compromise by
the several agents. These problems are solved either by negotiation or by arbitration (if one
excludes the use of force); in the former case, the mode is unsupervised, and in the latter, it is a
supervised mode. The relations between Utopia and the negotiation solutions have been
extensively studied: the Nash bargaining solution (a Pareto-set solution that is the most
equitable), the Kalai-Smorodinsky solution (providing more weighting to the needier player),
and the Gupta-Liven solution (a Pareto-set solution in the line connecting the reference or focal
point solution with Utopia) are just examples [35][36].
13

In our study, this function is not carried out through a negotiation process but by adopting the
concept of a compromise viewpoint through the external assumption of what a good
compromise should look like this concept resembles the arbitration process, where an external
independent agent with supervisory powers (e.g., a regulator) may decide on the characteristics
of compromises among agents by considering criteria (or utility functions) pertaining to all
agents in a joint framework.
It is natural to measure the forecast value relative to the income obtained by the market
participant. However, the optimal forecast for a WGENCO may not be desired by the SO,
because this agent considers, in its own evaluation of solutions, distinct criteria from those that
the WGENCOs consider, namely, costs that are not included in the deviation costs of (2-2): for
instance, cost of energy not supplied, contracted reserve capacity, and environmental costs. The
aim in our approach is to offer a WPF training criterion function that suggests an acceptable
compromise between the two viewpoints. In the following section, several training criteria
described in the literature are presented.
It must be added, however, that an alternative exists: the decoupling of the problem that is,
when the distinct agents work with separate forecasts and do not take into account, for their own
decisions, the forecasts used by the other parties. In that situation, each agent can adapt the
models and training processes to accommodate its own interests without entering into direct
conflict with the others.
2.3

Training W2P Models

2.3.1

Aspects of Training W2P Models

A W2P model is a term used to designate a neural network, a fuzzy inference system, or, in
general, any system that emulates an input-output transfer function and whose performance
depends on the tuning of internal weights or parameters, which are used to translate wind
characteristics (e.g., wind speed, direction) into electric power. We can identify three basic
modules in a W2P: its internal structure, the training criterion, and the optimization algorithm.
Three types of actions can be applied in order to constrain the training of a W2P model (see Fig.
2-1) in:

The internal structure, by modifying the number of weights;

The training criterion, by selecting an adequate measure of performance related with the
forecast error between the WP2 output (y) and target value (T) (e.g., mean square error);
The optimization algorithm, by choosing a mechanism or procedure (e.g., gradient
descent) to close a feedback loop that updates the weights as a function of the training
criterion computed for a training dataset.

This chapter addresses the aspect of training criterion.

14

Fig. 2-1 Basic arrangement of a W2P identifying its three main modules.

2.3.2

Training Criteria

The traditional MSE and the mean absolute error (MAE) are training criteria with generalized
adoption under the forecasters paradigm. Information theoretic learning (ITL) concepts
especially challenge the unquestioned adoption of MSE, because this criterion is equivalent to
minimizing the variance of the error distribution, and MSE is only an optimal criterion if this
distribution is Gaussian which is not true in most cases, especially not in WPF models.
Bessa et al. [37] first introduced the idea of exploring ITL criteria [38] to use in training neural
networks for WPF (W2P model), where two training criteria from the ITL paradigm were
considered. The first training criterion is the Minimum Error Entropy (MEE) [39]. The basic idea
is as follows: if the error distribution of the output would become a Dirac function (meaning that
all errors would be equal), we would have reached a predictor whose output would reproduce
exactly the actual data series by just adding to the results a bias corresponding to the mean of
the probability density function (pdf) of the errors (i.e., the deviation from zero). But it so
happens that the Dirac function has minimum entropy. Renyis Entropy is combined with a
Parzen Windows estimation of the error pdf to form the basis of MEE under which neural
networks are trained.
The second training criterion was derived from an ITL measure named correntropy [40].
Correntropy is a generalized similarity measure between two arbitrary scalar random variables,
X and Y, defined by:

V ( X , Y ) = E[k ( X Y )]

(2-4)

where k is the kernel function (usually Gaussian).

Correntropy is directly related to the probability of how similar two random variables are along
the line y=x in a neighborhood of the joint space defined by the kernel bandwidth , and it
provides the probability density of the event p(X=Y). The bandwidth controls the observation
window in which the similarity is assessed but makes it impossible to assess similarity in the
whole joint space. This limitation is actually good, because it leads to the rejection of outliers
whose consideration contaminates models. The training criterion consists of maximizing the

15

correntropy between the output y and the target T and was called Maximum Correntropy
Criterion (MCC). These two ITL criteria were only studied in the context of the forecasters
paradigm, and no attention was given to the value of the forecast.
Viewing the WPF problem from the forecast users paradigm, for a WGENCO, the training
criterion should be translated into economic value, which is related with profit in the market and
penalization of forecast errors. Hence, the cost function presented in (2-3) could in theory lead to
a better forecast under this paradigm. An alternative that we propose is to use the parametric
correntropy [41], which has been used so far in signal processing problems. The idea is similar
to that of correntropy; however, the comparison between X and Y is performed along the line,
aX+b=y, where a and b are parameters.
The training criteria that we use to train the W2P model (e.g., neural networks) in the WPF
problem are summarized as follows:
1. MSE Minimum Square Error. This value is the classical criterion that minimizes the
variance of the error distribution and has the form:

MSE(e ) min

1 N 1 2
ei
N i=1 2

(2-5)

where e = (Ti yi ) is the error of sample i relative to the target value Ti, and N the number
of training samples.
2. MCC Maximum Correntropy Criterion. This criterion is based on a correntropy and
may be expressed as:

1 N
k (ei )
N i =1
where k is the Gaussian Kernel with bandwidth .
MCC (e ) max

(2-6)

3. MAE Minimum Absolute Error. This criterion minimizes the absolute error without
taking into account the sign of the error:

MAE (e ) min

1
N

e
i =1

(2-7)

4. MPC Minimum Penalty Costs. This criterion minimizes the error penalties, as shown
in (2-3).
5. MPCC Maximum Parametric Correntropy Criterion. This criterion is based on
parametric correntropy and may be expressed as:

16

MPCC (e, a ) max

1
N

k (a y
i =1

Ti )

(2-8)

where a is a parameter that is defined by the forecaster or computed from the market
balancing costs, and b is set to zero because the bias of the forecast can be introduced
only with parameter a, without increasing the number of parameters of the problem.
2.4
2.4.1

Case Studies
Data Characteristics

Three real wind farms with different rated power (ranging from 20 MW to 50 MW) and situated
in different types of terrain are used as case studies. Data collected in the wind farms include
supervisory control and data acquisition registers with an average package containing
10 minutes of power delivered by the wind farms to the grid. We also have available forecasts
produced for the same period by a mesoscale weather model for mean wind speed and wind
direction for a reference point in the wind farm, with forecasting horizons ranging from 0 to
48 hours in 1-hour intervals. These data cannot be discussed in further detail for reasons of
confidentiality.
To organize the tests, the available data were divided into three sets. The first set, with several
months worth of data, was used as a training set. The second set, with one months worth of
data, was used as a validation set. And the third set, with the remaining months, was used as a
testing set for comparative purposes. Table 2-1 presents the months used for training, testing, and
validation of the neural network.
Table 2-1 Training, validation, and testing sets.

Wind Farm
A
B
C

Training
FebDec
(2007)
MarDec
(2007)
FebAug
(2007)

Validation
Jan (2007)
Feb (2007)
Jan (2007)

Testing
JanDec
(2008)
AprDec
(2008)
SepDec
(2007)

Data from the Iberian electricity market was used in this study. The data consists of hourly spot
and balancing prices from two years, 2007 and 2008, and was collected from e-sios (the
information system of the system operator of Red Elctrica de Espna, http://www.esios.ree.es).
A very complete statistical analysis of the price data used in this study can be found in [42] (see
Chapter 5). Moreover, the author analyzed the correlation between the market prices (spot and
balancing) and the wind generation for the same data used in this chapter. The author concluded
that wind power is still having only a limited impact on the spot prices in the Iberian market;
however, it has already made an impact on the balancing prices. Table 2-2 shows the annual
mean energy prices in 2007 and 2008 for the Spanish market (in /MWh).

17

Table 2-2 Annual mean market prices.

Spot Price
Up Balancing
Down Balancing

2007
39.34
2.97
7.99

2008
64.47
4.04
8.02

The balancing prices are very asymmetric, where the down balancing is more expensive. Also
evident is a considerable change in prices from one year to the next, but this variation does not
affect the results of our study what is relevant is the asymmetry in balancing prices.
2.4.2

Training Characteristics

We have trained a feed-forward multilayer perceptron (MLP) neural network with only one
hidden layer consisting of 7 neurons by using a hyperbolic tangent activation function. The
inputs of the MLP are NWP forecasted values: mean wind speed values and mean wind direction
values. Because of the cyclic character of the wind direction, this variable comprised two
components (i.e., the sine and cosine components). This means a total of three input variables,
which were standardized by using the min-max method, were used.
Because the purpose is to compare results, we do not describe details of the training here;
however, the training details can be found in [37]. Moreover, determining the best topology of
the neural network is beyond the scope of this work.
The penalization factors of the training criterion MPC (2-3) consist in average values of
balancing prices for each look-ahead time step (in this case, there are 24 time steps) computed
from year 2007 data. Hence, the diurnal cycle of the prices is included in the model. Regarding
the parameter a of the MPCC criterion, the adopted methodology is:
1. In the beginning of each training epoch, the parameter a is estimated from the training
dataset by using the average value of balancing prices of one year for each look-ahead
time step as penalization factors of underestimation (wdown_reg) and overestimation
(wup_reg). The training criterion is:
k ' ((a yi Ti ) wdown_ reg ), yi < Ti
max
a
k ' ((a yi Ti ) wup _ reg ), yi > Ti

(2-9)

where is the kernel bandwidth and depends on the size of the penalization factors.
2. After estimating the parameter a, the following training criterion is maximized:

max k (a ' yi Ti )

(2-10)

where is a kernel bandwidth much smaller than , and a is equal to 1/a.

18

The kernel size used by MCC is the same size used in [37]. The kernel size in MPCC was
found in the validation set to be equal to 0.1, and was equal to 3. As expected, the kernel size
does not vary widely from one wind farm to another. Yet, presents a major importance in this
problem, because small values indicate small sensitiveness to asymmetric costs. What is desired
is a value that gives importance to the asymmetric costs but that also maintains a balance
between over- and underestimation. The same values are used for the three wind farms since
what is desired in WPF is to train a neural network for any wind farm without concern about
tuning parameters.
2.4.3

Testing Methodology

The neural network was used to predict the power pt+k|t produced by the wind farm at time stamp
7:00 GMT (when new NWP predictions become available) for each look-ahead t+k of the next
day. The wind power prediction was performed for each day of the test dataset. We have taken
each wind farm and simulated its participation in the electricity market in the year 2008, offering
the power prediction generated by prediction systems trained with different training criteria and
using the hourly spot and balancing prices from 2008.
2.4.4

Analysis of the Results

In this section, we present results for training of neural networks in three real wind farms and one
electricity market, when training under the different criteria. Table 2-3 through Table 2-5
summarize the results obtained from the participation of wind farms A, B, and C in the Spanish
market with bids equal to the forecasts provided by each training criterion. Q95% is the quantile
95% of the overestimation errors distribution, which means that the probability of having an
upward deviation larger or equal to this value is 5%; the conditional tail expectation (CTE) of
this quantile can be interpreted as the expected maximum forecast deviation.
Table 2-3 Simulation of wind farm As participation in the Iberian market.

MSE
Surplus [% of produced energy]
33.79
Shortage [% of produced energy]
33.74
Total deviations [% of produced energy]
67.54
Deviations against the system [% of produced energy] 30.09
Q95% [% of rated power]
41.74
CTEa [% of rated power]
53.67
Total Income (deviation from the best) [k]
-10.16

MCC MAE
35.55 39.08
29.64 24.37
65.21 63.47
28.68 28.21
41.70 43.58
58.05 57.89
-7.61 -12.47

MPC MPCC
19.71 30.89
58.07 35.56
77.79 66.46
34.97 29.02
61.50 48.30
73.28 63.26
-4.1
0.0

CTE = conditional tail expectation.

Table 2-4 Simulation of wind farm Bs participation in the Iberian market.

Surplus [% of produced energy]

Shortage [% of produced energy]
Total deviations [% of produced energy]
Deviations against the system [% of produced energy]
Q95% [% of rated power]
CTEa [% of rated power]
Total Income (deviation from the best) [k]
19

MSE
34.09
25.16
59.25
26.33
28.19
34.21
-20.09

MCC
34.13
24.14
58.28
26.10
30.22
40.27
-20.43

MAE
39.29
19.67
58.96
26.41
26.98
34.78
-30.92

MPC
18.21
48.31
66.52
28.83
40.67
49.78
0.0

MPCC
25.85
34.37
60.23
26.49
32.92
42.13
-7.45

CTE = conditional tail expectation.

Table 2-5 Simulation of wind farm Cs participation in the Iberian market.

Surplus [% of produced energy]

Shortage [% of produced energy]
Total deviations [% of produced energy]
Deviations against the system [% of produced energy]
Q95% [% of rated power]
CTEa [% of rated power]
Total Income (deviation from the best) [k]
a

MSE
34.46
19.71
54.17
23.07
31.08
38.33
-14.2

MCC
36.91
16.09
53.00
22.88
34.00
44.42
-15.35

MAE
37.63
15.02
52.64
23.14
38.34
46.88
-16.59

MPC
19.58
33.12
52.70
20.54
46.07
52.42
0.0

MPCC
24.62
27.42
52.04
21.13
39.44
48.21
-4.78

CTE = conditional tail expectation.

Wind Generation Companies Viewpoint

The first three training criteria are not economically oriented. One may see that wind farm A
realized a lower level of income when using them than it did with MPC and MPCC. The
comparison of the results from MPC with the first three training criteria (which are typical of the
forecaster paradigm) illustrates that a forecast of worse quality (i.e., larger deviations) results
nevertheless in higher income for a WGENCO. By comparing MCC with MPCC, it is evident
that accepting a small increment in the error leads to an increase of around 7,600 in revenue. In
addition, for wind farm A and over the one-year test period, the forecasts generated by the
MPCC led to the highest profits (around 4,100 higher) than did those forecasts obtained with
MPC.
For wind farm B, the first three training criteria also led to lower income levels. In this case,
MPC presents a higher profit potential than does MPCC (by around 7,500 ). The same
conclusions for wind farm B are also valid for wind farm C. The economic-oriented training
criteria led to higher levels of income, and once more the MPC realized a higher level of income
than did the MPCC.
From the results of the three wind farms, it becomes clear that using MPCC instead of any
statistical criterion enables increases to the income without significantly increasing the
deviations resulting from forecast errors.
System Operator Viewpoint
For wind farm A, the forecasts generated with MCC and MAE produce fewer total deviations
and deviations against the system. MCC and MAE underestimate the wind generation, but
despite this underestimation, a higher income was achieved with MCC when compared with
MSE. The MSE criterion places a heavy penalty on large errors, and therefore the CTE for MSE
presents the lowest value of all of the training criteria this result means that wind farm A is
usually perceived as a good property by the SOs. It is therefore satisfactory that the new
criterion, MPCC, could lead as well to a high level of income without significantly increasing the
total deviations and the deviations against the system. On the other hand, the MPC has a higher
bias on the side with lower balancing costs (e.g., a shortage of generation), which is in opposition
to the interests of the SO in terms of security of the system. The MPCC for wind farm A is near
the compromise viewpoint, as it allows some increase in the income without a major increase

20

in the total deviation. The comparison of MPC and MPCC in the quantile 95% and CTE values
also show that MPCC is better for the SO when compared with MPC.
For wind farm B, the first three training criteria led to lower total deviations and deviations
against the system. The tendency for this wind farm is to underestimate the wind generation. For
this wind farm, the training criterion with fewer total deviations was MCC, which is surprising
because MAE was expected to achieve fewer total deviations. Once more, MSE presents the
lowest value of CTE. MPC presents a higher realized income than does MPCC; however, MPCC
allowed (or enabled) increases in realized income while the total number of deviations and
deviations against the system are maintained at an acceptable level.
The same behavior is verified for wind farm C, where, while the MPCC realized a lower income
than did the MPC, the total deviation values and CTE are lower. In this case, MPC presented a
lower value for deviations against the system.
2.5 Conclusions
This chapter addresses the WPF problem by recognizing that the value and quality of forecasts
must be appreciated in the framework that decision making is carried out by multiple agents and
that agreement may not be possible among the agents (WGENCOs or SOs) regarding what
constitutes a good forecast. The issues discussed the relation between forecast error and
market profit also occur in situations involving more complex forecasting systems and
electricity market rules. The results in this chapter provide evidence that forecasts with higher
accuracy (in the forecasters paradigm sense) do not necessarily lead to higher incomes for a
WGENCO. As such, a WGENCO may be willing to reduce the forecasts accuracy in exchange
for an increased income.
The main contributions of the chapter are summarized in the following:
1. A challenge to the assumption of the mathematical neutrality of forecasting models.
2. A thorough discussion of conflicting interests associated with the goodness of
forecasts for two types of agents in an electricity market.
3. A comparison of the relative advantages and drawbacks of several criteria referred to
in the literature and used in training WPF models.
4. A new approach proposed to achieve acceptable compromise solutions to market
participants in the choice of a forecasting model by introducing the ITL concept of
parametric correntropy.
5. The verification of the theoretical concepts in a representative example, namely, the
Iberian market, MIBEL.
What is the significance of the results shown in this chapter for future forecasting practices and
market policies? First, these results reveal the importance of the choice of a training criterion for
a forecasting tool. In this sense, it is not necessarily true that it is impossible to achieve a high
remuneration in the electricity market and maintain the forecast error within an acceptable level.
Methods that produce acceptable compromises are available. Second, in systems with high wind
penetration levels, it is important to look at the impact of the wind generation market bids on the
balancing needs. In fact, the market rules and the way that wind power is remunerated in the
market should be revisited and possibly redesigned to better facilitate the decisions made by the

21

SO regarding the security of the system. This chapter shows that an asymmetry in balancing
prices makes possible the opportunity for profit by those forecasters paradigm WGENCOs that
make careful selection of the characteristics of the output of the forecasting systems. Whenever
there is an asymmetry in the price structure, whether in balancing prices or spot prices (which is
more likely to occur in cases of large penetration of wind power, with larger impacts on price
volatility), a neutral model (in the forecasters paradigm sense) may not be the model that leads
to maximizing the profits of WGENCOs that bid in the market.
This chapter also points toward areas for wider discussion, namely, toward an analysis on
possible reactions of other agents in the market to wind power bids, especially if indications
appear that bids do not correspond to a neutral forecast (in the forecaster paradigm sense),
which could lead to speculation against admittedly biased wind power forecasts. One possible
evolution that the results in this chapter suggest would be to decouple the concepts of forecasting
and bidding. In some markets these days, SOs demand that WGENCOs bid the forecast they
have produced and then trust it as a valid forecast. However, WGENCOs may wish to
develop a bidding strategy, and this impetus leads to the possibility of adopting models that,
while seeming to be mathematically neutral, will have characteristics that serve their particular
market strategy and will possibly be contrary to SOs interests. So, it follows naturally that
WGENCOs may be allowed to bid in an unrestricted manner on whatever power value they
prefer, but that SOs will also have their own independent forecasts and will act according to
those and not solely to the forecasts announced by WGENCOs in order to ensure reliability in
the power system.

22

WIND POWER FORECASTING IN U.S. ELECTRICITY MARKETS 2

WPF is becoming an important tool in electricity markets as the amount of wind power rapidly
increases. However, the use of WPF in market operations and among market participants is still
at an early stage. We discuss the current use of WPF in U.S. ISO/RTO markets, and give
recommendations for how to make efficient use of the information in state-of-the-art wind power
forecasts.
3.1 Introduction
Wind power generation has increased rapidly in the United States over the last few years, and at
the end of 2010, there was an installed capacity of more than 40,000 MW at a national level. 3
Wind power is already having a significant impact on the operation of electricity markets and
power systems in areas with the highest penetration of wind power, such as in the Electric
Reliability Council of Texas (ERCOT) and the Midwest ISO (MISO). A number of challenges
arise when integrating renewable generation into the power system, from transmission planning,
resource adequacy, and interconnection standards to dealing with the increased uncertainty and
variability in short-term operations. Many of these challenges are discussed in recent reports by
the North American Electric Reliability Council (NERC) [43] and the U.S. Department of
Energy [44].
It is generally agreed that forecasting will play an increasingly important role in the operation of
power systems as the level of wind power increases. WPF models have been continuously
improving over the last decade. A number of WPF providers have emerged that are competing to
provide the best forecasts to the electric power industry. There are several different users of the
wind power forecasts, including generation companies, wind power producers, utilities, market
analysts and traders, and power system and electricity market operators. The desired forecasting
capabilities typically vary among user groups. For instance, system operators are particularly
concerned with the ability to forecast rapid changes in wind power production, so-called ramping
events. Wind power producers, on the other hand, may be more concerned with minimizing their
forecasting error over a longer time period to better schedule their resources in the marketplace.
Some companies already have considerable experience in the use of WPF, as documented in
[7][45][46].
In this chapter, we first give a brief introduction to WPF models. We then present an overview of
the current use of WPF in U.S. electricity markets, with focus on the ISO/RTO markets that
currently have the highest penetration of wind power. Finally, we discuss the need to revise
market design to make better use of the information provided by advanced WPF systems in
market operations.
3.2 Wind Power Forecasting Models
A WPF system, as illustrated in Fig. 1-1, uses input data from different sources, including results
from numerical weather prediction (NWP) models, local meteorological measurements, SCADA
2

This chapter is reproduced from the Electricity Journal, Vol. 23, No. 3, A. Botterud, J. Wang, V. Miranda, and R.J.
Bessa, Wind Power Forecasting in U.S. Electricity Markets, pp. 7182, 2010, with permission from Elsevier.
3
According to the American Wind Energy Association (AWEA), http://www.awea.org.

23

data describing the real-time (RT) state of the wind power plants, and additional information
about the characteristics of the wind power plants and the nearby terrain and topography.
Modern WPF models are usually based on a combination of physical and statistical models. The
physical approach describes the physical relationship between atmospheric conditions, local
topography, wind speed, and the output from the wind power plant. In contrast, the statistical
approach estimates a statistical relationship between all of the relevant input data and the
resulting wind power generation without considering the physics of the system. The performance
of wind power forecasts and the forecast accuracy depends on the availability of good NWP
forecasts, the complexity of the terrain, and the availability of RT weather and wind power plant
data. Hence, there can be large differences in forecasting errors between wind power plants at
different locations. WPF systems typically produce forecasts for a time horizon up to 23 days
ahead in time. In general, the forecasting error increases with the forecast horizon.
WPF models are continuously being improved with the goal of increasing the forecast accuracy.
Ongoing research also focuses on producing probabilistic forecasts, because information about
the uncertainty in the wind generation can be of great value for system operators and market
participants. As an example, Fig. 3-1 shows two different types of uncertainty representation in
WPF. Ramp forecasting is another important area of research. For a detailed review of the
current state of the art in WPF, we refer to our state-of-the-art report [1].
1

0.8

90% int.
80% int.
70% int.
60% int.
50% int.
40% int.
30% int.
20% int.
10% int.
Forecast
Measured

0.6

0.4

0.2

Power [p.u.]

Power [p.u.]

0.8

0.6
Scenarios
Forecast
Measured
0.4

0.2

0
00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00

0
00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00

Time Horizon [hr]

Time Horizon [hr]

Fig. 3-1 Uncertainty representation in wind power forecasts: (a) probabilistically by a set of
intervals or quantiles (left), and (b) scenarios of wind power generation representing the temporal
dependency of forecast errors (right).

3.3 Use of Forecasting in Market Operations

In this section, we discuss the potential use of WPF in the short-term operation of ISO/RTO
markets in the United States. The markets in different regions of the country have seen a
considerable degree of convergence in market design in recent years. ISO/RTO markets already
have several features that are advantageous for the integration of wind power into the electric
power system, such as the following:
1. Large Balancing Authority Areas give the system operator the control of generation and
demand resources over a larger area, increasing the overall flexibility of the system to
handle variations in wind power generation. At the same time, the variation in total wind

24

power generation, and also the WPF error, generally decreases as a function of
geographical size. Each ISO/RTO is operated as one balancing authority area.
2. Centralized unit commitment and scheduling of energy and operating reserves gives the
system operator control of a large pool of generation and load resources, which can be
used to effectively accommodate fluctuations in wind power output. Combined operation
of energy and operating reserves markets also ensures cost-efficient and market-based
procurement of energy and operating reserves.
3. Frequent dispatch of resources in real time. Most ISO/RTO markets re-dispatch the
system resources every five minutes, upon which new set-points are sent to generators
and eligible demand resources. The frequent system dispatch reduces the need to
maintain expensive regulation reserves to respond to short-term fluctuations in load and
non-dispatchable generation.
4. Locational marginal prices (LMPs) give locational market-based incentives for shortterm operation and long-term expansion. LMPs can also enable price-based and efficient
curtailment of wind generation during periods with surplus wind power generation, which
are typically caused by a combination of transmission bottlenecks and minimum
generation limits for thermal power plants. LMPs also provide efficient incentives for
location of new wind power generation, taking network congestion into account.
The experience with WPF, however, is relatively limited so far. A brief summary of the current
state of WPF for MISO, New York ISO (NYISO), PJM, ERCOT, and California ISO (CAISO) is
provided in Table 3-1. A typical timeline for the operation of the electricity market is shown in
Fig. 3-2. The procedures and timeline in the figure are based on the current rules in the MISO
market. However, other ISO/RTO markets are operated in a similar way. The main steps in the
market operations, including determination of reserve requirements, DA operations, reliability
assessment commitment, and RT operations, are discussed below. As indicated in Fig. 3-2, WPF
can play an important role in all of these steps as wind power penetration increases.
3.3.1

Operating Reserve Requirements

The requirements for operating reserves in U.S. power systems are based on standards
determined by the NERC [47]. Operating reserves are categorized into several types depending
on how quickly they can respond to changes in the system (Fig. 3-3). The regulating reserve
responds immediately to balancing needs in the system, and is usually provided by generating
units with automatic generation control (AGC) responding to frequency deviations in the
network.
Contingency reserves are used to respond to contingencies that may occur, such as forced
outages of generators or transmission lines. The contingency reserve can be split into spinning
and non-spinning reserves, where the spinning component needs to be able to respond within
10 minutes. It is common that both generation and demand resources can provide contingency
reserves. In the event that operating reserves need to be used due to a contingency in the system,
the required level of reserve must be restored as soon as possible and at least within 60 minutes
[47].

25

The ISO/RTOs are required to maintain sufficient regulating reserve to meet NERCs criterion
for area control error. 4 The requirement for contingency reserves is based on the N-1 rule, that is,
that sufficient contingency reserve must be held to cover the loss of generating capacity due to
forced outages of generation or transmission equipment that would result from the most severe
single contingency. At least half of the contingency reserve must be spinning. These are the
minimum requirements set by NERC; however, regional variations exist and some ISO/RTOs
use more stringent requirements for their operating reserves. In systems with large and congested
networks, it is common to specify regional reserve requirements in addition to the system-wide
criteria. It is also interesting to note that some markets have introduced a demand curve for
different types of operating reserves rather than the traditional fixed requirements. Operating
reserve requirements may be updated to accommodate changes in the system conditions.
Although the update frequency varies between different markets (seasonal, monthly, and daily
updating), the requirements for the next operating day must be posted at least before operation of
the DA market begins.
Day ahead:
Clear DA market
using SCUC/SCED

Post operating
reserve requirements
Prepare and
submit DA bids

1100

Wind power
forecasting

Rebidding
for
RAC

1600

Post-DA RAC
using SCUC

1700

DA day ahead
RT real time
SCUC security constr.
unit commitment
SCED security constr.
economic dispatch
RAC reliability ass.
commitment

Post results
(DA energy
and reserves)

Operating day:
Clear RT market using
SCED (every 5 min)

Intraday RAC
using SCUC
Prepare and
submit RT bids

Operating hour

-30min

Post results
(RT energy
and reserves)

Fig. 3-2 Market operations timeline for MISO, indicating where WPF could play an important role.

The Area Control Error is a measure for the deviation between scheduled and realized exchange from a balancing
authority area and is defined by NERC as the instantaneous difference between net actual and scheduled
interchange, taking into account the effects of Frequency Bias including correction for meter error.

26

Table 3-1 Overview of WPF and market operation in five U.S. electricity markets. Based on
information from the Utility Wind Integration Group (UWIG) and ISO/RTOs.

Peak load

MISO
116,030 MW
(7/31/2006)
Ca. 138,000 MW

Total
installed
capacity
Wind
Ca. 7,600 MW
capacity
(end of 2009)
Wind power In operation since
forecasting
2008:
DA and intraday RAC
Transmission
security and
outage
coordination
Transmission
security and
peak load
analysis
Indication of
ramps

NYISO
33,939 MW
(8/2/2006)
Ca. 38,000 MW

PJM
144,644 MW
(8/2/2006)
Ca. 165,000
MW

ERCOT
62,339 MW
(8/17/2006)
Ca. 80,000 MW

CAISO
50,270 MW
(7/24/2006)
Ca. 59,000 MW
(incl. imports)

Ca. 1,300 MW

Ca. 2,500 MW

Ca. 9,000 MW

Ca. 3,000 MW

In operation
In operation
since 2004:
since 2008:
80%
Used to
exceedance
calculate
forecast used
energy
for DA
schedule in RT
planning
market
To be fully
Advisory role
integrated in
in DA market
new nodal
design, to be
introduced
end of 2010
Developing
ramp forecast
Market
DA bids due:
DA bids due:
DA bids due:
DA bids due
DA bids due:
timeline
11 a.m.
5 a.m.
noon
(reserves only): 10 a.m.
DA results: 4 p.m. DA results:
DA results:
1 p.m./4 p.m.
DA results:
Re-bidding due:
11 a.m.
4 p.m.
DA results:
1 p.m.
5 p.m.
RT bids due: OH RT bids due:
1:30 p.m./6 p.m. RT bids: OH RT bids due:
-75 min
6 p.m. (DA)
RT bids due:
75 min
OH -30 min
OH -60 min
Wind power If wind is a
Wind required
If wind is a
Bilateral
Wind program
bidding,
capacity
to bid in RT
capacity
market
participants
Imbalances
dispatch,
resource, it must
market. DA
resource, it
RT bid based
imbalance
bid in DA
bidding
must bid in DA settled at RT
on ISO
zonal energy
settlements,
market and RAC
optional
market
forecast.
No deviation
Dispatch
Deviation
price
deviation
Deviations
penalties
signals
charges apply Penalty
penalties
netted over
No wind
exemption for
provided from Wind dispatch
month at
dispatch, but this
signals
+/- 50% of
SCED
average price
Penalty for
10-min
is being
provided in
scheduled
over-generation
imbalance
considered
constrained
generation
Ramping
in constrained
charges for
situations
limits
situations
nonparticipants
No penalties
Wind is not
for underdispatched
generation
Key: DA = day-ahead, OH = operating hour, RAC = reliability assessment commitment, RT = real-time, SCED =
security-constrained economic dispatch
In operation since In operation
2008:
since 2009:
Reliability
DA
assessment
transmission
commitment at
security and
DA stage
reserve
RT
adequacy
commitment
assessments
Developing
and dispatch
Ramping alert
automated
system under
procedures
Specific ramp
consideration
forecast

27

Operating
Reserve
Regulating
Reserve

Contingency
Reserve
Spinning
Reserve

Renewables
Reserve
NonSpinning
Reserve

Fig. 3-3 Traditional operating reserve categories as defined by NERC and a proposed new
category to address increased uncertainty and variability from renewable generation.

The additional uncertainty and variability caused by an increasing penetration of wind power
generation raises the question of whether current requirements for operating reserves are
adequate. In general, WPF can guide the procurement and use of the operating reserves by
increasing the system operators knowledge of projected wind power generation and therefore
the overall balance between supply and demand in the system. Several improvements to existing
requirements and procedures are relevant, including:
1. New definitions of operating reserve requirements that better reflect the needs in power
systems with a large share of renewable energy. For instance, changes in wind power
output unfold much more slowly than the traditional contingencies, which are caused by
sudden equipment failures. Hence, the system operator has more time to respond to these
changes, and a slower type of reserve than the current contingency reserves (at least
slower than the current spinning reserves) may be sufficient. At the same time, it is less
critical to replace these reserves immediately, as a second down-ramp event is not likely,
and oftentimes not even possible, if there has already been one major reduction in wind
power generation. An additional type of renewables reserve is therefore proposed in
Fig. 3-3. Such a reserve category would have slower response and restoration times and
so should be less expensive than the traditional contingency reserves. There is also likely
to be a need for additional regulation reserves. However, the short dispatch intervals
reduce the need for regulation to handle short-term variability in wind generation. Better
short-term wind power forecasts would also limit the amount of regulation reserves
needed.
2. Improved determination of reserve requirements to better accommodate the
characteristics of wind power and other renewable generation. Traditional deterministic
criteria (e.g., loss of the largest unit in the system) should be replaced with probabilistic
approaches that directly reflect the uncertain nature of supply and demand. Ideally,
probabilistic wind power forecasts should be used in the calculation of improved reserve
requirements that account for the risk of losing load due to wind generation forecast
errors in addition to forced outages and unexpected changes in load. The historical
forecasting errors should also be taken into account. Ongoing research looks at
probabilistic determination of optimal spinning reserve criteria [30][48] in systems with
high wind power capacity, as well as demand curves for operating reserves [49].

28

3. Dynamic and frequent updating of operating reserve requirements based on the most
updated system information, including the most recent wind power forecast. This will
result in better-informed decisions, and thus in more efficient operations. More frequent
updating of reserve requirements would enable operating reserves to be adjusted
according to how much wind generation is forecasted for the next hours and days, as well
as the projected uncertainty in the forecast.
Several ISO/RTOs are currently considering the need to change their procedures and
requirements for operating reserves. ERCOT, which is the ISO/RTO with the highest level of
wind power capacity in the United States (Table 3-1), is already considering wind power
penetration and forecasting uncertainty in their determination of requirements for regulation and
non-spinning reserves [50]. In a recent study of the ancillary services requirements in ERCOT
[51], GE recommends that ERCOT make adjustments to its procedures as the level of wind
power generation increases, pointing out the importance of making better use of forecasting in
the procurement of different types of operating reserves, and that the level of reserves should be
adapted to actual conditions.
3.3.2

Day-Ahead Operations

The deadline for market participants to bid into the DA market varies among the different
ISO/RTOs, as shown in Table 3-1. The bids of the market participants must reflect how much
energy and operating reserves they can provide. Information on unit commitment (UC)
constraints (e.g., ramping rates, startup costs/times, minimum down-times, etc., for generating
units), self-scheduling, and bilateral schedules is also provided to the ISO/RTO.
The clearing of the DA market is a two-stage procedure in most markets. First, a security
constrained unit commitment (SCUC) is run to commit resources. Second, a security-constrained
economic dispatch (SCED) algorithm is run based on the commitment schedule from the SCUC.
Transmission constraints may be simplified or omitted from the SCUC formulation to solve the
problem in a reasonable time. In contrast, the transmission constraints are always included in the
SCED formulation, although usually with a simplified linear representation (DC-OPF). In most
U.S. markets, the SCUC/SCED procedure co-optimizes energy and operating reserves. The
output from the DA market clearing therefore includes schedules for energy and operating
reserves. In addition, LMPs are derived for each transmission node, and market-clearing prices
are also calculated for each category of operating reserves.
In the United States, about 70% of the installed wind power through 2008 was sold through
Power Purchasing Agreements (PPAs), mainly to utilities [52]. Most of the wind power
generation is therefore not being bid directly into the DA market by the wind power producers,
but rather is integrated into the utilities scheduling plans, along with loads and other generating
resources. It is therefore hard to estimate to what extent wind power forecasts are taken into
account in the bids and schedules submitted into the DA market. In many cases, wind power
generation is first being dispatched in the RT market. This may be one of the reasons why RT
prices currently tend to be lower than DA prices in areas with large wind penetration.5 Wind
5

As an example, within MISOs footprint in 2009, the average RT prices in areas with a substantial amount of wind
energy (like North Dakota, Iowa, and Illinois) were as much as 5% lower than average DA prices. The difference
between DA and RT prices was only 12% in the eastern part of the MISO footprint, which has less wind.

29

power producers may therefore benefit from using wind power forecasts to bid into the DA
market and thereby receive a higher and less volatile price. 6
It is important to integrate the information in wind power forecasts into the DA market clearing.
Wind power will have an increasing impact on the marginal cost of electricity, and this should be
properly reflected in the DA market clearing, where most of the energy is settled. For the DA
market to reflect the expected RT conditions as closely as possible, it is important that the market
participants have the right incentives to provide the best possible wind power forecast
information through their bids. Since the DA operation should be market-driven and based on the
market participants schedules and bids, the system operator should not use its own forecast to
directly represent wind power generation in the SCUC/SCED procedure in the DA market. The
system operators forecast information should still be reflected in the operating reserve
requirements, as discussed above, and in this way would also influence the market results.
One way to encourage participation of wind power generation in the DA market would be to
reduce the time period between DA bidding and RT operations. The due times for DA bids vary
among markets, but are at least 12 hours before the start of the operating day (Table 3-1). By
shortening the time period and possibly introducing additional markets between DA and RT, the
market participants, and particularly the wind power producers, could clearly make better use of
forecast information, as the forecast error decreases with a shorter time horizon. It is also
important to avoid situations in which market participants are incentivized to submit biased DA
bids. As an example, the ERCOT market used to have protocols that unintentionally gave wind
power producers in some areas incentives to over-schedule, due to allowances for uninstructed
deviations combined with rules for congestion management [53]. In the Iberian electricity market
(Spain and Portugal), asymmetric deviation penalties make it more attractive for wind power
producers to bid above their expected generation [42].
3.3.3

Reliability Assessment Commitment

After the clearing of the DA market and before the start of the operating day, the ISO/RTO
usually performs a revised commitment procedure, with more focus on reliability. The so-called
reliability assessment commitment (RAC) is also performed using SCUC. However, at this stage
the demand bids that were used to clear the DA market are replaced with the forecasted load for
the next day. The ISO/RTO may therefore decide to change the commitment schedule from the
DA market clearing based on the results from the RAC. The RAC procedure may take place
according to a fixed time schedule or whenever the system operator finds it necessary to consider
system re-commitment, due to unexpected operating conditions (forced outages, deviations from
forecasted loads, etc.). Rules are typically in place to make sure that generating resources
committed in the RAC recover all of their operating costs. This may sometimes require makewhole side payments, in addition to the regular payments based on the market clearing prices, in
order to recover all costs (including no-load costs, start-up costs, etc.).
ISO/RTOs are currently focusing on how to integrate the information in wind power forecasts
into the RAC. In contrast to the DA market clearing, the system operator can now use its own
6

By using wind power forecasting for scheduling and bidding, wind power producers may also reduce their
exposure to negative prices and lower their imbalance penalties (if such penalties exist). This is discussed in more
detail in Chapter 7. In addition, WPF can play an important role in maintenance planning.

30

forecast information directly as input to the SCUC. Hence, both demand and wind power
bids/schedules are replaced with the system operators own forecast information in order to
ensure reliable operation. An important question is how to consider the additional uncertainty
from wind power at this stage in the market operation. One can address the uncertainty and
variability from wind power by adjusting and increasing the operating reserve requirements, as
discussed above. However, the treatment of wind resources in the RAC will also influence how
much other generation capacity is committed and available to accommodate the predicted and
unpredicted system variability.
An interesting approach is taken by ERCOT, which is currently using an 80% exceedance
forecast 7 for wind power generation as input to its DA resource planning procedures [54].
ERCOT is considering whether to use the same type of forecast in the RAC as the nodal market
system is introduced. In reality, using a conservative projection of wind power means that more
capacity is committed than what would be the case if the mean or median forecast were used.
Hence, while the resulting security margin may increase the reliability of the system, repeated
over-commitment would result in inefficient operations.
An alternative way to address the uncertainty from wind power generation is to use a stochastic
formulation of the UC problem. Several different approaches have been proposed in the literature
recently to address uncertainty from wind power generation in the UC problem
[55][56][57][58][59][60]. Preliminary results indicate that stochastic models can contribute to
reducing costs while maintaining system security under increased uncertainty and variability.
However, more research is needed into developing and testing stochastic models for UC and into
evaluating their real-world applicability for system and market operations. In particular, it is
important to generate consistent and representative information about the forecast uncertainty for
use in such stochastic models. It is also important to consider the close interaction between the
operating reserve requirements and the UC policy because they both influence the level of
committed and available capacity. An important consideration is that while capacity committed
as operating reserve receives reserve payments, additional capacity committed in the RAC is
likely to receive make-whole side payments to recover costs, as pointed out above. The required
level of capacity should be achieved in the most cost-efficient manner. Some of these issues are
further explored in Chapters 4, 5, and 6.
Another potential application of a wind power uncertainty forecast at this stage in the market
operation is in congestion management analysis to assess the risk of congestion in some network
branches and perform corrective measures to hedge against a higher risk of congestion [61].
3.3.4

Real-time Operations

Market participants can bid their remaining resources into the RT market. The deadline for
submitting bids to the RT market varies quite widely between different markets (Table 3-1). 8
During the operating hour, the ISO/RTO uses SCED to dispatch the system on the basis of the
committed capacity in the system after the DA and RAC operations have been completed. At the
same time, RT prices for energy (LMPs) and operating reserves are calculated. Conventional
7

This means that there is an 80% chance that the actual wind power generation will exceed the forecast. Hence, it is
equivalent to a 20% forecast quantile.
8
Some ISO/RTOs, like NYISO and CAISO, also have a formal hour-ahead market clearing procedure.

31

power generation sources are penalized if they deviate from their RT dispatch signals. Until
recently, however, renewable generation such as wind power has not been given dispatch signals
from the ISO/RTO and has typically been dispatched as a price taker in the RT market. Wind
power generation also has exemptions from RT deviation penalties in most markets (Table 3-1).
ISO/RTOs are currently working on improving the integration of wind power into their RT
dispatch procedures. Short-term wind power forecasts, which have relatively low uncertainty,
should be taken into account in the ISO/RTOs RT SCED. At the same time, it is important that
the ISO/RTO is able to control the generation from wind power plants and enforce curtailment of
wind power generation in situations where this is needed, either from an economic or reliability
perspective. 9 CAISO is requiring wind power plants in the Participating Intermittent Resource
Program (PIRP) to bid into their hour-ahead market according to CAISOs short-term wind
power forecast. Rules are also in place to limit the deviation penalties for wind power (Table
3-1).
NYISO recently introduced new rules to incorporate wind power in its SCED [62]. With the new
rules, wind power plants will be required to bid into the RT market. A price-bid is offered into
the market based on the wind power producers preference, whereas the bid quantity is
determined by the system operators short-term wind power forecast. During unconstrained
hours, wind power plants can operate freely. However, in constrained situations, wind power
plants are directed to curtail their output when the clearing price at their location falls below their
economic bid. Penalties are in place for exceeding the dispatch instructions. The new procedure
makes certain that the economic preferences of the wind power producers are reflected in the
SCED. It also contributes to a more efficient dispatch overall and reduces the need for using outof-market actions to maintain system reliability. PJM has also introduced a scheme for RT
dispatch of wind power plants based on their bids. Other ISO/RTOs should consider similar
measures to improve the handling of wind power resources in the RT dispatch.
3.4 Summary and Conclusions
WPF is an important tool to improve the efficiency and reliability of power systems with a large
share of wind power. ISO/RTOs in the United States are currently working on integrating wind
power forecasts into their operating procedures, and WPF is already used for a number of
important applications. 10 However, as the amount of wind power capacity is rapidly increasing,
there is a need to better integrate WPF into different parts of electricity market operations, from
DA scheduling to RT operations. It is also important to design electricity markets that give
market participants the opportunity and right incentives to provide their unbiased forecast
information through their scheduling and bidding.
Fig. 3-4 summarizes how we foresee that WPF will play important roles in the main steps in
electricity market operations. ISO/RTOs and market participants will need to develop new tools
and procedures to make efficient use of the forecast information, with the overall goal of making
9

Modern wind power plants include a number of features that make them more similar to conventional power
plants, including reactive power contribution, voltage regulation, disturbance ride-through, grid frequency response,
smoothing wind ramps, and controlled startup/shutdown.
10
More information on the status of the integration of wind power and forecasting in U.S. ISO/RTOs is provided by
UWIG in [63][64].

32

better operational decisions under the increased uncertainty and variability from wind power and
other sources of renewable generation. A key challenge, therefore, is to improve decision making
under uncertainty and the understanding of the impact of uncertainty on operational decisions
(such as UC). At the same time, it is important to continue the improvements of WPF models,
and to better tune them to the specific needs of the forecast users. For electricity market
operators, it is particularly important to improve predictions of forecast uncertainty and ramping
events.
Producer Forecast
Supply Bids/Schedules
ISO Forecast
Reserve Requirements

Producer Forecast
Re-bidding/Scheduling
ISO Forecast
Wind Power Projection

ISO Forecast
Wind Power Projection
Real-Time Schedules

Day-Ahead
Market
Clearing

Reliability
Assessment
Commitment

Real-Time
Market
Clearing

Day-Ahead Schedules
Day-Ahead Prices

Unit
Re-Commitments

Dispatch
Real-Time Prices

Fig. 3-4 Role of WPF in electricity market operations.

33

Financial
settlement

This page intentionally blank.

34

WIND POWER FORECASTING UNCERTAINTY AND UNIT COMMITMENT 11

In this chapter, we investigate the representation of WPF uncertainty in the unit UC problem.
While deterministic approaches use a point forecast of wind power output, WPF uncertainty in
the stochastic UC alternative is captured by a number of scenarios that include cross-temporal
dependency. A comparison among a diversity of UC strategies (based on a set of realistic
experiments) is presented. The results indicate that representing WPF uncertainty with wind
power scenarios that rely on stochastic UC has advantages over deterministic approaches that
mimic the classical models. Moreover, the stochastic model provides a rational and adaptive way
to provide adequate spinning reserves at every hour, as opposed to increasing reserves to
predefined, fixed margins that cannot account either for the systems costs or its assumed risks.
The following nomenclature is used in this chapter:
Indices
i
j
k
l
m
s
Constants
,
(s)
a,b,c
Aj
Cens
CCj
CRrns,m
D(k)
DCj
HCj
MCl,j
probs

RLj
SUj
SDj

Index for wind unit, i = 1I

Index for thermal unit, j = 1J
Index for time period, k = 124
Index for generation block, thermal units, l = 1L
Index for reserve demand block, m=1M
Index for scenario, s = 1S
Capacity, block l, thermal unit j
Operating reserve percentage, scenario s
Unit production cost function coefficients
Operating cost at minimum load, thermal unit j
Cost of energy not served
Cold start cost, thermal unit j
Cost of reserve not served, block m
Load or demand, period k
Shut-down cost, thermal unit j
Hot start cost, thermal unit j
Marginal cost (or bid), block l, thermal unit j
Probability of occurrence, wind scenario s
Capacity, thermal unit j
Minimum output, thermal unit j
Forecasted maximum generation, wind unit i, period k, scenario s
Ramping limit (up/down), thermal unit j
Start-up ramp limit, thermal unit j
Shut-down ramp limit, thermal unit j
Time for cold start cost (in addition to minimum downtime), thermal unit j

11

This chapter is reproduced from Applied Energy, Vol. 88, No. 11, J. Wang, A. Botterud, R. Bessa, H. Keko, L.
Carvalho, D. Issicaba, J. Sumaili, and V. Miranda, Wind Power Forecasting Uncertainty and Unit Commitment,,
pp. 40144023, 2011, with permission from Elsevier.

35

Wi(k)
WR(k)
Variables
, (k)

ptj(k)

vj(k)

Minimum up-time, thermal unit j

Minimum up-time, initial time step, thermal unit j
Minimum down-time, thermal unit j
Minimum down-time, initial time step, thermal unit j
Actual maximum wind generation, wind unit i, period k
Additional wind reserve, period k
Generation, block l, thermal unit j, period k
Production cost, thermal unit j, period k
Start-up cost, thermal unit j, period k
Curtailed wind generation, wind unit i, period k, scenario s
Energy not served, period k, scenario s
Generation, thermal unit j, period k
Maximum feasible generation, thermal unit j, period k
Generation, wind unit i, period k, scenario s
Reserve requirement (spinning), scenario s, period k
Reserve curtailed, period k, scenario s
Binary on/off variable, thermal unit j, period k

4.1 Introduction
UC and economic dispatch (ED) operate in tandem in the power generation industry to enable
generation scheduling and grid management, thereby contributing to a systems overall
reliability. However, the increasing use of renewable generation sources (such as wind power) is
adding unprecedented amounts of uncertainty to a system operators grid management and
generation scheduling. Two key elements of this uncertainty the inherent intermittency and
the variability of renewable resources are thus driving system operators to alter current
industry approaches to UC and ED. Whereas considerable research effort has been directed to
formulating and improving UC algorithms [65], efforts to consider wind power uncertainty have
been more limited to date.
This chapter investigates the use of multiple wind power forecasts in UC and ED and analyzes
the impact of different reserve requirements and UC policies on system operation. Our aim is to
demonstrate that, despite the inherent uncertainties and risks when relying on wind power, it is
possible for operators to adopt a quantified method to deal accurately with this renewable source.
There are several reasons why UC and ED are greatly impacted when large amounts of wind
energy are part of the mix of generation sources. First of all, WPF errors bring greater levels of
uncertainty to system operation, because the RT amount of injected wind power may be very
different from what is forecasted. Forecasting errors, which are mismatches between the value(s)
plugged in during the UC stage versus the amounts used in the RT dispatch, can cause great
difficulty for system operators, who must balance the unexpected surpluses or deficits of wind
power.

36

This forecasting uncertainty can be manifested in several ways. For example, an unforeseen
reduction in wind power can hinder system reliability if the available ramping-up capabilities of
the systems other on-line units are not sufficient to accommodate the current load. A large
upward ramp in the supply of wind power may be unfavorable, however, in a system in which
other units lack sufficient downward reserves. Wind power is sometimes curtailed to mitigate
this situation, which nevertheless leads to an unwanted waste of available resources. In fact, a
surplus in wind power at night may often have to be curtailed: while the wind may be strong,
system load is usually low. In this case, wind generation may have to be spilled to maintain
stable operation of thermal units (e.g., coal-fired, nuclear) because of their physical and/or
economic constraints.
Variability is also a challenge for generation scheduling: because wind power is normally
assumed to have an operating cost of zero, system operators try to make use of wind power as
often as possible. To meet this objective, however, the output of other generation sources must
be adjusted to accommodate wind power variability, which complicates scheduling the
generation of non-wind resources. Therefore, even if wind power could be forecasted perfectly,
variability by itself (i.e., considered in isolation) would still be important, because the system
must still operate within the physical constraints of other non-wind units, such as their ramping
up/-down constraints and minimum-on/minimum-off time constraints, etc.
These uncertainty factors complicate the challenges that operators and researchers face in
revising and adjusting UC and ED algorithms to accommodate wind power, resulting in a variety
of approaches. Some researchers focus on revising the current security-constrained unit
commitment (SCUC) formulation. Others aim at novel UC methods. Barth et al. [55] presented
the early stage of the Wind Power Integration in the Liberalised Electricity Markets (WILMAR)
model in [66]. Tuohy et al. [59] extended their previous studies in [67] and [68] to examine the
effects of stochastic wind and load on the UC and ED of power systems with high levels of wind
power by using the WILMAR model. The WILMAR model calculates both UC decisions and
the associated RT dispatch decisions using a rolling planning approach that uses realized wind
power production and load in the first stage of the scenario trees. Ummels et al. [69] analyzed the
impacts of wind power on thermal generation UC and ED in the Dutch system, which has a
significant share of combined heat and power (CHP) units. Bouffard and Galiana [56] proposed
a stochastic UC model to integrate significant wind power generation while maintaining the
security of the system. Ruiz et al. [70] proposed a stochastic formulation to manage uncertainty
in the UC problem. In a related paper [58], the authors consider uncertainty and variability in
wind power in the UC problem by using the same stochastic framework. Wang et al. [57]
presented an SCUC algorithm that takes the intermittency and variability of wind power
generation into account. Delarue and Dhaeseleer [71] develop a mixed-integer programming
model to analyze the value of forecasting in unit commitment. Most of the models presented in
the literature so far, however, are used for planning (i.e., UC) purposes (e.g., [55][57][58][66]
[70]), while there is only very limited research that explores the use of different WPF methods in
actual market operation: in other words, the linkage between DA UC and RT ED is largely
missing.
To deal with these issues in this chapter, we use (1) point and probabilistic forecasts of wind
power that are generated by state-of-the-art techniques, and (2) two different methods: stochastic

37

and deterministic. The consequences of a UC schedule to system costs are then assessed by
running a dispatch simulation that takes into account the actual value of wind power instead of
forecasted values. The relationship between UC policies and reserve requirements is discussed in
the context of the large uncertainties arising from reliance on wind power. With added
uncertainty, the focus shifts to considering some amount of extra reserve margin (e.g., see [72]
and [73]). A detailed probabilistic analysis of the impact of renewable sources in the power
system of the Iberian Peninsula is given in [74]. A methodology to define operating reserve
considering conventional generation outages, load forecast uncertainty, and wind power forecast
uncertainty is described in [75]. Probabilistic forecasts are used for wind power. In [76], a survey
of the methods of determining the optimal amount of operating reserves for systems with high
wind penetrations is presented, showing that various methods may have significantly different
results. The challenge, therefore, is to define the amount of reserve needed for each hour
adaptively and adequately instead of aiming at an arbitrarily fixed value for the required reserve.
To contribute to this discussion, a number of cases are simulated in order to analyze and compare
the results in detail.
In this study, a 24-hour optimization procedure is run at the beginning of the RT operation to
represent the RT dispatch (assuming perfect knowledge of wind power in the following hours of
the day). This way, the real cost of operation may be assessed fairly for comparison. Therefore,
this constitutes a lower bound for the real cost of operation for a given commitment. In reality,
there is still uncertainty in wind power for the subsequent time steps in relation to the current
hour. This approach has the merit of taking into account the ramping constraints for the thermal
units across the 24-hour operation horizon. In addition, because the objective is to compare UC
models, we wanted to avoid blurring the analysis by simultaneously introducing a model for the
ED that also included the uncertainty component. For similar reasons, we do not consider
commitment decisions between DA scheduling and RT dispatch. In reality, the SO usually has
some flexibility to commit additional fast-starting resources after the DA UC to account for any
unexpected events, such as WPF deviations. This flexibility will contribute to improve the RT
operation regardless of the DA UC strategy. However, in order to isolate the effect of the DA
UC, we do not consider the impact of intra-day commitments in this analysis.
The rest of the chapter is organized as follows: Section 4.2 briefly describes WPF and the
scenario generation method we used. Section 4.3 presents the problem formulation. Numerical
examples are provided in Section 4.4 Section 4.5 presents conclusions based on the discussion.
4.2 Forecasting and Scenario Generation Approaches
An advanced WPF system uses input data from different sources, and the forecast accuracy
depends on a number of factors, including numerical weather prediction (NWP) models and the
quality of wind predictions, local meteorological measurements, SCADA data (e.g., active power
generated), and additional information (e.g., on the characteristics of the wind power plants, the
complexity of the terrain, the availability of RT weather and power plant data). Large differences
in forecasting errors can occur among wind power plants at different locations. WPF systems
typically produce forecasts for a time horizon of up to 23 days ahead of the present time. In
general, the forecasting error increases with the forecast horizon. We provide a detailed
description of the state-of-the-art in WPF in [1].

38

There are several different approaches to model uncertainty in wind power predictions [1]. In
this chapter, we use the approach from [77] to produce probabilistic forecasts based on point
forecast errors. The model employs a linear quantile regression, where the base functions are
formulated as cubic B-splines, in order to obtain the quantile with a proportion of the forecast
errors. The probabilistic forecast is represented through a set of quantiles ranging from 5% to
95% (in 5% increments), and the tails are modeled by exponential interpolation reflecting
improbable and extreme events [77].
Since the probabilistic forecasts (i.e., set of quantiles) produced by the quantile regression
method do not capture the temporal correlation between forecast errors (i.e., uncertainty) of
different hours, a scenario generation method described in [78] is used to generate a number of
wind power scenarios that provide information on the development of the prediction errors
through the set of look-ahead times. The method takes as inputs the forecasted quantiles for each
look-ahead time-step and also the observed wind power generation. Assuming that the
probabilistic forecasts are reliable, the forecast errors are made Gaussian by applying a
transformation with the inverse of the Gaussian cumulative distribution function. This results in a
Gaussian random variable with zero mean and unit standard deviation. Considering the vector
with all the forecasts for each look-ahead time-step, it is assumed that the random vector follows
a multivariate Gaussian distribution, with mean values being a vector of zeros and a covariance
matrix. The temporal interdependence structure is represented by the covariance matrix, which is
recursively estimated because of the nonstationary characteristics. A specified number of
scenarios are obtained through sampling from an inverse cumulative distribution function. More
details about the method can be found in [78]. The scenarios produced by the method have the
following characteristics: (i) they respect the marginal wind power forecasted distribution for the
coming horizon (i.e., probabilistic forecasts); and (ii) the temporal correlation of forecast errors
are embedded in the scenarios values.
4.3 Unit Commitment and Dispatch Formulations
The UC and dispatch formulations used in this chapter expand on our initial version of the UC
problem introduced in [60]. The general UC constraints follow the deterministic model in [79].
However, we make adjustments in the stochastic version on the basis of introducing WPF
uncertainty, which is represented in the form of scenarios.
4.3.1

Objective Function

The objective is to minimize the sum of three kinds of costs: expected production costs, the
expected cost of unserved energy and reserve curtailment, and start-up costs, as shown in (4-1).
Constraints on load and operating reserves are represented in (4-2)(4-3). We use a step-wise
reserve demand curve to mimic the reserve requirement practiced by some system operators,
such as MISO [80], as shown in (4-4). This formulation allows the reserve requirement, which is
represented by a certain percentage of load, to be reduced in some cases at the reserve
curtailment cost captured in (4-1) to avoid load curtailment. We also introduce the possibility of
dedicating an additional operating reserve at the DA UC stage specifically for use with wind
power generation. The idea is that this wind power reserve helps accommodate the uncertainty
and variability that arise from reliance on wind units. Wind units may also be curtailed, if
necessary, as shown in (4-5). Note that the dispatch of thermal units and therefore the costs
associated with production, curtailed energy, and reserve all vary by wind scenario.

39

Furthermore, the constraints for load, operating reserves, and wind curtailment must be met in all
wind scenarios. In contrast, thermal units start-up costs are independent of the various wind
scenarios because we assume that the commitment of thermal units has to be fixed at the DA
stage.
We also assume that each thermal unit is offered into the market as a step-wise, price-quantity
offer function and that the offers can be derived by linearizing a standard quadratic production
cost function. Thus, we can express the operating cost for one thermal unit with the equations in
(4-6)(4-9). The coefficients for the generation blocks are derived from the quadratic production
cost function. The last part of the objective function is the start-up cost, which is modeled by
assuming that there is a cold start-up cost and a warm start-up cost, depending on the length of
time that a given unit has been ramped down. The mathematical formulation is shown in (4-10)
(4-12).
4.3.2

Thermal Unit Constraints

The constraints for the operation of thermal units include generation limits, ramping-up and
-down limits, and minimum-up and minimum-down time. The upper and lower generation limits
for the thermal plants are shown in (4-13). The maximum power output of a unit,
,is
constrained by the generation limit of a unit in (4-14), limitations on start-up and ramp-up rates
in (4-15), shut-down ramp rates in (4-16), and ramp-down limits in (4-17). The availability of
spinning reserves is equal to the difference between the maximum potential generation and the
actual generation, that is,

. Thus, the reserve requirement in (4-3) takes into account

the constraints imposed by (4-13)(4-17). The reserve requirement is maintained for each
individual wind scenario.
The final constraints included are the minimumup and down time constraints. Minimumup
times are represented by (4-18)(4-20), which represent the initial status, the intermediate time
periods, and the final time steps of the planning period, respectively. The minimumdown time
constraints are represented analogously by (4-21)(4-23). Note that the equations for generation
and ramping limits, (4-13)(4-17), must be included for all wind scenarios, because thermal
dispatch depends on the wind generation. In contrast, the minimum-up and -down time
constraints, (4-18)(4-23), are functions of commitment only and do not vary with wind
,
variable depends on the initial state and
scenarios. Note that the calculation of the
minimumon time constraints of the unit. If the unit has been on for less than the minimum on
time, the unit has to remain on until it meets the minimum on time constraint. A similar logic
,
applies to
as well. For simplicitys sake, transmission constraints are currently not
represented in the model, although their absence does not change the conclusions qualitatively.

40

(4-1)

s.t.
,
,

(4-3)

,
,

, ,

, , ,

0, ,

,
,

(4-4)
(4-5)

,
,

(4-2)

(4-6)

(4-7)

(4-8)

(4-9)

,
(4-10)

0,

1 ,

(4-11)

(4-12)

1
1

(4-13)

(4-14)

1
,

1
(4-15)

1 ,

41

1. .23,

(4-16)

1 ,

(4-17)

0,

(4-18)

1 ,
1

0,

,
,

1, . . ,

2, ,

(4-19)

(4-20)

0,

(4-21)

4.3.3

1
,

(4-22)
1, . . ,

0,

1
,

2, ,

(4-23)

Deterministic Formulation

In a simplified representation, the formulation above would consider only one scenario for the
forecasted wind generation. In such a case, the formulation would be equivalent to a
deterministic version of the UC problem. The selected scenario could be the expected wind
power generation or point forecast or could also represent a certain quantile in the forecasting
probability distribution. This formulation is useful for comparison purposes.
4.3.4

Economic Dispatch

In order to assess the dispatch cost in RT, we also develop an ED formulation. The commitment
variables are now assumed to be fixed from the UC run. The representation of wind power
generation by scenarios is replaced by the realized wind power output (without considering
potential wind power curtailment). Hence, we formulate a deterministic ED problem consisting
of equations (4-1)(4-9) and (4-13)(4-17), with only one wind power scenario and fixed values
for the thermal commitment variables, vj(k). The start-up cost and minimum-up and -down time
constraints are not considered because of the fixed commitment. Ramping constraints are in
(4-13)(4-17), and the 24-hour problem is solved in one shot. Notably, the operating reserve
requirement in (4-3) is also imposed in the ED formulation. As discussed in the introduction, we
assume that the dispatch is carried out with perfect knowledge of the actual wind power being
injected into the system for future time steps.

42

A market simulation has been set up that first solves the UC by making use of the wind power
forecast, and then solves the ED by using the realized wind conditions. This action is performed
in sequence for multiple days. An updated wind power forecast, along with the unit status and
generation output for the thermal units from the previous day, are taken as initial conditions for
the UC problem for the next day. The main results (UC, dispatch, available reserves, unserved
load, curtailed reserve, prices, etc.) are calculated and stored after each simulation day.
Because the focus is on quantifying the impact of wind power forecasts on system operation, the
only uncertainty we consider derives from wind generation. Other uncertainties, such as load or
forced outage, are not considered directly. We assume that operating reserves maintained in the
RT dispatch and based on classical criteria are adequate to accommodate these uncertainties. As
a result, the additional amount of wind power reserve becomes critical to addressing the
impact of uncertainty when using wind power. With the stochastic UC formulation, the need for
additional operating reserves is arguably already addressed because we include a representative
set of wind power outcomes in the scenarios. However, even if the scenarios are highly accurate
in capturing all of the possible wind power outputs, additional reserve may be needed even in the
stochastic formulation to cover any extra uncertainty that the simulated scenarios are not able to
capture. In the case study below, we run a number of different cases to investigate the impact of
UC strategy and operating reserve policy on the system dispatch.
4.4
4.4.1

Case Study
Assumptions

To provide a didactic and clear case study, we use a test power system to simulate the impact of
using different wind power forecasts and operating reserve policies. The UC and ED are run in
sequence as described above. The main assumptions for the case study are outlined below.
The characteristics of the thermal power plants are based on the case studies presented in [79]
and [81], although we have modified some of the cost coefficients and also introduced data for
ramping rates. The resulting input parameters are shown in Table 4-1. Each unit is represented
with four blocks of equal size. The bid price of each block is calculated by linearizing the
quadratic cost function. The production cost increases from baseload units to peak units. The
hourly profile of the loads is taken from the historical data for two utilities in the state of Illinois
for the three months of October through December of 2006. The overall load level is scaled
down to match the generation capacity in the test power system.
The operating reserve (spinning) requirement in the UC formulation is assumed to be 10% of the
system load as the default value. The required operating reserves have to be met by the thermal
power plants (i.e., wind power plants cannot provide reserves). Note that the stochastic model
allows the use of different reserve requirements in each scenario. In the case study, however, we
use the same value in all scenarios. We investigate the consequences of increasing the reserve
requirement in the UC, as discussed above, whereas the reserve requirement is kept constant at
10% of load in the ED problem, representing RT operations. We assume only one step in the
spinning reserve demand curve. The cost of reserve curtailment is $1,100/MWh, and the cost of
unserved energy is $3,500/MWh, which corresponds to the current practice at the MISO in the

43

United States [82]. Thus, reserves will be curtailed ahead of load. It should be noted that the
computational results are influenced by these penalty prices for unserved load and reserve. 12
Table 4-1 Generator data.
Unit

[MW]

[MW]

RLj a
[MW/h]

Base1
Base2
Int1
Int2
Int3
Int4
Int5
Peak1
Peak2
Peak3

[h]

[h]

In.
state
[h]
8
8
5
5
6
3
3
1
1
1

455
150
200
8
8
455
150
200
8
8
130
20
100
5
5
130
20
100
5
5
162
25
100
6
6
80
20
80
3
3
85
25
85
3
3
55
10
55
1
1
55
10
55
1
1
55
10
55
1
1
bj
cj
CCj
HCj
aj
Unit
[$/h] [$/MWh] [$/MW2h] [$/h] [$/h]
[h]
Base1
1,000
16
0.00048 9,000 4,500
5
Base2
970
17
0.00031 10,000 5,000
5
Int1
700
30
0.002
1,100
550
4
Int2
680
31
0.0021
1,120
560
4
Int3
450
32
0.004
1,800
900
4
Int4
370
40
0.0071
340
170
2
Int5
480
42
0.00079
520
260
2
Peak1
660
60
0.0041
60
30
0
Peak2
665
65
0.0022
60
30
0
Peak3
670
70
0.0017
60
30
0
a
Start-up and shut-down ramps, SUj and SDj, are equal to ramp rate RLj

4.4.2

Wind Power and Forecast Characteristics

We used time series of the DA deterministic point forecasts and realized wind power output for
15 hypothetical locations in the state of Illinois in 2006 as our wind power data, which were
obtained from the Eastern Wind Integration and Transmission Study [83]. The data were
produced by combining a mesoscale weather model with a composite power curve for a number
of potential sites for wind power farms. The DA forecasts were generated based on observed
forecast errors from four real wind power plants. The data methodology is explained in [84].
Data for the 15 sites were aggregated into one time series. The accuracy of the DA wind power
forecast varies from day to day. For the point forecast, the normalized mean average errors
(NMAEs) over a 91-day simulation period vary between 8.4% and 12.4% for different hours of
the day, with the highest forecast errors occurring in the afternoon between noon and 6:00 p.m.
The total installed capacity of wind power is assumed to be 500 MW, and for simplicity we
model this amount as one large wind power plant. For the simulated period, the wind power
capacity factor is 39.9%, and the wind power meets 20.2% of the load (with no wind
12

We have run sensitivity case studies on the penalty prices. The results show that, whereas the commitment and
dispatch of thermal units do not change much, the costs of reserve and load curtailment do naturally depend on these
values.

44

curtailment). Wind power and load are uncorrelated with a correlation coefficient of 0.01. With
these assumptions, the total installed capacity of the thermal units is 10.8% higher than the peak
load. If we assign a capacity value of 20% to the wind power capacity, the system reserve margin
increases to 17.4%.
To generate the wind power scenarios, we used the wind power data (forecasts and realized
generation) for the period from January to July to train the quantile regression, and then the
months from August to December are used as a test dataset. Because the first month is used only
to initialize the estimation of the covariance matrix, the scenarios are only produced for October,
November, and December.
4.4.3

Simulated Cases

To compare the results of using different wind power forecasts and UC formulations
(deterministic vs. stochastic), and also the different UC policy and reserve requirements, we have
conducted tests of a list of 10 simulation cases as summarized in Table 4-2. The cases are in
three subgroups: forecasts, reserve requirements, and stochastic UC. In the forecasts subgroup,
the first case (F1) is a reference case with a perfect wind power forecast. Case F2 uses the wind
power point forecast. In case F3, the UC is performed without considering the use of wind power
at all. In F4 and F5, different forecasts in terms of different quantiles are used. F4 uses the 20%
quantile forecast instead of the point forecast, which is similar to the current practice for DA
scheduling in ERCOT [85]. F5 uses the 50% quantile forecast and aims to investigate the
performance of the median forecast.
Table 4-2 Simulated cases.
Case
Description
Subgroup 1 Forecasts
F1
Deterministic (Det.) UC w/perfect forecast
F2
Det. UC w/point forecast
F3
Det. UC w/no forecast
F4
Det. UC w/20% quantile forecast
F5
Det. UC w/50% quantile forecast
Subgroup 2 Reserve Requirements
Det. UC w/additional reserve
R1
(20% of point forecast)
Det. UC w/additional reserve
R2
(point forecast 10% quantile forecast)
Det. UC w/additional reserve
R3
(additional 5% of load)
Subgroup 3 Stochastic UC
S1
Stochastic UC w/regular reserve
S2
Stochastic UC w/additional reserve (additional 5% of load)

UC

Forecast

Det.
Det.
Det.
Det.
Det.

Perfect
Point
No
Point
Point

Det.

Point

Det.

Point

Det.

Point

Stoch.
Stoch.

Scenarios
Scenarios

The reserve requirements subgroup illustrates the impact of different reserve requirements on the
deterministic UC results. We assume that a basic reserve requirement of 10% of the load is
adequate to accommodate load forecast errors and forced outages of generating units.
In R1, an additional 20% of the point forecast is added to the reserve requirement to
accommodate wind power uncertainty. In other words, the wind reserve, WR(k), in Eq. (4-4) is
equal to 20% of the point forecast. In R2, we calculate the additional wind reserve requirement

45

by deducting the 10% quantile forecast from the point forecast. Because the realized wind power
will most likely be higher than the 10% quantile forecast (90% of the time), R2 should be able to
accommodate the uncertainty in WPF to a large extent. In R3, we increase the level of reserves
from 10% to 15% of the load, so that in this case, the additional wind reserve does not consider
the information in the wind power forecast.
Finally, case S1 uses stochastic UC with a regular reserve requirement, and case 2 is the
stochastic UC with the additional reserve (15% of the system load instead of 10% in S1). The
stochastic UC cases use a total of 10 wind power scenarios, which not always be sufficient to
capture the full range of potential outcomes for the wind power generation for the next day. The
purpose of increasing the reserve in S2 is to use the additional reserve to cover the extra
uncertainty in wind power not captured by the scenarios.
4.4.4

Results

In this section, we provide detailed dispatch results for one selected day and the overall
simulation results to show the short-term impact of different system scheduling methods and
longer-term simulation statistics with different wind power forecasts.
4.4.5

Results for One Day

We present the dispatch results for day 87, which is selected because there is a relatively large
forecasting error on this day. Fig. 4-1 shows that both the point forecast and most of the
scenarios predict too much wind power for most of the day. Fig. 4-2 shows the number of units
on-line in cases F2, R3, and S1, which are the most likely alternatives to represent how system
operators may incorporate wind power forecasts into their UC. In this figure, we can see the
number of on-line units in S1 is higher than in F2 and higher in R3 than in S1.

500
Scenario 1

450

Scenario 2

400

Wind Power [MW]

Scenario 3

350

Scenario 4

300

Scenario 5

250

Scenario 6

200

Scenario 7

150

Scenario 8

100

Scenario 9
Scenario 10

50

point forecast

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Real generation

Time [Hour]

Fig. 4-1 Wind power forecast (deterministic point forecast and

10 stochastic scenarios) and realized wind generation for day 87.

46

This result is not surprising, given that more generating units should be expected to be scheduled
on-line in S1 compared to F1, to provide sufficient ramping capability to handle the different
wind power scenarios (Fig. 4-1). A parallel effect is obtained with R3 by imposing a higher
reserve requirement at the UC stage. Fig. 4-3 shows the level of available operating reserves in
the RT dispatch for the three alternatives. The available capacity surplus is derived by
, for each
subtracting the dispatch, ptj(k), from the sum of maximum feasible generation,
thermal unit and adding up the results. It is of note that in F2, the available capacity drops below
the required ED reserve requirement for several hours. This is particularly the case in hours 11
and 12 when there is a large deviation between the realized wind generation and the point
forecast. This deviation causes the use of all of the available capacity, including operating
reserves, and eventually leads to the load curtailment in those two hours (Fig. 4-3) for F2.
In this case, it is evident that the stochastic strategy S1 provided a better overall solution. The
available operating reserves from S1 match well with the ED reserve requirement in Fig. 4-3
with only minor shortages in hours 8 and 9, while R3 leads to an excessive amount of available
reserves as shown between hours 8 and 22 in Fig. 4-3.
4.4.6

Results for a Three-month Period

Fig. 4-4 shows the total hours of commitment for the ten thermal units in a three-month
simulation period. Total hours of commitment are the highest in F3, which does not take wind
power into account in the UC and therefore needs more units to be committed to compensate for
the absence of the wind power generation (available in all of the other cases). This case
illustrates that if system operators are not using the information in the wind power forecast, the
over-commitment of thermal units may easily result. The higher reserve requirement in S2, plus
the wind uncertainty (represented by the scenarios), led to a high commitment of units in S2, as
well. Case F2 has the lowest commitment of units, which leads to lower available capacity for
reserve and higher curtailment, as shown next.
Table 4-3 shows the average dispatch of thermal units for all of the cases. The results show that
the differences in dispatch are relatively smaller than are the differences in commitment. This
result occurs because the same realized wind generation output is used in the RT dispatch in all
of the cases. The resulting average thermal dispatch therefore becomes similar in the ten cases,
despite the differences in commitment. For the peaking plants, note that they are being
dispatched at their minimum levels most of the time while committed.

47

Number of units online

5
4
3
2
Deterministic (R3)

Deterministic (F2)
Stochastic (S1)

0
1

11 13 15 17 19 21 23
Hour

Fig. 4-2 Number of on-line units for day 87.

450
ED reserve requirement

Available reserve [MW]

400

Deterministic (F2)

350

Deterministic (R3)

300

Stochastic (S1)
Load curtailment in F2

250
200
150
100
50
0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Hour

Fig. 4-3 Available operating reserves in RT dispatch.

Fig. 4-4 Total hours of commitment for thermal units in all cases.

48

Unit
Base1
Base2
Int1
Int2
Int3
Int4
Int5
Peak1
Peak2
Peak3
Unit
Base1
Base2
Int1
Int2
Int3
Int4
Int5
Peak1
Peak2
Peak3

Table 4-3 Average dispatch for thermal units [MW].

F4 (20%
F1 (Perfect
F2 (Point
F3 (No
quantile
forecast)
forecast)
forecast)
forecast)
441.3
440.5
436.0
440.8
315.4
316.6
300.9
313.6
13.9
8.4
22.4
15.6
1.9
1.2
3.7
1.8
15.3
21.3
23.0
16.2
2.3
1.9
3.5
1.9
0.1
0.1
0.5
0.2
0.2
0.2
0.3
0.2
0.0
0.0
0.1
0.0
0.0
0.0
0.0
0.0
R2 (wind
R1 (wind pf
R3 (load
S1 (Stochastic
quantile
resrve)
reserve)
UC)
reserve)
440.5
440.5
440.5
440.5
314.0
312.5
313.1
312.0
9.5
15.5
12.9
14.1
2.1
2.0
1.9
4.0
21.0
16.8
18.6
16.8
2.9
2.6
3.1
2.4
0.1
0.2
0.1
0.3
0.3
0.2
0.2
0.3
0.0
0.0
0.0
0.1
0.0
0.0
0.0
0.0

F5 (50%
quantile
forecast)
440.8
315.9
11.2
1.2
18.2
2.5
0.2
0.3
0.0
0.0
S2 (Stochastic
+ reserve)
440.3
308.1
17.1
3.9
17.0
3.4
0.3
0.4
0.1
0.0

Table 4-4 and Table 4-5 summarize operating costs and other main results in ED. We can see, as
expected, that F1 has the lowest total operating cost because it assumes that a perfect wind power
forecast is available. In other words, the dispatch from the UC run does not need to change in the
RT ED. F2 has the highest cost as a result of the high load curtailment, which is caused by WPF
errors. Both of the stochastic cases have relatively low operating costs, and the lowest cost
occurs in S2. It is interesting to note that R2 performs slightly better than S1 in terms of the
operating cost. This result occurs because the additional reserve requirement was derived from
the WPF information, which leads to bringing more units on-line to provide the ramping
capability to accommodate the forecasting errors. The column of reserve curtailment in Table 4-5
shows the amount of curtailed reserve in the various cases. Because we assume the reserve
requirement is in the form of a demand curve, and the cost of unserved reserve is lower than the
cost of unserved load, a certain amount of reserve will be curtailed ahead of system load when
system reliability is hampered. Most cases have some unserved reserve, whereas only F2 and F5
have unserved load.

49

Case

F1
F2
F3
F4
F5
R1
R2
R3
S1
S2

Case

F1
F2
F3
F4
F5
R1
R2
R3
S1
S2

4.4.7

Table 4-4 Summary of operating costs.

Description
Fuel
Load
Reserve
Cost
Curtailment Curtailment
[M$]
Cost
Cost
[K$]
[M$]
Perfect forecast
34.86
0
0.00
Point forecast
34.78
282
13.02
No forecast
36.81
0
0.00
20% quantile
35.11
0
2.49
forecast
50% quantile
34.86
574.87
7.70
forecast
Wind pf reserve
35.06
0
5.05
Wind quantile
35.26
0
0.86
reserve
Load reserve
35.19
0
1.42
Stochastic UC
35.34
0
1.05
Stochastic +
35.83
0
0.06
reserve
Table 4-5 Summary of other results.
Description
Load
Reserve
Curtailment
Curtailment
[MWh]
[MWh]
Perfect forecast
Point forecast
No forecast
20% quantile
forecast
50% quantile
forecast
Wind pf reserve
Wind quantile
reserve
Load reserve
Stochastic UC
Stochastic + reserve

0
80.64
0

0.57
0.01
0.00

2,259.11

164.25

7,003.10

4,588.75

781.5

0
0
0

1,288.42
952.91
58.78

Total
Dispatch
Cost
[M$]
34.87
48.08
36.81
37.59
43.14
40.11
36.12
36.61
36.39
35.90

Average
Available
Reserve
[MW]
178
172
294
194
176
189
204.5
199
207
237

Use of Different WPFs

In subgroup 1, various WPFs are used. The total cost ranges from $34.87M in F1 to $48.08M in
F2. The total cost in F3 is lower than the operating cost in F2 as a result of F2s high curtailment
cost. The difference between F2 and F3 illustrates the importance of avoiding extreme errors in
WPF, where the wind power forecast has a very large magnitude or phase error compared to the
realized wind power generation. This event could occur within one hour or a limited number of
hours even though the average forecast error over the entire period may be small. In fact, load
curtailment occurs in only four hours out of the 91-day simulation in F2. Nevertheless, reserve
curtailment occurs as often as 17% of the time in F2.
Forecasting errors also lead to reserve curtailment. In these situations, the system operator is
assumed to use the RT operating reserve intended for unexpected deviations in load and thermal
generation to accommodate the WPF errors. These events cause great challenges for system
50

operations because the generating units are not scheduled accordingly, and sufficient ramping
capability is therefore not available. However, F3 ends up with fuel costs that are higher than
they are in any of the other scenarios, because more high-cost generating units are being
dispatched compared to what is needed, given that wind power is not considered at all in the UC
problem.
F4 is the 20% quantile forecast, which means that the realized wind power is higher than this
forecast with a probability of 80%, and is used in place of the point forecast in the UC. The
commitment results show that more units are committed in F4 compared to F2 because of using
the conservative forecast for wind power. As a result, the curtailment cost and the total operating
costs are both much lower in F4 than in F2. The operating cost in F5 is much higher than it is in
F3 and F4, which shows that it is too risky to rely on the 50% quantile forecast in the UC.
4.4.8

Reserve Requirements

Subgroup 2 compares the impact of the reserve requirements on the system dispatch. The
additional reserve in this subgroup is meant to compensate for the WPF error in a predefined
manner. This predefined amount could be inaccurate as discussed above. The total hours of
commitment in Fig. 4-4 show that the three reserve requirements have a similar effect on the
commitment results because more units are required to be on-line to provide extra reserve.
However, case R1 suffers from the wind power point forecast errors, to some extent, because the
additional reserve requirement is a function of the point forecast. If the forecast deviates
substantially from the realized wind power, the resulting reserve may not be sufficient to handle
the forecast error. In this subgroup, R2 uses the best strategy. In this case, the possible
forecasting errors are largely compensated for by using the larger extra reserve, which is the
difference between the point forecast and the 10% quantile forecast. The operating cost in R3
turns out to be slightly higher than that of R2 because the reserve requirement in R3 is linked
with the system load, not the WPF. Thus, because the load is not strongly correlated with the
wind power output, the reserve associated with load may be incorrect.
In comparison to subgroup 1, the strategies in R2 and R3 lead to lower costs than those of F3 and
F4, where no forecast or a conservative forecast is used. This result shows, as expected, that
increasing the reserve requirements in the deterministic approach has the effect of better
accommodating wind variability and uncertainty. The resulting operating reserve is higher in R2
and R3 compared to all of the cases in subgroup 1 except F3 (Table 4-5). Because F3 can be
regarded as the most conservative way to commit the units that is, by ignoring the existence
of wind power the level of realized operating reserves is therefore much higher than it is in
the other cases.
4.4.9

Deterministic vs. Stochastic UC

Fig. 4-4 shows that the units are on more frequently in S1 and S2 than in the deterministic cases,
except in F3 where there is no wind power forecast. This result occurs because the units need to
be on to provide the additional ramping capacity to deal with the variability of wind power
output in the scenarios in the stochastic approach.
In the stochastic cases S1 and S2, we can see that the total operating costs end up being close to
deterministic case F3, which has no wind power forecast, and cases R2 and R3, which have
51

additional operating reserves. S2 shows the best performance in terms of the total cost among all
of the cases because both the wind power variability in the scenarios and the limited accuracy of
the scenario generation are taken into account. For the same reason, the available reserve is also
relatively higher in S2. This observation is consistent with [9], where the increased reserve is
used to cover the additional generator outages and load variations, which are not captured by the
scenarios.
The results show that the UC strategy and the reserve requirements have important implications
for managing the costs and reliability of power systems with large amounts of wind power. The
ramping capability and reserves provided by on-line units are, to a large extent, determined by
these factors, which therefore influence the RT dispatch results to a great extent. Again, it should
be noted that the RT dispatch can be further improved if intra-day rescheduling of generating
units is applied, which is not considered in this chapter.
4.5 Conclusions
This chapter analyzes the impact of WPF on UC and ED. More than elaborating on theoretical
models, it is important to assess whether the difference in decisions, resulting from moving from
deterministic models to stochastic programming models, is relevant enough to justify the
attention of the industry to changing the way they operate the system. Therefore, the studies
reported in the chapter were carefully organized in a series of experiments to cover a whole set
of strategic variants. The results show that the added uncertainty resulting from WPF uncertainty
requires the adoption of an improved UC policy that can define higher spinning reserves than
those used in classical systems that do not rely on wind resources. This result is of major
economic impact to system operators.
The simplest approach to this requirement of setting a larger reserve margin is to define a larger
fixed value (in terms of percentage of the load) for the spinning reserve so that not only errors
in WPF but also in load forecast and the possibility of unplanned outages are covered. Our
results show, however, that an adaptive strategy to define adequate reserves should be put in
place, because WPF errors vary importantly from hour to hour. Our results also show that a
probabilistic wind power forecast can be used to define more adequate reserve requirements.
In addition, the example shows that the desired level of reserves is more likely to be achieved
with a stochastic UC formulation that explicitly takes into account the varying uncertainty in
wind power predictions. One of the major contributions of this chapter is to adopt scenarios with
temporal correlation of the errors within each scenario when applied to the UC problem. Most
approaches published to date have been using Gaussian representations of WPF uncertainty that
are in the form of marginal distributions, which do not represent time dependency 13 a
treatment that results in gross approximations, because the UC problem also has a strong timedependency component in its definition (ramping, shut-down/start-up decisions, and, if hydro
power is present, the use of water). A stochastic UC formulation with representative scenarios is
clearly better, because it can handle the temporal correlation in forecasting errors as opposed to
explicit reserve requirements.
13

An exception is the WILMAR model, which does not use a Gaussian representation of WPF uncertainly, but uses
an approach taking the temporal correlations of the forecast errors into account [66].

52

In conclusion, the chapter sustains that system operators may move with confidence to the
adoption of probabilistic (stochastic) decision methods in order to reduce costs and risks. In fact,
the adoption of a blind fixed criterion for the reserve tells nothing about the credibility of its
assumptions or about the risks. It may happen that in some instances, a fixed reserve value may
approximate the results from a stochastic decision process however, one can never be certain
about this beforehand. In the presence of the large and unavoidable wind power prediction errors,
especially in thermal-dominated power systems, stochastic approaches should prove to be more
useful. A first step would be to move toward the use of adaptive reserve requirements, which are
a function of the wind power forecast. Next, operators can deploy an entirely stochastic
approach, in which the wind power forecast uncertainty is fully represented in the UC
formulation.

53

This page intentionally blank.

54

REPRESENTATIVE WIND POWER SCENARIOS AND THEIR PROBABILITIES

FOR STOCHASTIC MODELS 14

This chapter analyzes the application of clustering techniques for wind power scenario reduction.
The results show the unimodal structure of scenarios generated under a Monte Carlo process.
The unimodal structure has been confirmed by the modes found by the information theoretic
learning Mean Shift algorithm. The chapter also presents a new technique that is able to
represent the WPF uncertainty by a set of representative scenarios capable of characterizing the
probability density function of the wind power forecast. From an initial large set of sampled
scenarios, a reduced discrete set of representative scenarios associated with a probability of
occurrence can be created finding the areas of high-probability density. This discrete set will
allow the reduction of the computational burden in stochastic models that require scenario
representation, such as the stochastic UC models presented in Chapter 4.
5.1 Introduction
The continuous increase of wind power penetration is introducing more variability and more
volatility in electricity generation. This challenges the management and planning of the electric
power system. To cope with uncertain wind power production, efforts are constantly invested in
developing more accurate wind power forecasts. A common form of delivering the wind power
forecast is called point forecast and represents a single value for each look-ahead time horizon.
However, no forecast can be perfect. When the forecast is delivered as a point forecast, it
commonly contains only the conditional expectation at each time step. However, an expectation
cannot help in calculating risks or the conditional values at risk: one needs a description of the
probability density function (pdf).
The power industry is in need of estimates of forecasting uncertainty, besides the point forecasts.
The most common description has been in the form of marginal distributions for each time step.
The presentation of these results, usually, is done in the form of intervals or quantiles, but this is
a somewhat misleading form of presentation, because it gives the illusion of trajectories or
time sequences. However, at each time step what one has is a marginal distribution, regardless of
what happened in the previous time steps the time dependency is absent from the
interval/quantile representation.
However, for some applications, one cannot neglect time dependencies: it is the case, for
instance, of UC decisions to be made by system operators. Because of technical constraints, such
as ramping limits or minimum shutdown and start-up times, and because of the nature of costs,
the decisions to be made also involve time dependencies.
The traditional practice calls for deterministic models to be used by system operators in the
decision-making process. This was acceptable when the uncertainties were of limited amplitude,
such as in load forecasting. However, wind generation uncertainty tends typically to be
significantly greater than load forecasting uncertainty. Hence, the description of uncertainty must
be incorporated in the models.
14

This chapter is reproduced from Proceedings 16th Int. Conference on Intelligent System Application on Power
Systems (ISAP), J. Sumaili, H. Keko, V. Miranda, Z. Zhou, A. Botterud, J. Wang, Finding Representative Wind
Power Scenarios and their Probabilities for Stochastic Models, Greece, Sept. 2011, with permission from IEEE.

55

Because the decision models are usually based on mathematical programming approaches, the
logical way to include uncertainty in a multiple time step optimization process is to adopt
stochastic programming models. These models usually work with scenarios associated with
probabilities. In the case of WPF, a scenario is a time series or a particular sequence of values
representing an assumed possible realization of wind power along some period. A thorough
description of wind power uncertainty would be by a probability density function in a space of
multiple dimensions (the number of time steps within the prediction horizon). An approximate
representation of this pdf can be obtained with a set of scenarios, sampled from the probability
density function representing the historical (observed) error distribution. While this method is
favorable since it is able to carry the information on the temporal development of uncertainty, the
primary drawback of the scenario representation is its implied computational burden: a high
number of scenarios is needed to accurately represent the associated uncertainty. This means that
the decision making involving stochastic optimization may turn prohibitively hard for such a
large set of scenarios.
In this chapter, we analyze a technique of representing the wind power forecast related
uncertainty with a reduced set of scenarios. The development of a reduced scenario set is based
on clustering methods. Within the process of reducing the initial large set of scenarios, the newly
obtained scenarios are also associated with the probability related to the cluster it represents. The
newly obtained set of representative scenarios could then be applied to computationally heavy
problems.
The proposed methodology assumes departing from a large set of wind power scenarios
generated using a Monte Carlo sampling based method. Each scenario is actually a Monte Carlo
sample or a point in a space whose dimension is equal to the number of time steps considered.
This large set of scenarios should be a suitable approximation of the probability density function
of wind power. One of the possible approaches to generate these scenarios is presented in [78].
The high-density areas are substituted by a single representative or focal scenario. Departing
from the reasoning that the initial samples are sampled by using a Monte Carlo process, the
relative weight of each high-density area with regard to all of the samples becomes the
probability of the representative scenarios representing that area.
In this work, the scenarios sampled by using the process described in [78] are analyzed and the
structure of such scenarios is shown to be unimodal, by means of using the information theoretic
learning (ITL) Mean Shift algorithm [86], [87].
The goal of the presented work is to find an adequate representation of the shape of the
probability density function by using a reduced set of scenarios with assigned probabilities.
Having the assigned probabilities makes the proposed reduction method adequate for risk
assessment. The scenario reduction topics from the literature commonly use the scenario tree
construction method from [88], [89]. This work instead uses a clustering approach without
assuming any particular temporal development characteristic of errors. Given a set of initial
scenarios, the reduction level is only determined by the tolerance admitted.

56

Depending on the particular power system characteristics, the wind forecast uncertainty may
carry large hedging costs. The reduced set of scenarios obtained by clustering is a discrete
representation of the probability density function using a smaller weighted set of wind power
scenarios. Such a smaller set of scenarios may increase the feasibility of using risk assessment
models in a practical setting, which is favorable for managing the hedging costs.
This chapter describes the details of the model developed and presents the results for three DA
forecasted wind power scenarios, in a study case built with realistic data. The use of scenario
reduction methods for stochastic UC is further explored in Chapter 6.
5.2 Scenario Generation and the Need for Scenario Reduction
The initial sets of scenarios used refer to DA wind power forecasts in 2006 for a hypothetical
wind farm located in the state of Illinois. Time series of DA deterministic point forecasts were
obtained from the Eastern Wind Integration and Transmission Study [83].
To produce wind power scenarios, wind power data (forecasted and realized) were used to train
the uncertainty estimation model, as well as to generate scenarios for the forecasted wind power,
according to the methodology introduced by Pinson et al. [78], which is equivalent to producing
scenarios under a Monte Carlo process.
The work presented shows how one can organize a large set of scenarios into a small set of
clusters, based on a principle of maximum density; associate each cluster with an empirical
probability value; and represent each cluster by a focal or representative scenario. This technique
allows the organization of a cumulative distribution of clusters. This permits not only the
selection of the most representative equivalents but also keeps a quantitative (probability)
evaluation of the coverage of possibilities guaranteed and, of course, of the risk of missing the
actual wind power series.
5.3
5.3.1

Wind Power Scenario Clustering and Modes Finding

Application of Classical Clustering Techniques to Wind Power Scenarios

We apply the classical pattern clustering techniques in order to identify high-probability density
areas characterized by similar wind power scenarios.
The hierarchical clustering methods rely on a measure of similarity (for agglomerative
clustering), in order to build up the clusters from the samples. One starts with the lowest level of
hierarchy where each cluster contains a single member, that is, the number of clusters is equal to
the number of scenarios and merges (agglomerates) progressively the two most similar clusters
[90]. At the very highest level of hierarchy, there is a single cluster containing all of the data. At
the intermediate levels of hierarchy, a common behavior of the hierarchical clustering methods is
to group the samples into a single relatively large cluster, while the rest of the clusters contain
outliers. This behavior is observed for the wind power scenarios generated according to [78].
On the other hand, the classic K-means algorithm [91] leads to the separation of the initial
scenario set into different clusters; however, this clustering method maintains relatively high
dispersion inside the clusters. This is due to the unimodal structure of the initial scenario set
generated by the model [78] behind the Monte Carlo sampling.

57

5.3.2

Modes Finding using the ITL Mean Shift Algorithm

The hypothesis that the initial scenario distribution produced by the technique proposed in [78] is
unimodal is confirmed by using the ITL Mean Shift algorithm [86], [87]. The Mean Shift is a
nonparametric, mode-finding method that relies on a kernel-based construction of the search
space surface.
The mode-finding method was applied to confirm that the actual distribution represented by the
initial set of scenarios is unimodal, which seems to present a difficulty for the classic clustering
methods, as discussed in the previous subsection. When the Mean Shift algorithm is applied to a
single-day data set, it finds just a single mode, confirming the hypothesis.
A test of the ability of the ITL Mean Shift algorithm to deal with multimodal wind scenario
distributions was designed, as well. A total of 1,000 scenarios from three different days (Fig. 5-1
through Fig. 5-3) were used as the input data set, totaling 3,000 scenarios. Each of the separate
days should have a single mode, and the three-day data set, when merged together as if it were a
single day, should then be three-modal.

Fig. 5-1 Day 1 wind power scenarios.

58

Fig. 5-2 Day 2 wind power scenarios.

Fig. 5-3 Day 3 wind power scenarios.

Fig. 5-4 illustrates the results of the ITL Mean Shift algorithm with = 1 and = 0.18 (for
details of the algorithm, refer to [86][87]).

59

Fig. 5-4 Day 3 Mode finding results for the three-day set of
scenarios thick lines represent the modes discovered, coherent
with the merged sets.

The ITL Mean Shift algorithm was successful in finding the three modes in the merged scenario
set resulting from joining the three days into a single set. Each mode is represented by a thick
line, and it represents the three main evolutionary trends present in the global scenario set.
This property may suggest that the ITL Mean Shift algorithm can be used effectively when the
set of scenarios is produced from ensembles and merged the information is not lost and can be
recovered.
5.4 New Methodology
Given a set of M wind power scenarios sampled according to some probability density function
describing the uncertainty in forecasts, the aim of the reduction process is to merge scenarios
presenting high similarity in order to produce a new reduced set, containing the representative
scenarios only.
Each set of merged scenarios forms a cluster and is associated to a certain frequency within the
sample its representative or focal scenario is therefore associated with the same probability
value.
Let us denote the forecasted wind power as a series:
p(m) = { p1(m), , pT(m)},

(5-1)

for time horizon t = 1, , T in scenario m = 1, , M. This is also a point in a space of

dimension T.
The forecasted wind power scenarios p(m) are normalized with respect to the installed power of
the wind farm (or generator) in order to obtain the vectors x(m), whose components lie in the

60

range [0, 1]. Alternatively, this range can be changed to [0, 100] if the forecasted power is
expressed as a percentage of the installed power.
Different metrics can be used to define the distance between two scenarios x and x . For the
work presented in this paper, the maximum deviation has been used:
d

x , x

max X

, t

1, , T

(5-2)

where x and x are the two scenarios that are being compared, and T is their dimension.
The reduction procedure begins with an extremely large set of wind power scenarios, and then
one finds the scenario presenting the highest number of neighbors according to a defined
tolerance . Two scenarios x and x are considered to be neighbors if the following is
satisfied:
d

x , x

(5-3)

The identified scenario along with its neighbors are then cut from the initial set and grouped in a
cluster. The process is repeated with the remaining part until the scenarios do not have any
neighbor within the defined tolerance. In this way, a set of clusters is defined. Each of them is
associated with a focal element together with a probability value.
For this method, it is necessary to define a tolerance for the scenario aggregation process. The
value of this tolerance is a trade-off between accuracy and set reduction capability, which is
related to the computational effort.
After identifying a cluster and its members, the attracting scenario is taken as the representative
or focal scenario of the cluster and replaces the cluster as a condensed discrete representation.
Another option is to calculate the mean value of the cluster to obtain the centroid of the cluster.
This scenario is thus produced by averaging the cluster members. A third option is to apply the
ITL Mean Shift algorithm to find the mode within the cluster. In these latter cases (average or
mode), the result is usually not an existing scenario.
One should be careful in choosing the options for calculating the representative scenario.
Averaging the members of the cluster may artificially introduce smoothing effects in the new
artificial scenario. This average scenario would not have been generated by the initial
scenario-generation process, so its behavior may not be in line with the original probabilistic
model that captured the predicted behavior of the wind power.
On the other hand, if a cluster member is chosen as the representative scenario, therefore
becoming the focal element, the total expected value may deviate from the expected value of the
whole set of scenarios. The preferred method for the focal scenario calculation thus depends on
the purpose of the scenario reduction.
Each focal scenario is then associated with a probability value, calculated as the ratio between
the number of scenarios in the cluster it represents and the total number of scenarios. Finally,
after choosing the representative scenarios and calculating the corresponding probabilities, the
61

continuous probability density function of wind power initially represented by a large discrete
sample becomes represented by a much smaller discrete set of focal scenarios, each with a
probability value assigned.
5.5 Results
The methodology described above was applied to the sets of 1,000 scenarios for the three days
illustrated in Fig. 5-1 through Fig. 5-3 in Section 5.3.2. The scenario reduction results are
summarized in Fig. 5-5 through Fig. 5-7 below.
probability distribution

set of 1000 scenarios

1

representative scenarios

16

wind power

0.6

0.4
0.2

0.8

12
wind power

scenario probability (%)

14
0.8

10
8
6
4

0.6

0.4
0.2

2
0

10
15
time horizon

20

25

40
cluster #

60

80

0.8

0.8

0.8

0.4
0.2

wind power

0.6

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

20

25

5 most representative scenarios (42 % )

cluster # 2 with 107 members

wind power

wind power

cluster # 1 with 151 members

20

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

20

25

Fig. 5-5 Day 1 scenario reduction results with = 0.22.

probability distribution

set of 1000 scenarios

wind power

0.8
0.6

0.4
0.2
0

10
15
time horizon

20

40

0.8

30
20
10
0

25

0.6

0.4
0.2

20

40

60

cluster # 2 with 74 members

0.8

0.8

0.8

0.6
0.4
0.2

wind power

wind power

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

20

25

5 most representative scenarios (60 % )

cluster #

cluster # 1 with 403 members

wind power

representative scenarios

50

wind power

scenario probability (%)

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

Fig. 5-6 Day 2 scenario reduction results with = 0.18.

62

20

25

probability distribution

set of 1000 scenarios

wind power

0.8
0.6

0.4
0.2
0

10
15
time horizon

20

20

0.8

15
10
5
0

25

0.6

0.4
0.2

20

40

60

cluster # 2 with 128 members

0.8

0.8

0.8

0.6
0.4
0.2

wind power

wind power

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

20

25

5 most representative scenarios (60 % )

cluster #

cluster # 1 with 245 members

wind power

representative scenarios

25

wind power

scenario probability (%)

0.6
0.4
0.2

10
15
time horizon

20

25

10
15
time horizon

20

25

Fig. 5-7 Day 3 scenario reduction results with tolerance = 0.18.

On the top line, the three windows illustrate the initial set of scenarios; the probability
distribution associated to the clusters, ordered by decreasing probability value; and the whole set
of representative scenarios, obtained with the full distilling of the data set. On the lower line, the
two first windows illustrate the two more populated clusters with their corresponding focal
scenarios, and the last figure presents the five most representative focal scenarios (with highest
probability). The coverage of the possibilities, when only these five scenarios are retained, is
indicated.
With the admissible tolerance set to 0.22, the initial 1,000-scenario set of day 1 has been
reduced to 70 clusters and 150 outliers (Fig. 5-5). The 70 clusters represent 85% of the total
number of scenarios. The five most representative scenarios represent only about 42% of the
1,000 initial scenarios.
If one decides to run a stochastic model only with the five most representative focal scenarios,
one is missing 58% of the scenario representation, which may be considered as a risk measure
the risk of missing the adequate representation of wind power, for accepting the 5 focal scenarios
as representative of the real distribution. To reduce the risk, one has to consider the addition of
more scenarios. Naturally, this addition should be in an order following a decreasing value of the
probability assigned to each cluster.
Fig. 5-6 and Fig. 5-7 present results for days 2 and 3 with tolerance set to 0.18. The scenario
reduction procedure has achieved 58 representative scenarios and 112 outliers (the five most
representative scenarios contain about 60% of the 1,000 initial scenarios) for day 2 and
52 representative scenarios and 50 outliers (the five most representative scenarios contain about
60% of the 1,000 initial scenarios) for day3.
The number of obtained representative scenarios depends on the tolerance value and on the
dispersion of the initial scenario data set. Fig. 5-8, Fig. 5-9, and Fig. 5-10 illustrate the
distribution of the maximum deviation between any pair of the original scenarios for days 1, 2,
63

and 3, respectively. Notice that day 1 presents the lowest number of pairs of scenarios displaying
a maximum deviation lower than 0.1; in other words, day 1 has a lower number of similar
scenarios according to the maximum deviation distance metric. This result sets the base and
rationale for defining tolerance values with a distinct value for different days.
10

x 10

9
8

occurences

7
6
5
4
3
2
1
0

0.1
0.2
0.3
0.4
maximum deviation between 2 scenarios

0.5

Fig. 5-8 Day 1 distribution of the maximum deviation between each pair of 2 scenarios.
x 10

8
7

occurences

6
5
4
3
2
1
0

0.1
0.2
0.3
0.4
maximum deviation between 2 scenarios

0.5

Fig. 5-9 Day 2 distribution of the maximum deviation between each pair of 2 scenarios.
5

x 10

4.5
4

occurences

3.5
3
2.5
2
1.5
1
0.5
0

0.1
0.2
0.3
0.4
maximum deviation between 2 scenarios

0.5

Fig. 5-10 Day 3 distribution of the maximum deviation between each pair of 2 scenarios.

64

5.6 Conclusion
Decision-making processes in the operation of a power system must take into account the
uncertainty in wind power prediction. The most convenient way to represent this uncertainty is
by a set of scenarios.
A discrete approximation of the probability density function representing the uncertainty in WPF
may be provided by a sampling procedure of the Monte Carlo type. However, to have an
accurate representation of the probability density function covering all of the forecasting space,
an extremely large set of elementary scenarios is needed. But the size of this set is usually
incompatible with the computational complexity of stochastic programming algorithms, which
are time demanding and require high computational effort.
The methodology proposed in this paper can substitute a large scenario set by a smaller set of
clusters, each one being replaced by a representative scenario associated with the probability of
the cluster it represents.
The reduction capability depends on the admitted tolerance, the distance metric used to define
the clusters, and the scenario dispersion. The choice of the metric and the admissible tolerance
has to be defined in accordance with the problem for which the scenario reduction is being made
and the level of accuracy needed for the results.
The method not only produces a set of focal or representative scenarios, but also orders them by
probability value. This allows retaining a selected number of scenarios and understanding the
risk of missing an adequate representation for truncating the scenario set. It also allows one to
choose the necessary number of scenarios in order to satisfy a maximum level of risk.
This work, therefore, is a contribution to facilitating the adoption of stochastic optimization
models in operation planning, a move that is seen as necessary to embed the uncertainty in WPF
in the system operators decision-making processes.
Stochastic optimization models for UC and wind power trading are presented and applied in
Chapters 4, 6, and 7.

65

This page intentionally blank.

66

UNIT COMMITMENT AND OPERATING RESERVES WITH PROBABILISTIC

WIND POWER FORECASTS 15

This chapter expands on the previous two and discusses in more detail how probabilistic wind
power forecasts can serve as an important tool to efficiently address wind power uncertainty in
power system operations. In particular, we compare different probabilistic forecasting and
scenario reduction methods, and test the resulting forecasts on the stochastic UC model presented
in the Chapter 4. The results are compared to deterministic UC, where dynamic operating reserve
requirements can also be derived from the probabilistic forecasts. In both cases, the use of
probabilistic forecasts contributes to improve the system performance in terms of cost and
reliability.
6.1 Introduction
The increasing penetration of wind power and other renewables are posing new challenges to
power system operators in grid management and generation scheduling. The inherent uncertainty
and variability from wind power calls for new approaches to the traditional UC and ED
problems. Stochastic UC has been proposed as one approach to better handle the wind power
uncertainty in market operations [55][58] and [66][68],[70]. Furthermore, new approaches to
calculating operating reserve requirements under wind power uncertainty have also been
proposed (e.g., based on assumed normal distributions of WPF errors [29][48], or on a
probabilistic wind power forecast [75]). It is obvious that WPF can provide important inputs to
the operational decisions of the system operator. However, probabilistic WPF is still a relatively
new area of research, and there is limited experience in the use of such forecasts in system
operations so far.
In order to investigate the consequences of using different UC strategies and reserve
requirements in the operation of the electricity market, we have developed a stochastic UC
model [60][86] as presented in Chapter 4. The stochastic model determines the commitment of
thermal generating units by minimizing the expected cost over a set of WPF scenarios.
Alternatively, quantiles from probabilistic WPF can be used to determine operating reserve
requirements in a deterministic UC. In either case, the quality of the probabilistic forecasts is
clearly of major importance for the cost effectiveness and reliability of the resulting scheduling
and dispatch.
In this chapter, we use the model to analyze the scheduling of energy and operating reserves
under wind power uncertainty. We compare different statistical methods for the generation of
probabilistic forecasts and also compare two different scenario reduction approaches.
Furthermore, we compare the use of stochastic and deterministic UC strategies and operating
reserves requirements. The main contribution of the chapter is in the systematic analysis of
probabilistic WPF methods and their potential use in power system operations. The results and
recommendations are highly relevant for ongoing efforts to improve the management of wind
power uncertainty and variability in electricity markets.

15

This chapter is based on Proceedings IEEE Trondheim PowerTech 2011, A. Botterud, Z. Zhou, J. Wang, J.
Valenzuela, J. Sumaili, R.J. Bessa, H. Keko, V. Miranda, Unit Commitment and Operating Reserves with
Probabilistic Wind Power Forecasts, Trondheim, Norway, June 2011, with permission from IEEE.

67

The chapter has the following structure: We provide a brief overview of the UC/ED model used
in this chapter in Section 6.2. In Sections 6.3 and 6.4, we describe different approaches to
probabilistic WPF and scenario reduction. Then, in Section 6.5 we compare the use of different
forecasts and operational strategies in a case study of a test system with wind power and thermal
generation. Finally, we summarize our findings, draw conclusions, and point out directions for
future work.
6.2 Scheduling of Energy and Operating Reserves
We use the model first from the previous chapter to analyze the scheduling of energy and
operating reserves in the electricity market. 16 As a reminder, the objective of the UC model can
be expressed as:

(6-1)

The first component corresponds to minimizing the expected sum of fuel costs from thermal
generators (FC), the cost of unserved reserve (C(RNS)), and the cost of unserved energy
(C(ENS)) over a set of scenarios (s). The last part of the objective function is the thermal
generator start-up costs (SC). The problem is assumed to have an hourly time resolution and is
solved for the next day. Two important constraints are to meet the demand for load and operating
reserves (OR) in all time periods (t) and scenarios (s):
,,

,,

(6-2)

(6-3)

Both thermal generation and wind power contribute to meet the load, but we assume that only
thermal generation can meet the operating reserve. We only consider one type of operating
reserves: spinning reserves, which are provided by committed thermal units. 17 However, we
differentiate between reserve requirements to meet uncertainty in thermal generation and load
(ORreg) and wind power uncertainty (ORwind). If there are scenarios with scarcity where operating
reserves or load are not being served, this is penalized in the objective function. The traditional
UC constraints are also included, such as ramping constraints and minimum-up and -down
constraints. Note that the model becomes deterministic simply by assuming that there is only one
scenario.

16

Some changes have been made to the model to improve the run-time. First, an improved representation of the
binary commitment variables for thermal units has been implemented based on [92]. Second, the model is now
implemented in AMPL and uses the state-of-the-art solver CPLEX for the resulting mixed integer linear (MIP) and
linear (LP) programming problems.
17
This assumption is obviously a simplification. For a discussion of different categories of operating reserve
requirements, we refer to [76].

68

In this analysis, we focus on the impact of wind power uncertainty. We assume that commitment
decisions are made based on a DA wind power forecast. Then, we calculate the resulting RT
dispatch with the realized wind power generation. 18 The systems ability to handle RT
fluctuation in wind power will depend on both the UC strategy and the operating reserve policy.
UC schedules the generating units over the next day, taking inter-temporal constraints into
account. Operating reserves impose additional constraints on unit availability for each individual
hour. The wind power uncertainty can be accommodated by using a stochastic UC, which
considers a range of scenarios for wind power generation for the next day. The wind power
scenarios can be generated from probabilistic WPF. If the scenarios give a good representation of
the wind power uncertainty, it should not be necessary to impose an additional operating reserve
for wind, ORwind. Alternatively, a deterministic UC model could be used to handle the wind
power uncertainty through the additional reserve requirement. In this case, ORwind could be
derived from probabilistic WPF. With the first approach, additional reserves are implicitly added
through the stochastic UC, as opposed to the second approach, which uses an explicit reserve
requirement for wind power. In either case, the performance of the scheduling decisions would
depend on the quality of the probabilistic forecast. The different operating strategies are explored
in the case study in Section 6.5.
6.3 Probabilistic Wind Power Forecasts
There are three main methods for estimating WPF uncertainty: (1) the Numerical Weather
Prediction (NWP) point forecast approach, where the wind power uncertainty is directly
computed from the NWP point forecast for weather; (2) the power output point forecast
approach, which consists of forecasting uncertainty based on the historical WPF errors; and
(3) the NWP ensemble approach, where NWP ensembles are used as input to the generation of
probabilistic power forecasts. For a detailed discussion of state-of-the art in WPF and
probabilistic forecasts, we refer to [1].
In our research, we are focusing on the statistical algorithms used in probabilistic WPF methods
(1) and (2) above. A common approach is to use quantile regression (QR) to estimate the WPF
uncertainty in terms of set of forecast quantiles [77]. In the project, we have developed novel
statistical uncertainty forecasting approaches using kernel density forecasts (KDFs). Two new
KDF-based WPF methods based on the Nadaraya-Watson (NW) and Quantile-Copula (QC)
estimators are proposed in [93], [94], and [95].
KDF methods are based on conditional density estimation, which consists of estimating the
density of a random variable, Y, knowing that the explanatory random variable, X, is equal to x.
For the wind power problem, this consists of forecasting the wind power probability density
function (pdf) at time step t for each look-ahead time step t+k of a given time-horizon when
knowing a set of explanatory variables (e.g., NWP variables and measured wind power output).
Mathematically, this set of conditions can be formulated as:

18

Note that we do not consider the possibility of changing commitment decisions between DA scheduling and realtime dispatch. Intra-day commitment would clearly add flexibility to operational decisions. However, we focus on
the impact of different UC and operating reserve strategies and do not consider the additional flexibility from intraday commitments.

69

f (p , x )
fP ( pt + k | X = xt + k |t ) = P, X t + k t + k |t
f X (xt + k |t )

(6-4)

where pt+k is the wind power forecasted for look-ahead time t+k, xt+k|t are the explanatory
variables forecasted for look-ahead time step t+k and available at time step t, fP,X(pt+k,xt+k|t) is the
multivariate density function, and fX(xt+k|t) is the marginal density of X. The nonparametric
conditional density estimation [96] for (6-4) is as follows:
N

f ( p | X = x ) = K h p ( p Pi )
i =1

K hx ( x X i )

(6-5)

K (x X )
i =1

hx

where the kernel bandwidth hp controls the smoothness of each conditional density in the p
direction, while hx controls the smoothness between conditional densities in the x direction.
The output is a pdf for wind power generation for each look-ahead time step of a specific time
horizon. The pdf is general, from which several uncertainty representations can be derived
(e.g., standard deviation, quantiles). For details on the implementation of the KDF WPF
methods, we refer to [93], [94], and [95]. The forecasted pdfs for each look-ahead time step are
the input of the scenario generation method described in Section 6.4. In the case study in
Section 6.5, we compare the use of probabilistic forecasts from splines QR and KDF based on
the NW estimator.
6.4 Scenario Generation and Reduction
The probabilistic forecasts described above express uncertainty information for specific points in
time. However, in the UC problem, it is important to take inter-temporal relationships in the
forecast uncertainty into account. This makes scenarios a more appropriate representation of the
uncertainty. We use the WPF scenario generation approach described in [78] to convert quantiles
or pdfs to forecast scenarios. In this approach, scenario sampling is performed using a Monte
Carlo approach. A covariance matrix is estimated based on historical forecasting errors, and this
is used to represent inter-temporal (hour-to-hour) correlations.
We can only use a limited number of scenarios as input to the stochastic UC model because of
the computational complexity of the problem. In most cases, there will be a need for scenario
reduction. In this chapter, we are testing different methods to generate a reduced set of scenarios.
One simple approach is to select a random subset of the original scenarios. In the case study
below, this approach is referred to as scenario reduction method 1 (SR1). Within the power
systems domain, it is common to use the scenario reduction method from [89], which we refer to
as SR2. Here, a reduced scenario set is derived from the Kantorovich distance between the
original and reduced set of scenarios, taking scenario probabilities and distances of scenario
values into account.

70

We are also exploring alternative approaches to generate a reduced set of representative

scenarios, as outlined in Chapter 5. The idea in Chapter 5 and in [97] is to find the most
attractive scenario that has a higher number of neighbors according to a flexible distance metric,
which is selected to be compatible with the problem to be solved. For the stochastic UC problem,
we use the maximum absolute deviation distance:
,

(6-6)

1, ,

where
and
are the two scenarios being compared, and T is their dimension. The two
scenarios are considered to be neighbors if the distance between them is lower than the
admissible tolerance, that is,
,
. When the more attractive scenario and all its
neighbors are found, they are cut from the initial set of scenarios, thereby forming a cluster.
The procedure is repeated until the attractive scenario cannot find a neighbor within the
tolerance. The remaining scenarios without a neighbor are considered outliers. Starting from a
small tolerance value, the total number of clusters may be higher than the desired number of
scenarios set for the stochastic problem. The algorithm is then repeated with a lower in order to
find the admissible tolerance that provides the desired number of clusters. Finally, each cluster is
represented by its more representative scenario. Here, the scenario presenting the lower sum of
distances, according to (6-6), with respect to all other members of the cluster has been used. The
probability associated with each representative scenario is given by the sum of the probabilities
of its cluster components. We refer to this method as SR3.
6.5 Case Study
The purpose of the case study is to test the different probabilistic WPF approaches, scenario
reduction methods, and operating strategies on DA UC decisions and the resulting RT ED.
Thermal units are committed based on a DA wind power forecast by using stochastic or
deterministic UC. The RT ED is then solved with the realized availability of wind power. We
focus solely on the impact of WPF uncertainty. Other uncertainties, like generator outages and
load forecast errors, are therefore omitted. We first discuss the quality of the probabilistic wind
power forecasts and resulting wind power scenarios. Then, we examine the impact on UC and
ED results in a simple case study with wind power and 10 thermal units.
6.5.1

Probabilistic Wind Power Forecasts

Time series of the DA deterministic point forecasts and realized wind power output for
15 hypothetical locations in Illinois in 2006 were aggregated and used as the starting point for
the probabilistic forecasts. The data were obtained from the Eastern Wind Integration and
Transmission Study (EWITS) [83].
We use both QR and NW KDF to generate probabilistic wind power forecasts. The wind power
point forecast is used as the only explanatory variable in both cases. The period from January to
July is used to train the two statistical models, whereas the resulting forecast data for October to
December are used in the analysis. An example of a quantile forecast is shown in Fig. 6-1.
In order to evaluate the outputs from the two different forecast uncertainty estimation
procedures, we focused on the calibration forecasts. Calibration is a measure of how well the
forecasted quantiles match the observed values [1]. Fig. 6-2 depicts the calibration diagram for
71

the QR and NW methods. There is clearly a quite significant negative deviation (corresponding
to an over-forecast) for both methods, with QR having the largest deviation. 19 We provide a
more detailed comparison of different probabilistic WPF methods in [93], [94], and [95]. The
results indicate that KDF forecasts outperform QR in terms of calibration, whereas QR tends to
perform better in terms of sharpness, which is a measure of the width of the forecast distribution.
1

WindPower[p.u]

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Time[hour]
Fig. 6-1 Probabilistic forecasts of wind power quantiles (5%, 10%, ... 95%) based on QR and
realized wind power generation (dotted line) for day 1.
0.0
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

Deviation[%]

2.0

4.0
QR
NW

6.0

8.0

10.0

NominalProportionRate[%]
Fig. 6-2 Calibration diagram for QR and NW methods, Oct.Dec. 2006.

Based on the probabilistic QR and NW forecasts, we generated 1,000 scenarios with the
sampling method described in [78]. Scenario reduction is then used to reduce the full set to either
10 or 100 scenarios by using the three scenario reduction methods described in Section 6.4. For
SR2, we use the implementation in GAMS/SCENRED [98] with standard parameters. The SR3
method is implemented in Matlab.
19

The large deviations may have to do with the short evaluation period.

72

Fig. 6-3 shows that the averages of the reduced scenarios are close to the average of the original
1,000 scenarios, although some deviations can be seen with random selection (SR1) and for the
clustering-based scenario reduction method (SR3). Results in Fig. 6-4 and Fig. 6-5 show that the
reduced scenarios based on SR2 and SR3 tend to have a significantly lower variance than the
original 1,000 scenarios. The variance of the randomly selected scenarios (SR1) is higher on
average than for SR2 and SR3, and with 100 scenarios, this method yields a closer match to the
variance of the original scenarios.
Table 6-1 shows a comparison of the means and variances for the original and reduced scenarios.
The statistics are calculated from the hourly forecasted scenario data over the full 3-month
period. All of the reduced sets are close to the original scenarios in terms of the overall average
over the period. However, SR2 has a lower mean absolute error (MAE) than the other two
methods. When it comes to variance, SR1 provides a closer fit than SR2 and SR3 do, both in
terms of average and MAE. It is evident that both SR2 and SR3 result in a significant reduction
in the scenario variance compared to the original scenario set. Still, the fit improves by
increasing the size of the reduced scenario set from 10 to 100.
Further analysis of statistical properties could be performed by analyzing additional moments of
the scenario distributions. A comparison of scenario reduction [89] and moment matching [99]
approaches for scenario tree construction is conducted in [100]. The authors found that the
moment-matching approach yields a better fit compared to the original scenario set in terms of
mean, variance, skewness, and kurtosis. However, the scenario reduction provides better
correspondence in terms of autocorrelation. Scenario reduction is also considerably faster
computationally than is moment matching. The authors also tested the two approaches as input to
a scheduling model for the electricity market within a region of Germany and found that scenario
trees derived from moment matching yields the lowest operational cost [100].
0.9

PowerForecast[p.u]

0.8
0.7
0.6
0.5

Realized

0.4

1000

0.3

10SR1

0.2

10SR2

0.1

10SR3

0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46

Time[hour]
Fig. 6-3 Averages of 10 scenarios with reduction methods SR1, SR2, and SR3 as compared to the
average of the original 1,000 scenarios, QR, days 1 and 2.

73

ScenarioVariance[p.u.]

0.035
0.03
0.025
0.02

1000

0.015

10SR1

0.01

10SR2
10SR3

0.005
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46

Time[hour]
Fig. 6-4 Variances for 10 scenarios with reduction methods SR1, SR2, and SR3 as compared to the
average of the original 1,000 scenarios, QR, days 1 and 2.

ScenarioVariance[p.u.]

0.025
0.02
0.015
1000
100SR1

0.01

100SR2

0.005

100SR3

0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46

Time[hour]
Fig. 6-5 Variances for 100 scenarios with reduction methods SR1, SR2, and SR3 as compared to
the average of the original 1,000 scenarios, QR, days 1 and 2.
Table 6-1 Summary statistics for scenarios (QR) with different reduction methods: average (AVG),
mean absolute error (MAE), Oct.Dec.
1,000
101010100100100SR1
SR2
SR3
SR1
SR2
SR3
Mean
(AVG)
0.419 0.417 0.418 0.417 0.418 0.418 0.417
Mean
(MAE)

0.026

0.010

0.024

0.008

0.004

0.021

Variance
(AVG)

0.011

0.011

0.006

0.005

0.011

0.008

0.007

Variance
(MAE)

0.0041 0.0055 0.0060 0.0012 0.0027 0.0042

74

6.5.2

Overview of Simulated Cases

We use the same assumption for thermal generators, loads, and realized wind power availability
as in [86]. The system consists of ten thermal generators with a mix of base, intermediate, and
peak units. The characteristics of the units are the same as in Chapter 4 (Table 4-1). A time series
with 91 days (OctoberDecember) of data for loads and wind power were simulated. The load
data are based on scaled historical load profiles for Illinois from 2006 and therefore are
synchronized with the wind power data. The total installed capacity is 1,662 MW of thermal
power and 500 MW of wind power, and the peak load is 1,500 MW. The total wind energy
amounts to 20% of the load within the 91-day simulation period. The costs of unserved reserve
and load are assumed to be $1,100/MWh and $3,500/MWh, respectively. These costs are
included in the objective function of the UC problem, as shown in (6-1).
An overview of the simulated cases is provided in Table 6-2 and Table 6-3. The cases differ in
terms of which UC strategy, operating reserve requirements, wind power forecast, and scenario
reduction method is employed. For the deterministic cases (Table 6-2), the first case uses a
perfect forecast and therefore serves as a benchmark for the analysis. The PF-QR and PF-NW
cases use deterministic forecasts (i.e., the 50% quantile from either the QR or NW forecasts),
without additional operating reserve for wind uncertainty, ORwind. In RF-QR and RF-NW, we
add a fixed ORwind equal to the average difference of the 50% and 5% WPF quantiles. Finally, in
the RD-QR and RD-NW cases, ORwind is dynamic, updated each day, and set equal to the
difference between the 50% and 5% quantiles. Because the 50% quantile is used as the point
forecast, the demands for energy and regular operating reserves should be met with a probability
of 0.95. The optimal choice of quantile range for ORwind generally depends on the cost of
scheduling additional reserves compared to the cost of not meeting the regular reserve
requirements. It should be noted that a more rigorous approach could be used to derive ORwind
(e.g., based on the ideas in [75]).
For the stochastic cases, we vary the probabilistic WPF approach, the scenario reduction method,
and the number of scenarios, as shown in Table 6-3. ORwind is set to zero for the stochastic cases,
with the assumption that the stochastic UC strategy internally addresses the wind power
uncertainty.
Table 6-2 Simulated cases with deterministic UC.
Additional Reserve:
Case
Forecast
, *
PERF
None
Perfect
PF-QR
None
QR: 50% quantile
PF-NW
None
NW: 50% quantile
RF-QR
Fixed: avg. 505% quantile
QR: 50% quantile
RF-NW
Fixed: avg. 505% quantile
NW: 50% quantile
RD-QR
Dynamic: 505% quantile
QR: 50% quantile
RD-NW
Dynamic: 505% quantile
NW: 50% quantile
* This reserve is applied at the UC stage only. All cases use a regular reserve,
, , equal to 10% of
hourly loads in both UC and ED.

75

Table 6-3 Simulated cases with stochastic UC.

Case
Forecast
Scenario Reduction*
No. of Scenarios
S10-QR1
QR
SR1
10
S10-QR2
QR
SR2
10
S10-NW1
NW
SR1
10
S10-NW2
NW
SR2
10
S100-QR1
QR
SR1
100
S100-QR2
QR
SR2
100
S100-QR3
QR
SR3
100
*SR1 = random, SR2 = traditional scenario reduction, and SR3 = scenario clustering.

6.6 UC and ED Results

The total operating costs for the different cases over the 91-day period are shown in Fig. 6-6 and
Fig. 6-7. Additional results are summarized in Tables 6-4 and 6-5.
48.00
Unservedload

46.00

Unservedreserve

44.00

Startup

Cost[M$]

42.00

Fuel

40.00
38.00
36.00
34.00
32.00
30.00
PERF

PFQR

PFNW

RFQR

RFNW

RDQR

RDNW

Case

Fig. 6-6 Overview of simulated total costs for deterministic cases.

48.00
Unservedload

46.00

Unservedreserve

44.00

Startup

Cost[M$]

42.00

Fuel

40.00
38.00
36.00
34.00
32.00
30.00

S10QR1 S10QR2 S10NW1 S10NW2 S100QR1 S100QR2 S100QR3

Case

Fig. 6-7 Overview of simulated total costs for stochastic cases.

76

Table 6-4 Summary of other results: deterministic cases.

Case
No. of
Load
Reserve
Avg.
Start-ups Curtailment Curtailment
Reserve
[hours]
[hours]
[MW]
PERF
225
0
7
155.2
PF-QR
228
14
268
153.2
PF-NW
231
12
273
152.2
RF-QR
280
1
38
174.5
RF-NW
287
1
40
174.6
RD-QR
267
0
31
170.8
RD-NW
266
0
22
173.8
Table 6-5 Summary of other results: stochastic cases.
Case
No. of
Load
Reserve
Avg.
Start-ups
Curtailment Curtailment
Reserve
[hours]
[hours]
[MW]
S10-QR1
278
1
71
169
S10-QR2
282
1
101
164
S10-NW1
267
0
49
171
S10-NW2
252
0
79
168
S100-QR1
234
0
35
175
S100-QR2
235
0
50
171
S100-QR3
234
0
68
170

The performance of the system clearly depends on the UC and operating reserve strategy and on
the WPF approach. We make the following main observations:

Using a deterministic point forecast without additional reserve is too risky and results in
high levels of unserved load and reserve.
For deterministic cases, a dynamic reserve requirement based on the probabilistic forecast
performs better than a fixed additional reserve.
A dynamic reserve based on a NW forecast performs better than a dynamic reserve based
on QR forecast, possibly caused by the better calibration of the NW forecast (Fig. 6-2).
With stochastic UC, scenarios based on NW forecasts perform better than scenarios from
QR.
Formal scenario reduction based on SR2 and SR3 does not improve the overall system
performance under a stochastic UC strategy compared to a random selection of scenarios
(SR1). This may have to do with the decrease in scenario variance under SR2 and SR3
(Table 6-1), which may result in less hedging and more risky scheduling.
Increasing the number of scenarios improves the performance of the stochastic UC
strategy. However, the computational burden increases. In these cases, the run-time with
100 instead of 10 scenarios is 15 to 20 times longer.
Some additional reserve is needed also in stochastic UC to cover wind power uncertainty
not captured by the probabilistic forecasts and the limited number of scenarios. The best
stochastic case (S100-QR1) yields a slightly higher total cost than that of the best
deterministic case (RD-NW).
The two best cases (S100-R1 and RD-NW) both make use of probabilistic WPF
information.

77

6.7 Current Work: IL Case Study

In our recent work [101], we are scaling up the UC modeling framework to handle larger
systems in a more realistic manner. In particular, we are testing different UC models and
operating reserve strategies on two-settlement electricity markets to explore the potential use of
probabilistic WPF to help power system operators scheduling and dispatch decisions. We are
using the power system in Illinois as a test bed to analyze the implications of a large-scale
expansion of wind power in a real-world electricity market.
The Illinois system consists of thermal generation and wind power, with only a negligible
amount of hydro generation. In the case study, loads and thermal generation corresponds to the
situation in Illinois in 2006, whereas we assume that additional wind power is added to the
system to represent a future scenario with a high penetration of wind power. The Illinois system
in 2006 consisted of 210 thermal plants with a total capacity of 41,380 MW. We assume that the
total installed capacity of wind power in the system is 14,000 MW. The wind power generation
fluctuates between 0 and 12,680 MW. Under this assumption, wind power provides 20% of the
in-state load and 16.6% of the total load including export in the JulyOctober period. The hourly
load profile is based on historical data from two large utilities in the state of Illinois for the
months of July to October of 2006.
We simulate the main steps in the operation of two-settlement electricity markets with focus on
the use of wind power forecasts (Fig. 6-8). The commitment schedule for thermal units is first
determined based on a DA wind power point forecast in the DA market, using deterministic UC.
Then, we simulate the reliability assessment commitment (RAC), where the commitment of faststarting units is adjusted based on a 4-hour-ahead probabilistic forecast, using stochastic or
deterministic UC. The RT ED is then solved with the realized availability of wind power. We
focus solely on the impact of WPF uncertainty. Other uncertainties, like generator outages and
load forecast errors, are therefore omitted, similar to the 10-unit case study presented above.
Note that the analysis of the 10-unit case above and in Chapter 4 does not consider the RAC, as
all commitment decisions are made at the DA stage. The simulation set-up for the Illinois study
is therefore better aligned with current operational practices, as discussed in Chapter 3.

Day-ahead
point forecast

UC

ED

Day-ahead
schedule and price

4hr-ahead
probabilistic forecast

Actual
wind power

RAC

ED

Real-time
dispatch and price

Fig. 6-8 Simulation of two settlement electricity markets in Illinois case study.

78

The results from the Illinois case study show that the re-commitment decisions at the RAC stage
are very important for the cost and reliability of the system. We find that additional operating
reserves are required to handle the uncertainty in wind power. When basing the re-commitment
decisions on a deterministic UC model, the use of dynamic operating reserves derived from the
probabilistic forecast gives better performance than fixed reserve requirements, as the operating
reserve levels are better aligned with the wind power forecast uncertainty. A stochastic UC
model with wind power scenarios leads to different re-commitment decisions and RT dispatch.
However, dynamic operating reserves and stochastic UC give similar results in terms of total
system cost. Hence, from a qualitative perspective, the results from the Illinois case study are in
line with the 10-unit results presented earlier in this chapter, although a more comprehensive
modeling framework is applied. The detailed results from the Illinois case also illustrate that
inaccurate wind power forecasts can lead to over- and under-commitment of generation capacity
and therefore have large implications for system reliability and RT prices. Further details on the
Illinois case study are provided in [101].
6.8 Conclusion
The results of the analysis in this chapter clearly illustrate the potential value of probabilistic
WPF in power system operations. Such forecasts can be incorporated into a deterministic UC
through probabilistic reserve requirements or can provide scenarios as input to a stochastic UC.
Some advantages of the deterministic approach with dynamic reserves includes that it is better
aligned with current operating procedures. Furthermore, from a computational perspective, it is a
simpler model with less computational burden. However, this approach does not capture the
effect of inter-temporal ramping events. It also does not consider uncertainty and its cost in the
objective function. In contrast, stochastic UC does address inter-temporal variability through the
scenario representation of uncertainty. Moreover, the total cost, including the expected cost of
scarcity, is explicitly taken into account in the objective function. However, the switch to
stochastic UC involves a more radical departure from current practice; furthermore, it may run
into computational constraints in large systems, and the benefits in terms of cost savings may be
limited.
More work is needed to improve the quality of probabilistic WPF, both in terms of pdf
estimation and generation of adequate scenarios. However, there will always be inaccuracy in
probabilistic forecasts, just as in point forecasts. This calls into question the overall risk
paradigm under which scheduling decisions are made. In stochastic UC, with the minimization
of expected cost, one could adjust the risk level by changing the costs of unserved energy and
reserves in the objective function. However, it may also be required to consider other approaches
based on different decision paradigms, such as utility theory, value at risk, robust optimization,
or regret. The optimal decision strategy does not only depend on the risk preferences of the
system operator, but also on the quality of the probabilistic forecast.

79

This page intentionally blank.

80

WIND POWER TRADING UNDER UNCERTAINTY IN ELECTRICITY MARKETS 20

This chapter presents a new model for optimal trading of wind power in DA electricity markets
under uncertainty in wind power and prices. The model considers settlement mechanisms in
markets with locational marginal prices (LMPs), where wind power is not necessarily penalized
from deviations between DA schedule and RT dispatch. We use quantile regression and kernel
density estimation to produce a probabilistic wind power forecast, whereas uncertainties in DA
and RT prices are assumed to follow a normal distribution. Utility theory and conditional value
at risk (CVAR) are used to represent the risk preferences of the wind power producers. The
model is tested on data from a hypothetical wind farm located in the state of Illinois and
combined with historical LMP data. The results show that the model can effectively control the
trade-off between risk and return for wind power producers operating in volatile electricity
markets.
The following nomenclature is used in this chapter:

,
,
,
,
,
,
,
,

,
,

,
,

CVARh
th
w
,

energy market profit, hour h (h=124), scenario m (m=1...M) [US$]

quantity bid into DA market, hour h [MW]
quantity bid into RT market, hour h [MW]
max. potential quantity delivered, hour h, scenario m [MW]
quantity delivered, hour h, scenario m [MW]
projected DA LMP, hour h, scenario m [US$/MWh]
projected RT LMP, hour h, scenario m [US$/MWh]
mean DA, RT prices, hour h, scenario m [US$/MWh]
stdev DA, RT prices, hour h, scenario m [US$/MWh]2
DA-RT price corr., hour h, scenario m [US$/MWh]2
Random sample, DA, RT prices, scenario m [Norm~(0,1)]
probability of scenario m
penalty for deviation between DA schedule and RT delivery
[US$/MWh]
utility, hour h, scenario m
risk parameter in utility function
conditional value at risk, hour h
threshold for CVAR
weight assigned to CVAR
quantile th of profit distribution, hour h

20

The original ideas behind the model described in this chapter were first presented in [24]. Our most recent results
from applying the model on a real-world wind farm located in the Midwest are presented in [102].

81

7.1 Introduction
There has been a rapid expansion of wind energy in recent years, with the global installed
capacity of wind power exceeding 200 GW [3]. Wind power is expected to continue its growth
in many countries, including the United States [44]. For wind farm owners, it is therefore
becoming increasingly important to manage the risk of participating in the electricity market. In
the United States, most of the wind power is sold under long-term power purchasing agreements
(PPAs). The PPAs lock in the price of delivered electricity and thereby stabilize the revenue
stream [103]. Nevertheless, a significant share of wind power is sold directly to the electricity
market. In 2009, 38% of the new installed wind power capacity was sold as merchant generation,
and most of this energy was traded in regions with open access electricity markets [103]. For
merchant wind generation, it is very important to apply scheduling and trading strategies that
reflect the uncertainty in wind power and prices and the corresponding trade-off between risk
and return for the wind farm owner. At the same time, purchasers of PPAs will have to schedule
the contracted wind power into the electricity market, and consider how wind power influences
the scheduling strategy of the rest of their portfolio.
In this chapter, we discuss scheduling and trading of wind power with focus on DA electricity
markets. We first provide an overview of existing literature on this topic. We then discuss the
trading of wind power in U.S. markets with LMPs, including settlement procedures and their
influence on trading decisions. We present an optimization model, which finds the optimal wind
power bid under uncertainty in forecasts for wind power, as well as DA and RT prices. Finally,
we apply the model in a case study of a hypothetical wind farm in the state of Illinois. We
analyze how DA bids depend on risk preferences and deviation penalty schemes, under the
assumption that wind power is sold as merchant generation.
7.2 Literature Review
Trading of wind power in electricity markets is still a relatively new problem. A limited number
of methods have been proposed in the literature to address the problem, as outlined below.
Bathurst et al. [104] use a simple wind power uncertainty representation based on Markov
probabilities and show how this approach can be used to reduce imbalance costs from short-term
wind power trading in the UK market. Bremnes [23] demonstrates how probabilistic wind power
forecasts can be created by means of local quantile regression. He shows that if imbalance costs
are deterministic, the profit maximizing bid is equal to the c+/(c+ + c-) wind power forecast
quantile, where c+ and c- are the imbalance costs of excess and shortage, respectively.
Matevosyan and Sder [19] use an ARIMA model for the wind speed and a power curve to
produce wind power forecast scenarios. The authors propose a stochastic optimization model for
DA trading that considers a more complex imbalance cost scheme akin to the rules in the
Scandinavian electricity market (Nord Pool). Pinson et al. [17] also focuses on minimizing the
imbalance cost. In a case study, the authors demonstrate the advantages of using probabilistic
wind power forecasts to derive DA trading strategies similar to the one proposed in [23]. Bourry
et al. [20] use probabilistic wind power forecasts based on kernel density estimation (KDE) to
estimate a probability distribution for the imbalance cost. A risk-based decision approach based
on conditional value at risk (CVAR) is used to derive the optimal bid under the resulting
uncertainty in imbalance cost. Usaola and Moreno [105] focus on bidding of wind energy in
intra-day markets. This is framed as a profit-maximization problem with simple representations
of uncertainties in wind power predictions and prices (intra-day and balancing). Morales et al.

82

[106] propose a stochastic programming model for wind power trading in multiple consecutive
markets. Uncertainty in wind power and prices are represented as seasonal ARIMA models.
However, the probabilistic models are general and are not linked to forecast information. The
authors derive optimal offer curves (bid-quantity) using CVAR as a measure for risk preferences.
Finally, Dent et al. [107] derive analytical wind power bidding strategies under the assumption of
a continuous probability function for wind power. The authors also consider situations where the
RT price is a function of wind generation and use CVAR to represent risk-averse trading
behavior.
The main features of several of the wind power models outlined above are summarized in Table
7-1, with a comparison to the model described in this chapter.
The literature summarized above focus on the conditions in European electricity markets. In
general, these markets have different prices for positive and negative imbalances in RT [105].
Although there are regional differences in market rules within Europe, wind power producers are
therefore generally penalized if their deviation is against the system balance. In contrast, our
analysis adds new insights by focusing on typical settlement procedures in U.S. electricity
markets with LMPs. In these markets, there is only one RT price. Hence, a wind power producer
may actually benefit from deviating from its advance schedule, depending on the difference in
DA and RT LMPs. We emphasize the use of probabilistic WPF methods to support trading
decisions, and we use different methods to produce probabilistic forecasts. We also consider
uncertainties in both DA and RT prices and their impacts on the risk faced by wind power
producers operating in LMP markets. Moreover, we consider different approaches to represent
risk preferences, either through the expected value (i.e., risk neutral), a CVAR measure
(i.e., trade-off between expected value and risk), or with a classical utility function. As far as we
know, utility theory has not been used in previous literature on wind power trading. Finally, our
model considers the impact of market design on the optimal trading decision. In particular, we
analyze how a deviation penalty would influence the optimal DA bidding decision.
7.3 Wind Power in LMP Markets
Most U.S. electricity markets are operated ISO/RTOs. In most cases, ISO/RTOs use LMPs and
the so-called two-settlement system for market clearing, as briefly outlined in Chapter 3. This
means that the DA schedule is settled at the DA LMP, and that any deviation between the DA
schedule and RT delivery is settled at the RT LMP [108]. In most cases, there are no intra-day
markets between DA and RT in the United States.
Wind power producers typically have the option to participate in the DA market. In general,
there may be some charges for deviations between the RT delivery and the DA schedule. So far,
wind power has typically been exempt from imbalance charges in most U.S. markets [26][63].
However, this situation may change as several ISO/RTOs are revising their current rules [109].
The Federal Energy Regulatory Commission (FERC) is also revising wholesale market rules for
better integration of variable generation [110]. The handling of wind power in general and
particularly the use WPF in U.S. markets were discussed in more detail in Chapter 3.

83

Table 7-1 Overview of published research/models for wind power trading.

Model

WPF

Price Forecast

Objective/
Risk

Uncertainty
Representation

Market Rules

Intraday
Trading

Multi-Day
Simulator

Bathurst
et al. 2002 [104]

Persistence
w/prob. table

No, fixed bal.

price relations

Min. (max.)
imbalance cost

Probability table
for wind power

UK (NETA),
regulation up/dn,
short term

Yes, but no DA
trading

Yes, analysis of
one year

Matevosyan
and Sder 2006
[19]

Wind speed
forecast w/ power
curve

Imbalance price
scenarios from
last days

Min. imbalance
cost

WP: ARMA from

errors

Nord Pool /
Denmark
(reg. up/dn)

No, not
considered

Yes, one month

Pinson
et al. 2007 [17]

Yes, persistence
and fuzzy neural
network

No, fixed
imbalance prices

Min imbalance
cost, analytic
solution

Intervals for
WPF, no price
uncertainty

Europe/
Netherlands
(regulation up/dn)

No, not
considered

Yes, one year

Bourry
et al. 2008 [20]

Yes, NWP based,

kernel density
estimation (KDE)

Simple deterministic forecast

for imb. prices

Min. imbalance
cost, CVAR

WP: KDE

Denmark
(regulation up/dn,
one-way penalty

No

Yes, one year

Usaola and
Moreno 2009
[105]

Yes, from the

Sipreolico model

Yes, simple price

prediction

Expected profit,
no risk

Discrete pdf for

WP, pdf for
prices

Spain (regulation
up/dn)

Focus on
intraday, not DA

Yes, three months

Morales
et al. 2010 [106]

No, general
probabilistic
model (AR(2))

No, general
probabilistic
model (ARIMA)

Profit (hourly) or
CVAR metric
(daily)

Scenario tree
w/reduction,
price correlation

Spain (regulation
up/dn)

Yes, one
adjustment
market

No, one day

analysis

Argonne model
in this chapter
[24] [102]

Yes, probabilistic
(quantile regression or KDE
w/MC sampling)

Yes, normal dist.

based on price
history for DA
and RT prices

Profit, CVAR, or
Utility (hourly)

WP: Discrete pdf,

Prices: Random
sampling w/DART correlation

United States
(LMP, two
settlement), DA
deviation penalty

No

Yes, up to four
months in current
analysis

84

A wind power producer is facing three main uncertainties when bidding into the DA market: the
DA LMP, the RT LMP, and the RT delivery of wind power. We assume that a potential
deviation penalty depends on the absolute deviation between the RT delivery and the DA
schedule. 21 Then, if we define a set of joint samples, m = 1...M, of LMPs and wind power
delivery, a simple formulation of the wind power producers profit for a given hour, h, and
sample, m, is:

(7-1)

Note that a wind power producer has an incentive to curtail its generation whenever the RT price
drops below 0, in order to avoid selling its generation at a negative price. 22 This is the case
regardless of the position in the DA market, since the producer can buy back energy in the RT
market at a negative price for the energy scheduled DA. Experience shows that negative RT
prices may occur quite frequently in markets with LMPs and a high penetration of wind power.23
The model allows for three different assumptions of how the wind power is controlled when RT
prices 24 are low and negative:
1) The wind farm generates at maximum possible output regardless of the RT price.
2) The wind farm shuts down generation whenever the RT price is negative, as shown in
(7-2).
3) The wind farm maximizes profits during low and negative prices, also considering a
potential deviation penalty, as shown in (7-3).

,
0,
,

,
,

min

,
,

0,
,

(7-2)

(7-3)

,
,

21

The model allows for an alternative penalty representation, where the RT price is adjusted according to a penalty
factor and there is a deadband where no penalty applies. However, this representation has not been thoroughly tested
and is not used in the case study presented in this chapter.
22
In many cases, wind power producers receive additional benefits, such as production tax credits or renewable
energy credits. In these cases, they have an incentive to produce even with negative prices, as long as the net profit
remains positive. We do not consider the impact of such incentives here.
23
For example, hourly price data for the Minnesota hub in MISO for 2009 show that negative LMPs occurred
443 times in the RT market, but only 29 times in the DA market. We ignore the possibility of negative DA prices
here. The negative prices in the RT market may to some extent be linked to the wind power benefits discussed in the
footnote above.
24
New wind power plants can easily be controlled to reduce or shut down their generation, and this flexibility will
add to their revenue stream. Older wind turbines may not have this flexibility.

85

The wind power producer needs to decide on how much to bid into the DA market,
, . We
assume that the DA offer is bid in at zero price. Hence, the bid will always be accepted, since we
assume that the DA price is positive. We can re-write the wind power sample profit in (7-1) to
the following:

(7-4)

In this formulation, the first part ( , , ) is equal to the profit if all of the actual wind power
generation is settled at the DA price. This corresponds to offering a perfect forecast into the DA
market. The latter part of (7-4) is the consequence of deviating from a perfect forecast. Here, the
deviation from the forecast is settled at the difference between the RT and DA LMPs, minus the
potential deviation penalty. Note that the first part of (7-4) is entirely stochastic. The decision
variable,
, , occurs in only the latter part of (7-4). If LMPs and wind power delivery are
known, we can see from (7-4) that:
1) It is beneficial to deviate from a perfect forecast if the difference between the DA and RT
LMPs is larger than the deviation penalty. 25
2) In the absence of a deviation penalty, the profit maximizing bid is directly dictated by the
difference in DA and RT prices. If the DA price is higher (lower) than the RT price, the
producer benefits from bidding all (none) of its capacity in the DA market. Hence, the
optimal bid is independent of the wind power delivery and, therefore, also independent of
the wind power forecast.
The net deviation settlement as a function of the RT deviation is illustrated in Fig. 7-1. The
figure shows the deviation settlement for different assumptions about the difference between DA
and RT LMPs with and without a deviation penalty. When there is no deviation penalty (left), it
is evident that the wind power producer would want to maximize the deviation and bid into the
DA market according to the price difference between the two markets. A deviation penalty
(right) gives an incentive to minimize the deviation as long as the expected price difference is
smaller than the penalty. Note that the deviation settlement is not symmetric when there is an
LMP difference.
Of course, the LMPs and the wind power delivery are unknown at the time of bidding into the
DA market. A wind power producer therefore has to consider the combined risk from these
uncertainties when deciding on a bidding strategy. All of the three stochastic variables are taken
into account in the bidding model presented in Section 7.5.

25

Note that this approach is different from most market designs in Europe, with different balancing prices for realtime deviations up and down, so the wind power producer cannot benefit by deviating from the advance schedule.

86

300

Dev.settlement[$/MWh]

Dev.settlement[$/MWh]

300
200
100
0
60

40

20

20

40

60

100
200

200
100
0
60

40

20

300

Dev.settlement[$/MWh]

Dev.settlement[$/MWh]

0
0

20

40

60

100
200

0
60

40

20

20

100
200
300

Deviation[MWh]
300

Dev.settlement[$/MWh]

300

Dev.settlement[$/MWh]

60

100

300

200
100
0
20

40

200

Deviation[MWh]

40

60

300

100

60

40

300

200

20

60

Deviation[MWh]

300

40

40

200

Deviation[MWh]

60

20

100

20

40

60

100
200
300

200
100
0
60

40

20

20

100
200
300

Deviation[MWh]

Deviation[MWh]

Fig. 7-1 Examples of hourly financial settlements as function of deviation between RT delivery and
DA schedule with penalties: penDA = 0 (left), and penDA = 5 $/MWh (right), and LMP differences:
, , = 3, -3, 0 $/MWh (upper, middle, lower).

7.4 Probabilistic Wind Power Forecasting

The uncertainty in the wind power forecast is obviously of importance for a wind power
producer bidding into the electricity market, since the realized wind generation and the deviation
from the DA bid determine the profits as discussed above. There are several different approaches
for representing uncertainty in the projected wind power, as explained in [1], including
probabilistic representations, risk indices, and scenarios. The wind power trading model
presented below can take any discrete probabilistic representation of the wind power forecast. In
our work, we use statistical approaches to estimate forecast uncertainty based on historical
forecasting errors. We have applied both quantile regression (QR) and KDE to model forecast
uncertainty, as briefly outlined below.

87

In the case study included in this chapter, we use the QR approach from [77] as the main
approach to produce probabilistic forecasts based on the wind power point forecast errors. The
model employs a linear quantile regression with the base functions formulated as cubic B-splines
in order to obtain the quantile with a proportion of the forecast errors. Each quantile is modeled
as a sum of the nonlinear smooth functions of the forecasted wind power generation. Spline
bases are used to approximate each of the smooth functions as a linear combination of base
functions. The probabilistic forecast is represented through a set of quantiles for each hour of the
day ranging from 0.05 to 0.95 with a 0.05 increment.
The method described in [78] is used to generate a number of wind power scenarios that provide
information on the dependency of the prediction errors through the set of look-ahead times. The
method is based on the conversion of the set of random variables composing probabilistic
forecast series, as obtained with the QR method described above, into a multivariate Gaussian
random variable. The temporal interdependence structure is represented by the covariance
matrix, which is recursively estimated because of the nonstationary characteristics. Monte Carlo
simulation is used for the generation of equiprobable scenarios. Fig. 7-2 and Fig. 7-3 illustrate
the resulting forecast quantiles (which are equivalent to forecast intervals) and scenarios of wind
power generation, respectively.

Fig. 7-2 Probabilistic representation of wind power forecast as intervals or quantiles.

Fig. 7-3 Forecast scenarios of wind power generation.

88

In this project, we have developed novel time-adaptive KDE algorithms for probabilistic WPF,
based on the Nadaraya-Watson (NW) [93] and quantile copula (QC) [94] [95] estimators. The
KDE algorithms can also be used to estimate the forecast uncertainty as input to the wind power
trading model. In this case, we use conditional kernel density estimation to produce a
nonparametric probabilistic wind power forecast, from which we derive hourly samples for
available wind power generation for each hour of the next day, , . In short, conditional density
estimation consists of estimating the density of a random variable Y, knowing that the
explanatory random variable X is equal to x. For the wind power problem, it consists of
forecasting the wind power probability density function (pdf) at time step t for each look-ahead
time step t+k of a given time-horizon when a set of explanatory variables (e.g., numerical
weather prediction variables and current wind power) is known. When translated to an equation,
we have:
|

(7-5)

where pt+k is the wind power forecasted for look-ahead time t+k, xt+k|t is the explanatory variable
forecasted for look-ahead time step t+k and available at time step t, fP,X(pt+k,yt+k|t) is the
multivariate density function, and fX(x) is the marginal density of X. The output is the forecasted
pdf for wind power generation, which is then converted into a probability mass function (pmf)
with a set of M equally probable points for each hour. Scenario generation is not required, as
each hour is treated independently in the wind power trading problem, as further discussed
below. More details about the KDE forecasting methods can be found in [93][95]. In the case
study, we do a brief comparison of applying QR and the two KDE methods to the wind power
trading problem.
7.5 A Model for Wind Power Trading under Uncertainty
Our wind power trading model optimizes the quantity to bid into the DA market for a wind
power producer. We use a discrete representation of uncertainty in the hourly wind power
forecast, as outlined in the previous section. DA and RT LMPs are also represented as discrete
probability distributions, as explained below. We use the wind power and LMP information to
calculate discrete probability distributions for the wind power producers profit as a function of
the DA bid. This allows us to derive optimal DA bids under different assumptions for the
decision makers risk preferences.
The initial ideas for the wind power trading model were first presented in [24]. Since then, we
have extended the model and analysis in multiple directions. We use different statistical
approaches for probabilistic WPF, and we have developed a more complete description of the
decision problem. We have conducted a more detailed case study based on hypothetical wind
farm data, as documented in the next section. We have also applied the model to a case study of
a real-world wind farm [102].

89

7.5.1

Representation of Uncertainty in DA and RT LMPs

We use a straightforward model to project DA and RT LMPs for the next day. The price
forecasts are based on historical LMPs. 26 A window of four weeks from the most recent history
is used to estimate the mean, variance, and correlation for each individual hour of the next day.
Weekdays and weekend days are treated independently. Hourly samples of DA and RT LMPs
are then drawn from (7-6) and (7-7) with the estimated parameters. Note that the same two sets
of random samples ( ,
) are used in all hours:
,

(7-6)

(7-7)

This discrete representation of the uncertainty in LMPs assumes that prices follow a normal
distribution. However, any probability distributions could be used, as long as the distributions are
converted to discrete samples. A set of M joint random samples for DA and RT LMPs are
generated and paired with the wind power pmf to form a total set of M samples for all three
random variables. Note that the formulation in (7-6) and (7-7) includes the correlation between
DA and RT LMPs. In areas with large shares of wind power, there could also be a significant
negative correlation between LMPs and wind power. The wind-LMP correlation could also be
captured. For instance, a more elaborate price forecasting system that uses the wind power
forecast as input can be found in [111]. A more advanced price forecasting system could easily
be incorporated into the model, as long as the price forecasts have a discrete representation.
7.5.2

Mathematical Formulation of the DA Bidding Problem

The objective for the wind power producer is to find the optimal DA bid,
, , depending on the
decision criterion adopted. We consider three different objective functions for the problem, as
shown in (7-8)(7-10): (1) expected profit, (2) CVAR [112], and (3) expected utility [113]. The
decision makers risk preferences are reflected in the choice of objective function and
corresponding parameters. The expected profit criterion in (7-8) is risk neutral. The CVAR
criterion in (7-9) is risk averse, since the lower part of the profit distribution (i.e., CVAR) carries
additional weight in the objective function. For the utility criterion, we assume a standard
exponential utility function [113], as shown in (7-10). The utility function can range from risk
prone to risk averse, depending on the parameter.

(7-8)

26

(7-9)

We also experimented with more advanced ARIMA models for LMP projections, but since they did not give
significant improvements in forecast accuracy, we chose to use the simpler approach for price projections.

90

where
,

(7-10)

where
1
1

, is given by (7-1). Constraints (7-2) or

In all three objective functions, the sample profit,
(7-3) apply depending on the assumption of wind power control during negative prices, as
discussed above. In addition, we impose that the DA bid must be below the normalized capacity
of the wind farm (i.e., 0
1). In general, the resulting optimization problem is a
,
nonlinear programming problem. We use the generalized reduced gradient algorithm built into
Excels solver to find the solution. 27 The model currently has a simple user interface in Excel
(Fig. 7-4), and is a stand-alone Excel application. Multi-hour simulations can be conducted to
analyze the performance of the different bidding strategies over time, as illustrated in the case
study in the next section.

Fig. 7-4 Excerpts from Excel interface.

27

A common problem with nonlinear optimization models is the possibility of convergence to a local optimum. This
problem applies to the reduced gradient algorithm as well, and it cannot guarantee that the solution is a global
optimum. However, the optimization problem only has one decision variable, which is constrained between 0 and 1.
Possible local optima can therefore easily be identified and ruled out (experience shows that the problem occurs
most frequently with a risk-prone utility function). An initial starting point for the search is also provided to guide
the algorithm in the right direction.

91

7.6 Case Study: Hypothetical Wind Farm in Illinois

In this section, we apply the model outlined above in a case study of a hypothetical wind farm in
the state of Illinois. We analyze how DA bids depend on risk preferences and deviation penalty
schemes, assuming that wind power is sold as merchant generation.
7.6.1

Assumptions

WindGeneration[Fraction ofCapacity]

We analyze the bidding of a hypothetical wind farm in the state of Illinois, using wind power
data from one location (site 4848) in the Eastern Wind Integration and Transmission Study
(EWITS) [83]. The EWITS database provides both point forecast and realized wind power
generation. The underlying data methodology is explained in [84]. Data from January to
August 2006 are used to train the quantile regression method [77] in order to generate
probabilistic forecasts. A total of 1,000 scenarios of wind power generation are generated for
each hour of the day for October, November, and December using the Monte-Carlo sampling
scheme from [78]. The deterministic point forecast and 10 forecast scenarios are shown for one
selected day in Fig. 7-5. The figure illustrates that forecast scenarios span a relatively wide range
around the point forecast.
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Hour

Fig. 7-5 Wind power forecast: Deterministic point forecast (solid thick line) and 10 forecast
scenarios for selected day (Oct. 9, 2006).

We use market price data from the Illinois trading hub in the MISO to estimate the price
parameters for each individual hour with the methodology outlined in Section 7.5.1. Fig. 7-6
shows the resulting price parameters for October 9. Monte-Carlo simulation generates
1,000 scenarios for DA and RT prices for each hour. The price scenarios are combined with the
wind power scenarios described above. The resulting set of scenarios is used to evaluate DA
bidding, market profits, and risk measures under different decision criteria. Illustrations of price
scenarios are shown in Fig. 7-7, with RT prices clearly having a wider range of uncertainty than
DA prices. Note that the uniformity between the hours occurs because the same random sample
is used for all hours of each day. Capturing the inter-temporal correlation is not important in this
problem because each hour is optimized individually.

92

Fig. 7-6 Estimated DA and RT price parameters for selected day (Oct. 9, 2006).

DayAheadPrice[$/MWh]

140
120
100
80
60
40
20
0
20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

40
60

Hour

140

RealTimePrice[$/MWh]

120
100
80
60
40
20
0
20

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

40
60

Hour

Fig. 7-7 Thirty scenarios for DA (upper) and RT (lower) prices for selected day (Oct. 9, 2006).

7.6.2

Results

We first present results for one individual hour, looking at optimal strategies and risk-return
trade-offs for different decision criteria. We then calculate the optimal bidding decisions for all
the hours of the day under different assumptions for decision criteria and deviation penalty.
Finally, we present results where the effectiveness of the different strategies are evaluated over a
one-month period against actual historical price data. The default assumption in this case study is
that the wind power producer curtails the output during negative prices, according to (7-2). For
93

the CVAR decision criterion in (7-9), we assume a CVAR threshold, th, of 5% and a weight, w,
of 0.1. For the utility decision criterion in (7-10), we assume that the risk parameter, , is either
3 (risk averse) or +3 (risk prone). The min-max profit range in the utility function is set to
[-100, 120]. These values are chosen for illustrative purposes. In reality, the choice of decision
criteria and corresponding parameters should reflect the actual risk preferences of the wind
power producer.
7.6.2.1 Analysis of Bidding in Selected Hour

We now investigate the bidding problem for one selected hour (i.e., hour 5 on October 9 (Fig.
7-5 through Fig. 7-7) in more detail. Table 7-2 shows that the expected DA price is slightly
higher than the expected RT price for this hour. However, the RT price is much more volatile,
resulting in a much larger spread in RT price scenarios as compared to DA (Fig. 7-7). There is
also a significant correlation between DA and RT prices. The wind power forecast is relatively
low, with only 17.4% of installed capacity expected to be available. However, the 1,000 wind
power scenarios span a wide range from 0 to 72.8% of installed capacity.
Table 7-2 Price parameters and wind power point forecast (pf)
for hour 5, Oct. 9, 2006.
,

25.49

23.96

4.48

25.91

0.60

pf5
0.174

0.728 0.002

The objective functions for different decision criteria are shown in Fig. 7-8 and Fig. 7-9. For the
expected value criterion (E), it is evident that that the optimal DA bid is to bid all capacity in the
DA market. This result is not surprising given that
,
, . In this case, it will always be
optimal to bid maximum capacity in the DA market under the profit criterion, regardless of the
wind power forecast, as long as there is no deviation penalty. The CVAR criterion (C) yields an
optimal bid just slightly above zero. This is because the CVAR decreases as a function of bid
quantity (Fig. 7-10). The optimal bid under the risk-averse utility criterion is 0.39, whereas it is
the same as under expected profit with a risk-prone utility function. The results clearly illustrate
that the optimal bidding decisions depend on the risk preferences of the decision maker.
The level of risk for different bidding quantities is further explored in Fig. 7-10. The figure
shows that the variance in profit is lowest around the point forecast level of wind generation. The
profit variance increases quickly as a function of higher amounts of bidding in the DA market.
Although the variance also increases for bids below the point forecast, the increase is relatively
modest. In this case, the producer is likely to sell some of its generation in the RT market. Still,
the producer can hedge against selling at negative prices in RT by not producing and thereby
controlling some of the risk. In contrast, the producer cannot hedge against buying at high RT
prices, which is more likely to occur the more power is bid into the DA market. This explains the
high variance for high bidding quantities. It also explains why the one-sided CVAR measure
declines as a function of
, . Fig. 7-10 also indicates the impact on variance and CVAR from
curtailing wind at negative prices. Curtailment clearly reduces CVAR considerably for low
bidding quantities, because selling at negative prices in the RT market is avoided, reducing the
downside risk. Note that curtailing the wind power when prices are negative also tends to extend
the upside of the profit distribution, since the producer benefits from buying at negative prices
instead of producing, regardless of the bid quantity.

94

Fig. 7-8 Expected profit (E) and CVAR criteria (C) as function of DA bid quantity, no deviation
penalty (hour 5, Oct. 9, 2006).

Fig. 7-9 Expected utility for the risk-averse ( = -3) and risk-prone ( = 3) decision maker, no
deviation penalty.
5
0

250

5
200
150
100

10
Var
Var*
CVAR
CVAR*

15
20

CVAR[$/MW]

ProfitVariance [($/MW)2 ]

300

25

50

30

0
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1

35

DayAheadBidQuantity

Fig. 7-10 Variance and conditional value at risk, no deviation penalty (hour 5, Oct. 9, 2006).
*Without wind power curtailment during negative prices.

95

The trade-off between risk and return is further illustrated in Fig. 7-11, which shows the
expected profit vs. CVAR for the range of possible DA bids. In this hour, it is clear that all of the
bids lie on an efficient frontier. For a decision maker considering these two criteria, the optimal
strategy therefore depends on how much weight to assign to each of them. Profit maximizing
(i.e., risk neutral) and risk-prone decision makers opt for maximum profit. The optimal strategy
according to the conditional value at risk objective is to maximize CVAR, whereas decisions
based on point forecast and the risk-averse utility function lie somewhere in between.
Fig. 7-12 shows the same results with a $5/MWh deviation penalty. Note that the expected
profits are now clearly reduced due to the penalty. Furthermore, only a limited portion of the
potential bids is now on the efficient frontier. For instance, the optimal bid according to the riskprone utility function is not on the efficient frontier. This is not surprising, as the utility criterion
does not consider CVAR in the objective function. However, the optimal bids under all of the
other criteria and the point forecast are on the efficient frontier.

Fig. 7-11 Illustration of the trade-off between expected profit and conditional value at risk, no
deviation penalty (hour 5, Oct. 9, 2006).

Fig. 7-12 Illustration of the trade-off between expected profit and conditional value at risk, $5/MWh
deviation penalty (hour 5, Oct. 9, 2006).

96

The optimal bidding results for the individual hour are summarized in Table 7-3. Table 7-4
summarizes the results with a $5/MWh deviation penalty. The penalty drives the optimal bids
closer to the point forecast. At the same time, the expected profit (E) and the CVAR (C) criteria
are reduced due to the penalty.
Table 7-3 Summary of results at optimum for different decision criteria (hour 5, Oct. 9, 2006),
no deviation penalty.

C
Ua
VAR
CVAR

Criteria
qDA*

1.00

6.52

2.93

0.796

-26.8

-36.0

419.3

0.02

4.79

4.78

0.799

0.13

-0.14

43.3

U(=-3)

0.39

5.45

4.25

0.800

-7.74

-11.9

61.7

U(=3)

1.00

6.52

2.93

0.796

-26.8

-36.0

419.3

=-3

Table 7-4 Summary of results at optimum for different decision criteria (hour 5, Oct. 9, 2006),
$5/MWh deviation penalty.

C
Ua
VAR
CVAR

Criteria
qDA*

0.21

4.44

3.79

0.798

-4.28

-6.47

32.4

0.09

4.25

4.03

0.798

-1.07

-2.17

30. 2

U(=-3)

0.19

4.43

3.85

0.798

31.0

-3.64

-5.75

U(=3)

1.00

2.45

-1.61

0.781

-31.5

-40.6

427.5

=-3

7.6.2.2 Optimal Bidding Strategies for One Day

We now look at the optimal bidding strategy for the entire day (i.e., 24 hours). Fig. 7-13 shows
that under the expected profit criterion, the optimal decision is to bid full capacity in the DA
market whenever the expected DA price is above the expected RT price. No capacity is bid into
the DA market when the expected RT price is higher than the expected DA price under this
criterion. The risk-prone utility criterion gives almost exactly the same results (not included in
Fig. 7-13). With the CVAR and risk-averse utility criteria, the optimal bid is in between the
maximum and minimum capacity in hours with a small difference in expected DA and RT
prices. The bid quantity tends to be higher with the utility criterion than it is with the CVAR
criterion. Overall, the decision strategies are mainly driven by the price expectations, as long as
there is no deviation penalty.
We repeat the analysis with a deviation penalty of US$5/MWh and US$10/MWh (Fig. 7-14 and
Fig. 7-15). The results show that a deviation penalty would bring the bids closer to the projected
generation (i.e., the point forecast) under the expected profit, CVAR, and risk-averse utility
criteria. We can see a similar effect for the risk prone utility criterion (Fig. 7-16); however, the
impact of the penalty is less prominent because of the willingness to take more risk.
Finally, the trade-offs between expected profit and CVAR for the full day are illustrated in Fig.
7-17 and Fig. 7-18 for the different bidding strategies. The results show that the expected profit
criterion (E) yields the highest expected profit. The risk-averse CVAR and utility criteria lead to
lower profit but higher CVAR. Finally, it is interesting to note that the point forecast, pf, is not
on the efficient frontier for the full day, as the CVAR criteria score higher in both expected profit
97

and CVAR. This result is the case both with and without a deviation penalty. Note that profit and
CVAR are metrics that can be scaled up and calculated over any time resolution. In contrast, the
utility function requires a definition of a minimum-maximum range and therefore cannot be
scaled up in the same manner.

Fig. 7-13 Optimal DA bidding under different decision criteria for 24 hours (Oct. 9, 2006). No
deviation penalty. pf is the point forecast.

DayAheadBidQuantity

1.2
1
0.8
pf

0.6

E*
0.4

C*
U*(=3)

0.2
0
0

10 12 14 16 18 20 22
Hour

Fig. 7-14 Optimal DA bidding under different decision criteria for 24 hours (Oct. 9, 2006).
US$5/MWh penalty. pf is the point forecast.

98

DayAheadBidQuantity

1.2
1
0.8
pf

0.6

E*
0.4

C*
U*(=3)

0.2
0
0

10 12 14 16 18 20 22
Hour

Fig. 7-15 Optimal DA bidding under different decision criteria for 24 hours (Oct. 9, 2006).
US$10/MWh penalty. pf is the point forecast.

DayAheadBidQuantity

1.2
1
0.8
pf

0.6

Pen:0
0.4

Pen:5
Pen:10

0.2
0
0

10 12 14 16 18 20 22
Hour

Fig. 7-16 Optimal DA bidding with risk-prone decision criteria ( = 3) for different deviation
penalties (0, 5, 10 $/MWh). (Oct. 9, 2006). pf is the point forecast.

Fig. 7-17 Illustration of the trade-off between expected daily profit and conditional value at risk, no
deviation penalty (Oct. 9, 2006).

99

Fig. 7-18 Illustration of the trade-off between expected daily profit and conditional value at risk,
$5/MWh deviation penalty (Oct. 9, 2006).

7.6.2.3 Monthly Simulation with Realized Price and Wind Power Outcomes

In order to assess decisions based on stochastic optimization, it is necessary to analyze a

strategys performance over a range of outcomes. One could do this by sampling from the
probability distributions used in the optimization problem. However, a better test for
performance in the real world is to use historical realizations of the stochastic variables to
calculate realized profit over an extended time period. We have therefore developed a simple
simulator in which decision criteria can be tested based on real-world data. The optimal DA bid
for each hour is derived based on probabilistic forecasts for wind power and prices, as outlined in
Section 7.5. In turn, the realized profit for each hour is calculated with the historical wind power
and price data.
In this case study, we have used wind power data (forecast and realized generation) for
October 2006 from site 4848 in the EWITS study. We combined the wind power data with DA
and RT price data for the Illinois hub at the MISO for October 2009. 28 Summary statistics for
wind power and price data are provided in Table 7-3. Note that RT prices have a much higher
volatility and spread as compared to the DA prices. Table 7-5 summarizes the results from
testing the wind power trading model with different decision criteria over the one-month period.
Table 7-5 Summary statistics for wind power and DA/RT prices, October.

Average
Standard Deviation
Minimum
Maximum

Wind Power
[% of capacity]
0.39
0.31
0.00
0.96

DA Price
[$/MWh]
25.12
8.99
10.40
53.89

28

RT Price
[$/MWh]
24.65
15.39
-39.10
223.34

Note that the wind and price data are from two different years in this analysis. We do not consider the correlation
between wind power and price in the probabilistic forecasts. At the same time, the wind power data is for a
hypothetical site, so there is no relationship between the price and wind power data. Hence, it is not important to
have synchronized time series of prices and wind power in this case study. In current work, we are testing the model
on data from a real wind farm, and in this case we do use synchronized wind and price data [102].

100

We first analyze the results with no deviation penalty. The trade-off between total profit over the
month and the CVAR calculated based on the distribution of hourly realized results is shown in
Fig. 7-19. The expected profit (E*) and risk-seeking utility (U*(=0.3)) strategies give the
highest profit but also the lowest CVAR. The CVAR strategy (C*), in particular with the weight
of 0.3, gives a much higher CVAR, as expected. Note that bidding according to the point forecast
(pf) gives much lower profit but has a low risk level. This outcome is also the case for the
strategy of constantly bidding zero capacity (zero) into the DA market, thereby selling all of the
energy into the RT market.
The trade-off between the wind farm profit on the one hand and the deviation from DA schedule
on the other hand is shown in Fig. 7-20. Under most circumstances, the system operator will
benefit from keeping the deviations from the DA schedule as small as possible. 29 Among all of
the strategies considered, point forecast bidding gives the lowest absolute deviation from the DA
schedule. However, this strategy also yields a low total profit. Hence, in this case there are
clearly conflicting objectives between the wind farm owners and the system operator. This result
is in line with our analysis in Chapter 2, where we discuss the value of WPF for different market
participants and how training criteria for wind power point forecasts can be tuned according to
the objectives of the forecast users.
The results for all of the strategies under the assumption of no deviation penalty are summarized
in Table 7-8. One interesting observation is that the strategies that consider the price forecasts all
bid more into the DA market than the wind power point forecast, as shown by the average DA
bid. This is because the bidding strategy is driven by price expectations. The average DA price is
higher than the average RT price in the simulation period (Table 7-5), and this is reflected in the
price projections. Hence, in the absence of a deviation penalty, the wind power producers bid
above the point forecast to take advantage of expected higher DA prices. The price-based
strategies result in a higher profit, but also increase the wind power producers risk exposure
measured in terms of the hourly variance and CVAR (Table 7-8).
1.0
0.0

CVaR[$/MWhr]

1.07000

7200

7400

7600

2.0

7800

pf
zero

3.0

E*

4.0

C*(w=0.1)

5.0

C*(w=0.3)

6.0

U*(=3)

7.0

U*(=3)

8.0
9.0
TotalProfit[$/MWmonth]

Fig. 7-19 Total October profit vs. hourly conditional value at risk, no deviation penalty.

29

This is not always the case, however, as the system operator could actually benefit if the wind farm deviation goes
against the system deviation at the respective location.

101

AvgAbsDeviation [MW]

0.60
0.50
pf

0.40

zero
0.30

E*

0.20

C*(w=0.1)
C*(w=0.3)

0.10

U*(=3)
U*(=3)

0.00
7000.0

7200.0

7400.0

7600.0

7800.0

TotalProfit[$/MWmonth]

Fig. 7-20 Total October profit vs. average absolute deviation from schedule, no deviation penalty.
Table 7-6 Summary of results for October with different decision criteria, no deviation penalty.
Criteria
pf
zero
E*
C*(w=0.1)
C*(w=0.3)
U*(=-3)
U*(=3)

Total Profit
[$]
7,145.2
7,035.3
7,562.3
7,504.7
7,469.7
7,529.3
7,582.5

CVAR
[$/hr]
-2.17
0.03
-8.57
-7.53
-3.30
-8.11
-8.57

Variance
[$2/hr]
78.7
78.0
105.9
104.9
93.7
104.9
107.3

Avg. Bid
0.38
0.00
0.71
0.60
0.40
0.71
0.70

Avg.
Deviation
0.17
0.39
0.54
0.47
0.35
0.51
0.54

The results outlined above indicate that the market operator may want to introduce incentives to
avoid large deviations between DA bid and RT delivery. We now repeat the analysis with a
$5/MWh deviation penalty. The results are shown in Fig. 7-21 and Fig. 7-22. In this case, it is
actually the point forecast that performs the best in terms of total profit. Among the strategies
considering the price predictions, it is the risk-averse strategies that perform better (and in
particular, the CVAR ones), both in terms of total profit and CVAR. The poorer performance of
the profit-based and risk-prone strategies are probably caused by inaccuracies of the price
forecasts. When the price predictions indicate a large different between DA and RT prices, the
optimal strategy will be to deviate from the wind power forecast. However, if the price
difference does not materialize, the wind farm will be penalized by the deviation penalty. In this
case, it therefore appears to be better to rely on the wind power forecast without considering the
price predictions. 30 Furthermore, the interests of the wind farm owner and the system operator
are well aligned in this case (Fig. 7-22 ). The strategy that does not bid into the DA market (zero)
still gives a low risk, but the total profit is much smaller than for the other strategies. The results
with a $5/MWh deviation penalty are summarized in Table 7-8. Note that with a deviation
penalty there is less difference between the strategies in terms of the average DA bid.
30

Of course, the price predictions used in this study are somewhat rudimentary, simply assuming that prices follow
a normal distribution based on historical prices. However, the price predictions could easily be replaced with more
advanced price forecasting models. The only requirement is that the price distribution needs to be represented as a
discrete set of price realizations for each individual hour.

102

1.0
0.0

CVaR[$/MWhr]

1.05400

5600

5800

6000

6200

6400

6600

2.0

pf

3.0

zero

4.0

E*

5.0

C*(w=0.1)

6.0

C*(w=0.3)

7.0

U*(=3)

8.0

U*(=3)

9.0
10.0
TotalProfit[$/MWmonth]

Fig. 7-21 Total monthly profit vs. hourly conditional value at risk, $5/MWh deviation penalty.
0.45

AvgAbsDeviation [MW]

0.40
0.35
pf

0.30
0.25

zero

0.20

E*

0.15

C*(w=0.1)

0.10

C*(w=0.3)

0.05

U*(=3)

0.00

U*(=3)

5400.0 5600.0 5800.0 6000.0 6200.0 6400.0 6600.0

TotalProfit[$/MWmonth]

Fig. 7-22 October profit vs. average absolute deviation from schedule, $5/MWh deviation penalty.
Table 7-7 Results summary for October with different decision criteria, $5/MWh deviation penalty.
Criteria
pf
zero
E*
C*(w=0.1)
C*(w=0.3)
U*(=-3)
U*(=3)

Total Profit
[$]
6,517.7
5,578.0
6,304.3
6,442.9
6,417.7
6,382.9
6,279.3

CVAR
[$/hr]
-3.64
-0.62
-7.88
-4.65
-2.22
-6.61
-8.58

Variance
[$2/hr]
76.1
58.6
98.4
85.2
73.2
92.6
101.2

103

Avg. Bid
0.38
0.00
0.50
0.39
0.28
0.48
0.53

Avg.
Deviation
0.17
0.39
0.28
0.23
0.21
0.26
0.30

So far, we have considered WPF uncertainty as a set of scenarios in the DA bidding problem.
However, one could also use the forecast quantiles directly to decide on how much to bid into the
DA market. In fact, it was shown in [23] that for a risk-neutral wind power producer that always
faces a financial loss from deviations, the optimal schedule is the c+/(c++c-) quantile, where c+, care the cost of positive and negative deviations. These assumptions do not hold true in
U.S. LMP-based markets, as discussed in Section 7.3.
Fig. 7-23 shows the monthly profit for different quantile bids. When there is no deviation
penalty, the wind power producers profit increases by bidding a higher quantile, and the 90%
quantile gives the highest profit (this corresponds to overscheduling, as there is a 90% chance
that the actual generation will be below this quantile). However, the bidding strategy based on
the expected profit criterion (E*) gives a considerably higher profit. When a deviation penalty is
introduced, a bid according to 60% quantile produces a slightly higher profit than the point
forecast, whereas bids according to E*gives a lower profit in this case. As a reference, optimal
bidding with perfect information on prices and wind power would produce a total monthly profit
of $9,740 and $8,405, respectively, in the cases with and without a deviation penalty. Bidding
the realized wind generation would yield $7,255 in both cases. Fig. 7-24 shows that the CVAR
decreases for higher quantile bids, which means that the risk level increases as more capacity is
offered into the DA market.
6600

7500
7400
7300
7200

quantiles

7100

pf

7000

E*

6900

TotalProfit[$/MWmonth]

TotalProfit[$/MWmonth

7600

6800

6500
6400
6300
6200

quantiles

6100

pf

6000

E*

5900
5800

10 20 30 40 50 60 70 80 90

10 20 30 40 50 60 70 80 90

Quantile

Quantile

10

20

30

40

50

60

70

80

10

90

20

30

40

50

60

70

80

90

Quantile

Quantile
mean

0
1
2
3
4
5
6
7
8
9
10

8.9
8.8
8.7
8.6
8.5
8.4
8.3
8.2
8.1
8
7.9
7.8

CVaR[$/MWhr]

0
1
2
3
4
5
6
7
8

TotalProfit[$/MWmonth]

9.8
9.7
9.7
9.6
9.6
9.5
9.5
9.4
9.4

CVaR[$/MWhr]

TotalProfit[$/MWmonth]

Fig. 7-23 Monthly profit for different quantile forecasts compared to bidding based on point
forecast and based on the expected profit criterion, no penalty (left), $5/MWh deviation penalty
(right).

CVARh

mean

CVARh

Fig. 7-24 Average hourly profit and CVAR for different quantile forecasts, no penalty (left), $5/MWh
deviation penalty (right).

104

80
60

NW
QC
Linear QR
Splines QR

40

-2

Deviation [%]

Intervals mean length [% of rated power]

NW
QC
Linear QR
Splines QR

20

Finally, we test the use of different methods to estimate the probabilistic wind power forecast. In
the results presented above, the probabilistic forecasts were based on quantile regression, which
is a relatively common approach to statistically estimate WPF uncertainty. However, in this
project, we have developed two novel methods for probabilistic WPF with kernel density
estimation (KDE), namely based on the Nadaraya-Watson (NW) [93] and quantile copula (QC)
[94] [95] estimators. Results from [93], [94], and [95] indicate that the KDE methods produce
better probabilistic forecasts than quantile regression (QR) in terms of calibration, which is a
measure for how well the forecasted quantiles match the distribution of realized wind power
generation. We find the same results when comparing different probabilistic forecasting methods
on the wind farm data for this case study. Fig. 7-25 shows that the deviation from perfect
calibration is smaller for the KDF-based NW and QC methods compared to linear and splines
QR. When it comes to sharpness, which is a measure of the width of the confidence intervals of
the probabilistic forecasts, there are only small differences between the four methods.

10

20

30

40

50

60

70

80

90

98

10

Nominal proportion rate [%]

20

30

40

50

60

70

80

90

Nominal coverage rate [%]

Fig. 7-25 Evaluation of probabilistic forecasts for different methods: calibration (left) and
sharpness (right). Training period: JanuaryJuly 2006; test period: AugustDecember 2006.

In order to investigate the impact of the different forecasting methods, we repeat the monthly
runs using the NW and QC methods for probabilistic WPF and for the CVAR strategy under a
$5/MWh deviation penalty. The results are compared to the original run with probabilistic WPF
from QR and scenario generation. Table 7-8 shows that the choice of probabilistic wind power
forecast apparently has only limited impact on the results in this case. For the current
assumptions, the differences between the WPF methods are small for the CVAR bidding
strategy. The differences between the various bidding strategies, which are driven largely by the
price projections, are much larger, as we have seen above. However, further work is needed to
investigate how the decisions are influenced by the probabilistic WPF methods. Our initial
results from a real-world wind farm indicate that the choice of forecasting method has a larger
impact on the results.

105

Table 7-8 Summary of results for October with the C*(w=0.1) strategy for different wind power
uncertainty forecasts, $5/MWh deviation penalty.
Total Profit
[$]

CVAR
[$/hr]

Variance
[$2/hr]

Avg. Bid

Avg. Deviation

6,442.9

-4.65

85.2

0.395

0.231

QC

6,433.5

-4.74

85.3

0.401

0.234

NW

6,433.2

-4.70

85.1

0.401

0.235

Criteria
QR
scenario

7.7 Conclusion
Improved scheduling decisions and better risk management will increase the viability of wind
power in the long run. We conclude that stochastic models, like the one proposed here, will be
important tools to control the trade-off between risk and return for wind power producers
participating in electricity markets. The results presented in this chapter demonstrate that
probabilistic wind power forecasts can play an important role in guiding wind power trading
decisions in the DA and RT markets. The results also indicate that price forecasts are important
in order to derive good trading strategies, particularly under market designs without penalties for
deviations between DA schedule and RT delivery. Furthermore, risk preferences clearly
influence the optimal trading decisions, with risk-averse owners bidding less energy into the DA
market to reduce the exposure to high RT prices. The decisions are also highly dependent on
market design, and the results show that a deviation penalty drives the optimal DA bid closer to
the point forecast for all strategies. This is in the interest of the system operator, as it makes the
task of balancing supply and demand in the system an easier one. However, deviation penalties
also add a significant financial burden on wind power producers and therefore reduce the
profitability of wind power. Revising market rules to give efficient operational incentives
without adding an undue burden on wind power and other renewables will clearly be a challenge
as we move forward.
The wind power trading model and analysis outlined in this chapter are general in nature and can
be extended in several directions. For instance, the model could benefit from more advanced
price predictions, which capture potential relationships between wind power and prices. Further
exploration of different probabilistic WPF methods and their impact on trading decisions should
also be investigated. Moreover, alternative market rules (different deviation penalties and other
incentive schemes, intra-day markets, etc.) can be introduced for a more detailed analysis of the
impact of market design on bidding decisions, wind power profits, and the overall operation of
the power system. Considering the interplay between different assets, including wind power, in a
portfolio is also an interesting extension. Overall, more testing on real-world wind data would
contribute to validate the methodology and document the advantages of the proposed modeling
framework. We are addressing some of these issues in our recent work, which is documented in
[102].

106

CONCLUSIONS

This report presented our work on the application of advanced WPF in operational decisions,
from the perspectives of both system operators and wind power producers. The main findings
and conclusions are summarized below.
Different agents in the electricity market may disagree when it comes to assessing a wind power
forecast according to its value and quality. Our case study of the Iberian electricity market
showed that a neutral model (in the forecasters paradigm sense) might not lead to maximized
profits for wind power producers. In fact, if imbalance prices are asymmetric, the profitmaximizing bid will deviate from the unbiased point forecast. This means that a wind power
producer may be willing to reduce the forecasts accuracy in exchange for an increase in income.
Yet, electricity markets should provide the participants with incentives for delivering their
unbiased forecast information through their scheduling and bidding. Our contributions regarding
this topic include:

A challenge to the assumption of the mathematical neutrality of the forecasting models;

A thorough discussion of conflicting interests associated with the goodness of forecasts
for wind power producers and system operators;
A comparison of the relative advantages and drawbacks of several state-of-the art criteria
used in the training of statistical WPF models;
A new approach proposed to achieve acceptable compromise solutions to market
participants in the choice of a forecasting model with the parametric correntropy criterion
from information theoretic learning; and
The verification of the proposed theoretical concepts in a representative case study of the
Iberian electricity market (Spain and Portugal).

WPF can clearly contribute to a more efficient integration of wind power and other renewable
resources into the electric power grid. In a review of the current use of WPF in five ISO/RTO
markets in the United States, we found that most system operators are already using WPF to
support operational decisions; however, the experience to date is still relatively limited. There is
therefore a need to better integrate WPF into operational procedures, including determination of
operating reserve requirements, DA market clearing, reliability commitment decisions, and RT
dispatch. Improving the efficiency and reliability of power systems with high shares of wind
power can be achieved through:

Developing new tools and procedures for system operators and market participants, so as
to make more efficient use of WPF information, enabling better operational decisions
under the growing uncertainty and variability due to the increasing penetration of
renewables;
Making improvements in general decision making under uncertainty and particularly in the
handling of wind power uncertainty in commitment and dispatch of energy and operating
reserves; and
Promoting improvements in WPF models by better tuning them to the specific needs of the
forecast users, such as better predictions of uncertainty and ramping events for system
operators.

107

We have conducted a detailed analysis of the impact of WPF errors on UC and ED. In particular,
we have analyzed the potential use of probabilistic WPF in the scheduling and dispatch of power
systems with a mix of thermal power plants and wind power. We have developed a stochastic
UC model, where wind power uncertainty is represented in terms of a set of forecast scenarios.
We have also developed a scenario reduction algorithm to select an adequate set of wind power
scenarios for use in stochastic UC. Moreover, we have explored how probabilistic forecasts can
be used to derive dynamic operating reserve requirements for use in a deterministic UC. Based
on the application of these tools in several case studies, we conclude that:

Commitment decisions based on a deterministic wind power point forecast are too risky
and are likely to result in high levels of unserved load and reserve. Hence, additional
operating reserves are needed to handle the uncertainty in wind power.
For deterministic UC, a dynamic operating reserve requirement based on probabilistic
forecasts performs better than a fixed additional reserve, because the amount of
additional operating reserves is adjusted according to the predicted forecast uncertainty.
Stochastic UC with wind power scenarios has advantages over the traditional
deterministic scheduling approaches. First, stochastic UC addresses inter-temporal
variability through the scenario representation of uncertainty. Second, the expected cost
of scarcity is explicitly taken into account in the objective function. However, the high
computational burden may prevent implementation on large-scale systems.
The choice of scenario set is very important in stochastic UC. Formal scenario reduction
methods appear to lower the level of hedging in commitment decisions, as they tend to
reduce the overall scenario variance. At the same time, a higher number of scenarios
improves the operating results, but may be prohibitive for computational reasons. A
certain level of additional operating reserves is therefore also likely to be needed with
stochastic UC, as the scenarios do not capture the full range of forecast uncertainty.
Deterministic UC combined with dynamic operating reserves does not address intertemporal ramping events, but the approach is better aligned with current operating
practices and the computational burden is much smaller than with stochastic UC.
Stochastic UC with forecast scenarios and deterministic UC with dynamic operating
reserves give similar results in terms of cost and reliability in our case studies, although
there may be distinct differences in commitment and dispatch decisions.
Probabilistic forecasts contribute to improved operational decisions both in the cases of
dynamic operating reserves and stochastic UC.
More work should be conducted to analyze the potential use of probabilistic forecasting
and stochastic scheduling under more realistic assumptions, for instance by introducing
additional operational constraints (e.g., transmission) in the UC formulation.
It is important for system and market operators to consider their risk preferences in
determining operational strategies, and properly represent this in the mathematical
models used to support their decisions.
We recommend that system operators start making use of probabilistic forecasting
information in their scheduling decisions. A first step would be to move toward the use of
dynamic operating reserve requirements. Next, as probabilistic WPF and computational
resources improve, system operators should consider an entirely stochastic scheduling
approach, where WPF uncertainty is fully represented in the UC formulation.

108

We also investigated the use of probabilistic WPF for wind power producers. We presented a
model for optimal trading of wind power in DA electricity markets under uncertainty in wind
power and prices. We argued that such models constitute important tools to control the trade-off
between risk and return for wind power producers that participate in the electricity market. Our
results have demonstrated that:

Probabilistic WPF can play an important role in guiding wind power trading decisions in
the DA and RT markets;
Price forecasts are important in order to derive good trading strategies, particularly when
there are no deviation penalties between DA schedule and RT delivery;
Risk preferences influence the optimal trading decisions, with risk-averse wind power
producers bidding less energy into the DA market to reduce the exposure to high RT
prices;
Deviation penalties drive the optimal DA bid closer to the point forecast, which implies
that system operators have an easier task in balancing supply and demand, but also that
wind power producers income is reduced;
Improved scheduling decisions and better risk management will increase the viability of
wind power in the long run; and
Overall, an important challenge will be to define market rules that provide efficient
operational incentives without altering the burden on wind power and other renewables.

109

This page intentionally blank

110

REFERENCES
[1]

U.S. Department of Energy (DOE), 20% Wind Energy by 2030: Increasing Wind
Energys Contribution to U.S. Electricity Supply, DOE Office of Energy Efficiency and
Renewable Energy Report, July 2008. Online: http://www1.eere.energy.gov/
windandhydro/wind_2030.html.

[2]

European Union (EU), Climate change: Commission welcomes final adoption of

Europe's climate and energy package, Press Release, EU, Dec. 17, 2008. Online:
http://europa.eu/rapid/pressReleasesAction.do?reference=IP/08/1998.

[3]

Global Wind Energy Council (GWEC), Global Wind Report: Annual Market Update
2010, GWEC Report, April 2011.

[4]

C. Monteiro, R. Bessa, V. Miranda, A. Botterud, J. Wang, G. Conzelmann, Wind Power

Forecasting: State-of-the-art 2009, Technical report ANL/DIS-10-1. Argonne National
Laboratory, 2009. Online: http://www.dis.anl.gov/projects/windpowerforecasting.html.

[5]

J. Mendes, R.J. Bessa, H. Keko, J. Sumaili, V. Miranda, A. Botterud, Z. Zhou, J. Wang,

Development and Testing of Improved Wind Power Forecasting Methods, Technical
report, INECS Porto and Argonne National Laboratory, Sept. 2011.

[6]

A. Costa, A. Crespo, J. Navarro, G. Lizcano, H. Madsen, E. Feitosa, A review on the

young history of the wind power short-term prediction, Renewable and Sustainable
Energy Reviews, Vol. 12, pp. 17251744, 2008.

[7]

B. Ernst, B. Oakleaf, M.L. Ahlstrom, M. Lange, C. Moehrlen, B. Lange, U. Focken,

K. Rohrig, Predicting the wind, IEEE Power & Energy Magazine, Vol. 5, No. 6,
pp. 7889, 2007.

[8]

H. Madsen, P. Pinson, H. Kariniotakis, H.A. Nielsen, T.S. Nielsen, Standardizing the

performance evaluation of short-term wind prediction models, Wind Engineering,
Vol. 29, pp. 475-489, 2005.

[9]

Industry Work Group, Wind power forecasting pilot project industry work group
report, Technical Report, 2008. Online: http://www.aeso.ca/gridoperations/13825.html.

[10]

A.H. Murphy, What is a good forecast? An essay on the nature of goodness in weather
forecasting, Weather and Forecasting, Vol. 8, pp. 281-293, 1993.

[11]

A.H. Murphy, R.L. Winkler, A general framework for forecast verification, Monthly
Weather Review, Vol. 115, pp. 13301338, 1987.

[12]

P. Pinson, Estimation of the uncertainty in wind power forecasting, PhD Thesis, Ecole
des Mines de Paris, Paris, France, 2006.

111

[13]

T.R. Stewart, R.W. Katz, A.H. Murphy, Value of weather information: a descriptive
study of the fruit-frost problem, Bulletin of the American Meteorological Society,
Vol. 65, pp. 126137, 1984.

[14]

J. Usaola, O. Ravelo, G. Gonzlez, F. Soto, M.C. Dvila, B. Daz-Guerra, Benefits for

wind energy in electricity markets from using short term wind power prediction tools; a
simulation study, Wind Engineering, Vol. 28, pp. 119127, 2004.

[15]

R.J. Barthelmie, F. Murray, S.C. Pryor, The economic benefit of short-term forecasting
for wind energy in the UK electricity market, Energy Policy, Vol. 36, pp. 16871696,
2008.

[16]

J.L. Angarita-Mrqueza, C.A. Hernandez-Aramburo, J. Usaola, Analysis of a wind

farms revenue in the British and Spanish markets, Energy Policy, Vol. 35, pp. 5051
5059, 2007.

[17]

P. Pinson, C. Chevallier, G. Kariniotakis, Trading wind generation with short-term

probabilistic forecasts of wind power, IEEE Transactions on Power Systems, Vol. 22,
No. 3, pp. 11481156, 2007.

[18]

A. Fabbri, T. Gomez, J.R. Abbad, V.H. Quezada, Assessment of the Cost Associated
with Wind Generation Prediction Errors in a Liberalized Electricity Market, IEEE
Transactions on Power Systems, Vol. 20, No. 3, pp. 14401446, 2005.

[19]

J. Matevosyan, L. Sder, Minimization of Imbalance Costs Trading Wind Power on the

Short-Term Power Market, IEEE Transactions on Power Systems, Vol. 21, No. 3,
pp. 13961404, 2006.

[20]

F. Bourry, J. Juban, L.M. Costa, G. Kariniotakis, Advanced strategies for wind power
trading in short-term electricity markets, Proceedings European Wind Energy
Conference & Exhibition EWEC 08, Brussels, Belgium, March 31April 3, 2008.

[21]

U. Linnet, Tools supporting wind energy trade in deregulated markets, MSc Thesis,
Technical University of Denmark, Lyngby, Denmark, July 2005.

[22]

R.W. Katz, A.H. Murphy, Quality/value relationships for imperfect weather forecasts in
a prototype multistage decision-making model, Journal of Forecasting, Vol. 9, pp. 75
86, 2006.

[23]

J.B. Bremnes, Probabilistic Wind Power Forecasts Using Local Quantile Regression,
Wind Energy, Vol. 7, pp. 4754, 2004.

[24]

A. Botterud, J. Wang, R.J. Bessa, H. Keko, V. Miranda, Risk Management and Optimal
Bidding for a Wind Power Producer, Proceedings IEEE PES General Meeting,
Minneapolis, Minn., July 2010.

112

[25]

H.F. Ravn, Short term wind power prognosis with different success criteria,
Proceedings International Conference on Probabilistic Methods Applied to Power
Systems PMAPS 2006, Stockholm, Sweden, June, 2006.

[26]

A. Botterud, J. Wang, V. Miranda, R.J. Bessa, Wind Power Forecasting

U.S. Electricity Markets, Electricity Journal, Vol. 23, No. 3, pp. 7182, 2010.

[27]

K. Verhaegen, L. Meeus, R. Belmans, Development of balancing in the internal

electricity market in Europe, Proceedings the European Wind Energy Conference &
Exhibition EWEC 06, Athens, Greece, 2006.

[28]

Y. Rebours, D.S. Kirschen, A survey of definitions and specifications of reserve

services, Technical Report, University of Manchester, Oct. 2005.

[29]

R. Doherty, M. OMalley, A New Approach to Quantify Reserve Demand in Systems

with Significant Installed Wind Capacity, IEEE Transactions on Power Systems,
Vol. 20, No. 2, pp. 587595, 2005.

[30]

M.A. Matos, R. Bessa, Operating Reserve Adequacy Evaluation using Uncertainties of

Wind Power Forecast, Proceedings IEEE PowerTech Conference, Bucharest, Romania,
2009.

[31]

H. Holttinen, Impact of hourly wind power variations on the system operation in the
Nordic countries, Wind Energy, Vol. 8, pp. 197218, 2004.

[32]

R. Doherty, E. Denny, M. OMalley, System Operation with a Significant Wind Power

Penetration, Proceedings IEEE Power Engineering Society General Meeting, Denver,
Colo., USA, June 2004.

[33]

M.D. Ilic, Y. Makarov, D. Hawkins, Operations of Electric Power Systems with High
Penetration of Wind Power: Risks and Possible Solutions, Proceedings IEEE Power
Engineering Society General Meeting, Tampa, Florida, USA, June 2007.

[34]

UCTE NetWork of experts on Wind Power, Wind power in the UCTE interconnected
system, UCTE Position Paper, 2004.

[35]

E. Van Damme, Stability and perfection of Nash equilibria, Springer-Verlag, Berlin,

1991.

[36]

P. Faratin, C. Sierra, N.R. Jennings, Using similarity criteria to make issue trade-offs in
automated negotiations, Artificial Intelligence, Vol. 142, pp. 205237, 2002.

[37]

R. Bessa, V. Miranda, J. Gama, Entropy and Correntropy against Minimum Square

Error in Off-line and On-line 3-Day Ahead Wind Power Forecasting, IEEE Transactions
on Power Systems, Vol. 24, No. 4, pp. 16571666, 2009.

[38]

J.C. Principe, D. Xu, J. Fisher, Information theoretic learning, in Unsupervised

Adaptive Filtering, Simon Haykin (Editor), Wiley, pp. 265319, 2000.
113

in

[39]

D. Erdogmus, J.C. Principe, Generalized Information Potential Criterion for Adaptive

System Training, IEEE Transactions on Neural Networks, Vol. 13, No. 5, pp. 1035
1044, 2002.

[40]

W. Liu, P. Pokharel, J.C. Principe, Correntropy: Properties and Applications in NonGaussian Signal Processing, IEEE Transactions on Signal Processing, Vol. 55, No. 11,
pp. 52865298, 2007.

[41]

J. Xu, Nonlinear signal processing based on reproducing Kernel Hilbert space, PhD
Thesis, University of Florida, Gainesville, Florida, USA, 2007.

[42]

H. Bludszuweit, Reduction of the uncertainty of wind power predictions using energy

storage, PhD Thesis, Universidad de Zaragoza, Spain, 2009.

[43]

North American Electric Reliability Council (NERC), Accommodating High Levels of

Variable Generation, NERC Special Report, April 2009.

[44]

U.S. Department of Energy (DOE), 20% Wind Energy by 2030: Increasing Wind
Energys Contribution to U.S. Electricity Supply, DOE Report, July 2008.

[45]

W. Grant, D. Edelson, J. Dumas, J. Zack, M. Ahlstrom, J. Kehler, P. Storck, J. Lerner,

K. Parks, C. Finley, Change in the Air: Operational Challenges in Wind-Power
Production and Prediction, IEEE Power & Energy Magazine, Vol. 7, No. 6, pp. 4758,
2009.

[46]

M. Ahlstrom, L. Jones, R. Zavadil, W. Grant, The Future of Wind Forecasting and

Utility Operations: Planning for Improved System Operations, IEEE Power & Energy
Magazine, Vol. 3, No. 6, pp. 5764, 2005.

[47]

North American Reliability Council (NERC), Reliability Standards for the Bulk Electric
System
of
North
America,
NERC,
April
2009.
Online:
http://www.nerc.com/page.php?cid=2|20.

[48]

M.A. Ortega-Vazquez, D.S. Kirschen, Estimating the Spinning Reserve Requirements in

Systems With Significant Wind Power Generation Penetration, IEEE Transactions on
Power Systems, Vol. 24, No. 1, pp. 114124, 2009.

[49]

W.W. Hogan, A Model for a Zonal Operating Reserve Demand Curve, MIT Operations
Research Seminar, October 15, 2009. Online: http://www.whogan.com/.

[50]

Electricity Reliability Council of Texas (ERCOT), ERCOT Methodologies for

Determining Ancillary Service Requirements, ERCOT, December 2009. Online:
http://www.ercot.com/mktinfo/services/.

[51]

General Electric (GE), Analysis of Wind Generation Impact on ERCOT Ancillary

Services
Requirements,
GE
Report,
March
28,
2008.
Online:
http://www.uwig.org/opimpactsdocs.html.

114

[52]

R. Wiser, M. Bolinger, 2008 Wind Technologies Market Report, Lawrence Berkeley

National Laboratory, July 2009. Online: http://eetd.lbl.gov/EA/EMS/reports/2008-windtechnologies.pdf.

[53]

R. Sioshansi, D. Hurlbut, Market protocols in ERCOT and their effect on wind

generation, Energy Policy, Vol. 38, No. 7, pp. 31923197, 2010.

[54]

D. Maggio, Integrating Wind Forecasting into Market Operation ERCOT,

Presentation Wind Forecasting Workshop, Utility Wind Integration Group (UWIG),
Phoenix, AZ, February 2009.

[55]

R. Barth, H. Brand, P. Meibom, C. Weber, A Stochastic Unit-commitment model for the

Evaluation of the Impacts of Integration of Large Amounts of Intermittent Wind Power,
Proceedings 9th Int. Conf. on Probabilistic Methods Applied to Power Systems,
Stockholm, Sweden, 2006.

[56]

F. Bouffard, F. Galiana, Stochastic Security for Operations Planning with Significant

Wind Power Generation, IEEE Transactions on Power Systems, Vol. 23, No. 2,
pp. 306316, 2008.

[57]

J. Wang, M. Shahidehpour, Z. Li, Security-Constrained Unit Commitment with Volatile

Wind Power Generation, IEEE Transactions on Power Systems, Vol. 23, No. 3,
pp. 13191327, 2008.

[58]

P.A. Ruiz, C.R. Philbrick, P.W. Sauer, Wind Power Day-Ahead Uncertainty
Management through Stochastic Unit Commitment Policies, Proceedings Power
Systems Conference and Exhibition, Seattle, March 2009.

[59]

A. Tuohy, P. Meibom, E. Denny, M. OMalley, Unit Commitment for Systems With

Significant Wind Penetration, IEEE Transactions on Power Systems, Vol. 24, No. 2,
pp. 592601, 2009.

[60]

J. Wang, A. Botterud, G. Conzelmann, V. Miranda, C. Monteiro, G. Sheble, Impact of

Wind Power Forecasting on Unit Commitment and Dispatch, 8th Int. Workshop on
Wind: Large-Scale Integration of Wind Power into Power Systems, Bremen, Germany,
October 2009.

[61]

J. Usaola, Probabilistic load flow in systems with wind generation, IET Generation,
Transmission and Distribution. Vol. 3, No. 12, pp. 10311041, 2009.

[62]

R. Gonzales, R. Mukerji, M. Swider, D. Allen, R. Pike, D. Edelson, F. Nelson, J. Adams,

Integration of Wind into System Dispatch, New York ISO White Paper, Oct. 2008.
Online: http://www.nyiso.com/public/webdocs/documents/white_papers/wind_ manage
ment_whitepaper_11202008.pdf.

[63]

Utility Wind Integration Group (UWIG), Wind Power and Electricity Markets, UWIG
Report, August 2009. Online: http://www.uwig.org/.

115

[64]

Utility Wind Integration Group (UWIG), Central Wind Power Forecasting Programs in
North America by Regional Transmission Organizations and Electric Utilities, UWIG
Report, December 2009. Online: http://www.uwig.org/.

[65]

N.P. Padhy, Unit CommitmentA Bibliographical Survey, IEEE Transactions on

Power Systems, Vol. 19, No. 2, pp. 11961205, 2004.

[66]

Wind Power Integration in Liberalised Electricity Markets (Wilmar) Project. Online:

http://www.wilmar.risoe.dk.

[67]

A. Tuohy, E. Denny, M. OMalley, Rolling Unit Commitment for Systems with

Significant Installed Wind Capacity, Proceedings 2007 IEEE Lausanne PowerTech,
pp. 13801385, July 15, 2007.

[68]

A. Tuohy, P. Meibom, M. OMalley, Benefits of Stochastic Scheduling for Power

Systems with Significant Installed Wind Power, Proceedings 10th Int. Conference on
Probabilistic Methods Applied to Power Systems (PMAPS), Mayagez, Puerto Rico,
June 2008.

[69]

B.C. Ummels, M. Gibescu, E. Pelgrum, W.L. Kling, A.J. Brand, Impacts of Wind Power
on Thermal Generation Unit Commitment and Dispatch, IEEE Transactions on Energy
Conversion, Vol. 22, No. 1, pp. 4451, 2007.

[70]

P.A. Ruiz, C.R. Philbrick, E. Zak, K.W. Cheung, P.W. Sauer, Uncertainty Management
in the Unit Commitment Problem, IEEE Transactions on Power Systems, Vol. 24,
No. 2, pp. 64251, 2009.

[71]

E. Delarue, W. Dhaeseleer, Adaptive mixed-integer programming unit commitment

strategy for determining the value of forecasting, Applied Energy, Vol. 85, No. 4,
pp. 171181, 2008.

[72]

N. Amjady, F. Keynia, A new spinning reserve requirement forecast method for

deregulated electricity markets, Applied Energy, Vol. 87, No. 6, pp. 18701879, 2010.

[73]

P.J. Luickx, E.D. Delarue, W. Dhaeseleer, Considerations on the backup of wind

power: Operational backup, Applied Energy, Vol. 85, No. 9, pp. 787799, 2008.

[74]

M. Matos, et al. Probabilistic evaluation of reserve requirements of generating systems

with renewable power sources: The Portuguese and Spanish cases, International Journal
of Electrical Power & Energy Systems, Vol. 31, No. 9, pp. 562569, 2009.

[75]

M.A. Matos, R. Bessa, Setting the Operating Reserve Using Probabilistic Wind Power
Forecasts, IEEE Transactions on Power Systems, Vol. 26, No. 2, pp. 594603,
May 2011.

[76]

E. Ela, B. Kirby, E. Lannoye, M. Milligan, D. Flynn, B. Zavadil, M. OMalley,

Evolution of Operating Reserve Determination in Wind Power Integration Studies,

116

Proceedings IEEE Power and Energy Society General Meeting, Minneapolis, Minn.,
July 2010.
[77]

H.A. Nielsen, H. Madsen, T.S. Nielsen, Using Quantile Regression to Extend an

Existing Wind Power Forecasting System with Probabilistic Forecasts, Wind Energy,
Vol. 9, No. 1-2, pp. 95108, 2006.

[78]

P. Pinson, H. Madsen. H.A. Nielsen, G. Papaefthymiou, B. Klckl, From Probabilistic

Forecasts to Statistical Scenarios of Short-term Wind Power Production, Wind Energy
Vol. 12, No. 1, pp. 5162, 2009.

[79]

M. Carrion, J.M. Arroyo, A computationally efficient mixed-integer linear formulation

for the thermal unit commitment problem, IEEE Transactions on Power Systems,
Vol. 21, No. 3, pp. 13718, 2006.

[80]

Midwest ISO (MISO), Energy and Operating Reserve Markets Business Practices
Manual. Online: http://www.midwestiso.org.

[81]

S.A. Kazarlis, A.G. Bakirtzis, V. Petridis, A Genetic Algorithm Solution to the Unit
Commitment Problem, IEEE Transactions on Power Systems, Vol. 11, No. 1, pp. 8392,
1996.

[82]

MISO, Business Practices Manual Energy and Operating Reserve Markets:

Attachment B Day-Ahead Energy and Operating Reserve Market Software Formulations
and Business Logic, Jan. 2009. Online: http://www.midwestiso.org.

[83]

Eastern Wind Integration and Transmission Study (EWITS), EnerNex Corporation.

Online: http://www.nrel.gov/wind/systemsintegration/ewits.html.

[84]

M. Brower, Development of Eastern Regional Wind Resource and Wind Plant Output
Datasets, National Renewable Energy Laboratory, Golden, Co, Subcontract Report No.:
NREL/SR-550-46764, Dec. 2009.

[85]

Electricity Reliability Council of Texas (ERCOT). ERCOT Methodologies for

Determining
Ancillary
Service
Requirements,
Dec.
2009.
Online:
http://www.ercot.com/mktinfo/services/.

[86]

S. Rao, Weifeng Liu, J. Principe, A. de Medeiros Martins, Information Theoretic Mean

Shift Algorithm, Proceedings of the 2006 16th IEEE Signal Processing Society
Workshop on Machine Learning for Signal Processing, 2006, pp. 155160, 2006.

[87]

S. Rao, A. de Medeiros Martins, J.C. Prncipe, Mean shift: An information theoretic

perspective, Pattern Recognition Letters, Vol. 30, No. 3, pp. 222230, Feb. 2009.

[88]

J. Dupaov, N. Grwe-Kuska, W. Rmisch, Scenario Reduction in Stochastic

Programming: An Approach Using Probability Metrics, Mathematical Programming,
Series A, Vol. 95, pp. 493511, 2003.

117

[89]

N. Grwe-Kuska, H. Heitsch, W. Rmisch, Scenario reduction and scenario tree

construction for power management problems, Proceedings 2003 IEEE Bologna Power
Tech Conference, Bologna, Italy, June 2003.

[90]

T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data

Mining, Inference, and Prediction, 2nd Edition, 2009, Springer, New York.

[91]

J.B. MacQueen, Some methods for classification and analysis of multivariate

observations, Proc. 5th Berkeley Symposium on Mathematical Statistics and Probability,
1967, Berkeley, University of California Press, 1, pp. 281297.

[92]

D. Rajan, S. Takriti, Minimum Up/Down Polytopes of the Unit Commitment Problem

with Start-up Costs, Report RC23628, IBM Research, June 2005.

[93]

R.J. Bessa, J. Sumaili, V. Miranda, A. Botterud, J. Wang, E. Constantinescu, Timeadaptive Kernel Density Forecast: A New Method for Wind Power Uncertainty
Modeling, Proceedings 17th Power System Computation Conference (PSCC11),
Stockholm, Sweden, Aug. 2011.

[94]

R.J. Bessa, J. Mendes, V. Miranda, A. Botterud, J. Wang, Z. Zhou, Quantile-Copula

Density Forecast for Wind Power Uncertainty Modeling, Proceedings IEEE Trondheim
PowerTech 2011, Trondheim, Norway, June 2011.

[95]

R.J. Bessa, V. Miranda, A. Botterud, Z. Zhou, J. Wang, Time-Adaptive Quantile-Copula

for Wind Power Probabilistic Forecasting, Renewable Energy, in press, Sept. 2011.

[96]

R.J. Hyndman, D.M. Bashtannyk, G.K. Grunwald, Estimating and Visualizing

Conditional Densities, Journal of Computational and Graphical Statistics, Vol. 5, No. 4,
pp. 315336, 1996.

[97]

J. Sumaili, H. Keko, V. Miranda, A. Botterud, J. Wang, Clustering-based Wind Power

Scenario Reduction Technique, Proceedings 17th Power System Computation
Conference (PSCC11), Stockholm, Sweden, Aug. 2011.

[98]

GAMS/SCENRED. Online: http://www.gams.com/docs/document.htm.

[99]

K. Hoyland, M. Kaut, S.W. Wallace, A Heuristic for Moment-Matching Scenario

Generation, Computational Applications and Optimization, Vol. 24, pp. 169185, 2003.

[100] B. Hasche, Scenario Tree Generation for an Electricity Market Model, Report AP 3.3,
Institut fr Energiewirtschaft und Rationelle Energieanwendung (IER), Universitt
Stuttgart, August 2008.
[101] Z. Zhou, A. Botterud, J. Wang, R.J. Bessa, H. Keko, J. Sumaili, V. Miranda,
Application of Probabilistic Wind Power Forecasts in Electricity Markets, Wind
Energy, submitted, June 2011.

118

[102] A. Botterud, Z. Zhou, J. Wang, R.J. Bessa, H. Keko, J. Sumaili, V. Miranda, Wind
Power Trading under Uncertainty in LMP markets, IEEE Transactions on Power
Systems, accepted, Sept. 2011.
[103] R. Wiser, M. Bolinger, 2009 Wind Technologies Market Report, Lawrence Berkeley
National Laboratory, August 2010.
[104] G.N. Bathurst, J. Weatherill, G. Strbac, Trading Wind Generation in Short-Term Energy
Markets, IEEE Transactions on Power Systems, Vol. 17, No. 3, pp. 782789, 2002.
[105] J. Usaola, M.A. Moreno, Optimal Bidding of Wind Energy in Intraday Markets,
Proceedings 6th International Conference on the European Energy Market, Leuven,
Belgium, 2009.
[106] J.M. Morales, A.J. Conejo, J. Prez-Ruiz, Short-Term Trading for a Wind Power
Producer, IEEE Transactions on Power Systems, Vol. 25, No. 1, pp. 554564, 2010.
[107] C.J. Dent, J.W. Bialek, B.F. Hobbs, Opportunity Cost Bidding by Wind Generators in
Forward Markets: Analytical Results, IEEE Transactions on Power Systems, Vol. 26,
No. 3, pp. 16001608, 2011.
[108] S. Stoft, Power System Economics: Designing Markets for Electricity, IEEE Press,
Piscataway, N.J., 2002.
[109] R. Gonzales, R. Mukerji, M. Swider, D. Allen, R. Pike, D. Edelson, F. Nelson, J. Adams,
Integration of Wind into System Dispatch, New York ISO White Paper, Oct. 2008.
[110] Federal Energy Regulatory Commission (FERC), Integration of Variable Resources,
Notice of Proposed Rulemaking, Docket No. RM10-11-000, Nov. 18, 2010.
[111] T. Jnsson, Forecasting of Electricity Prices Accounting for Wind Power Predictions,
MSc Thesis, Technical University of Denmark, Lyngby, Denmark, 2008.
[112] R.T. Rockafellar, S. Uryasev, Optimization of Conditional Value-at-Risk, Journal of
Risk, Vol. 2, pp. 2141, 2000.
[113] R.L. Keeney, H. Raiffa, Decisions with Multiple Objectives. Preferences and Value
Tradeoffs, Cambridge University Press, 1993.

119

This page intentionally blank

120

rgonne end pag