Reading 38 Backtesting and Simulation - Answers
Reading 38 Backtesting and Simulation - Answers
In the presence of return distribution asymmetry and excess kurtosis, the most appropriate
approach would be to make use of a Monte Carlo simulation using a:
A) F-distribution.
B) normal distribution.
C) skewed Student’s t-distribution.
Explanation
The use of a skewed Student's t-distribution is most likely to help account for properties
such as skewness (asymmetry) and excess kurtosis in the underlying return data. .
Which of the following metrics are most likely to be reported in a backtest of an investment
strategy?
Explanation
The backtest of an investment strategy will produce return metrics, such as average
return, as well as risk measures, such as volatility and downside risk. Other measures
commonly calculated include the Sharpe ratio, the Sortino ratio, and maximum drawdown
(the maximum loss from a peak to a trough).
Explanation
Which of the following identifies problems that are most likely to arise in a backtest of an
investment strategy?
Explanation
Which of the following is the least likely to result from using information that would have
been unavailable at the time of the investment decision?
A) Survivorship bias.
B) Look-ahead bias.
C) Data snooping.
Explanation
Explanation
Backtesting is a natural fit for quantitative and systematic investment styles. Backtesting is
based on the implied assumption that the future will somewhat resemble history.
Methods such as Monte Carlo analysis can allow backtesting to take into account the
randomness of the future.
Explanation
Backtesting is not a new methodology; rather, it has been widely used in the investment
community for many years. Backtesting can be employed as a rejection or acceptance
criterion for an investment strategy. Backtesting is a natural fit for quantitative and
systematic investment styles, but it is also widely used by fundamental managers.
A) Positive skewness.
B) Negative skewness.
C) Excess kurtosis.
Explanation
Bill Cassidy, CFA, is the portfolio manager for Applied Logistics pension fund. Cassidy is
meeting with Alex Swary, the senior quantitative analyst, to discuss the results of backtesting
of a model developed by Swary. The model uses several factors in selecting stocks, including
EPS growth over the past year, the industry competitiveness index, and price-to-book ratio.
The model makes picks on the first trading day of each calendar year with annual
rebalancing.
While evaluating the results of backtesting, Cassidy should be most likely concerned with:
Explanation
Factors such as the price-to-book ratio rely on accounting data (from the balance sheet),
which is usually available with a lag—therefore, it may not be available at the time of stock
selection. This fact is often overlooked while using historical data and is called the look-
ahead bias. Survivorship bias results from inclusion of only survivors in the investment
universe, while data snooping involves selection of a winning model (from many) based on
statistical strength of the test results. The question does not provide any evidence to
support either the survivorship bias or the data snooping bias.
Explanation
Historical scenario analysis involves backtesting over discrete periods of structural breaks
—over different regimes. The specification of different investment universes is not
scenario testing.
In conducting a sensitivity analysis, an analyst is most likely to take fat tails and negative
skewness into account by repeating a Monte Carlo simulation using a multivariate:
A) Bernoulli distribution.
B) normal distribution.
C) skewed Student’s t-distribution.
Explanation
To conduct a sensitivity analysis, we fit return data to a distribution that accounts for
skewness and excess kurtosis, such as a multivariate skewed Student's t-distribution and
then repeat the Monte Carlo simulation. .
Explanation
Which of the following most accurately describes the steps in backtesting an investment
strategy?
Explanation
The three steps in backtesting an investment strategy are: (1) strategy design, (2) historical
investment simulation, and (3) analysis of output.
repeated sampling from the same data set leads to the use of redundant
A)
sources.
B) the out-of-sample data becomes the in-sample data for the subsequent period.
C) a data set is divided into two distinct samples.
Explanation
Rolling window relies on an overlap between in-sample and out-of-sample data, allowing
repeated in-sample training data to adjust portfolio positions based on information
available at that time. Data is not divided into just two samples (one for training and the
other for testing).
A) In the “strategy design” step, we form investment portfolios for each period.
In the “historical investment simulation” step, we rebalance the portfolio
B)
periodically.
In the “historical investment simulation” step, we calculate portfolio
C)
performance statistics.
Explanation
The three steps in backtesting an investment strategy are: (1) strategy design, (2) historical
investment simulation, and (3) analysis of output.
In the "strategy design" step, we specify investment hypothesis and goal(s), determine
investment rules and process, and decide key parameters.
In the "historical investment simulation" step, we form investment portfolios for each
period according to the rules specified in the previous step, and rebalance the portfolio
periodically based on predetermined rules.
Backtesting the performance of the strategy, assuming that the CBOE VIX Index
A)
is greater than 55.
Backtesting the performance of the strategy during the great recession, a
B)
period following the global financial crisis of 2008.
Backtesting the performance of the strategy during the high market return
C)
period of 2017–2018.
Explanation
Historical scenario analysis (or historical stress testing) involves backtesting a strategy
during actual historical periods. Assumed VIX level is not a historical period. The
recessionary period following the global financial crisis of 2008 and the high market return
period of 2017–2018 are both historical periods.