Abstract
Conditional value-at-risk (CVaR) and value-at-risk, also called the superquantile and quantile, are frequently used to characterize the tails of probability distributions and are popular measures of risk in applications where the distribution represents the magnitude of a potential loss. buffered probability of exceedance (bPOE) is a recently introduced characterization of the tail which is the inverse of CVaR, much like the CDF is the inverse of the quantile. These quantities can prove very useful as the basis for a variety of risk-averse parametric engineering approaches. Their use, however, is often made difficult by the lack of well-known closed-form equations for calculating these quantities for commonly used probability distributions. In this paper, we derive formulas for the superquantile and bPOE for a variety of common univariate probability distributions. Besides providing a useful collection within a single reference, we use these formulas to incorporate the superquantile and bPOE into parametric procedures. In particular, we consider two: portfolio optimization and density estimation. First, when portfolio returns are assumed to follow particular distribution families, we show that finding the optimal portfolio via minimization of bPOE has advantages over superquantile minimization. We show that, given a fixed threshold, a single portfolio is the minimal bPOE portfolio for an entire class of distributions simultaneously. Second, we apply our formulas to parametric density estimation and propose the method of superquantiles (MOS), a simple variation of the method of moments where moments are replaced by superquantiles at different confidence levels. With the freedom to select various combinations of confidence levels, MOS allows the user to focus the fitting procedure on different portions of the distribution, such as the tail when fitting heavy-tailed asymmetric data.





Similar content being viewed by others
Notes
See Sect. 4 for specifics.
When closed-form expressions are not available, we look to provide simple calculation methods that might still be utilized within parametric methods.
Also called the product logarithm or omega function.
Or minimized if we consider the negative.
If this was not true, it would imply that there exists a portfolio with smaller bPOE and hence the current portfolio is not bPOE optimal. A formal proof can be found in Mafusalov and Uryasev (2018).
Note that bPOE thresholds refer to loss thresholds instead of return thresholds. See Sect. 4.1.
Right superquantile of loss distribution.
Also sometimes called L-moments.
A simple empirical estimate from a sample of N observations \(S=\{X_1,\dots ,X_N\}\) can be obtained by sorting S and calculating the average of the largest \((1-\alpha )N\) observations. More precise estimates can be obtained by weighted averages; see Proposition 8 of Rockafellar and Uryasev (2002) for details.
Specifically, we used the leastsq function which implements MINPACK’s lmdif routine. This routine requires function values and calculates the Jacobian by a forward-difference approximation.
The fit from LS2 curves toward zero because it is the only fit with \(k > 1\).
References
Andreev, A., Kanto, A., & Malo, P. (2005). On closed-form calculation of CVaR. Helsinki School of Economics working paper W-389.
Artzner, P., Delbaen, F., Eber, J. M., & Heath, D. (1999). Coherent measures of risk. Mathematical Finance, 9, 203–228.
Davis, J. R., & Uryasev, S. (2016). Analysis of tropical storm damage using buffered probability of exceedance. Natural Hazards, 83(1), 465–483.
Everitt, B. S. (2006). The Cambridge dictionary of statistics. Cambridge: Cambridge University Press.
Karian, Z. A., & Dudewicz, E. J. (1999). Fitting the generalized lambda distribution to data: A method based on percentiles. Communications in Statistics: Simulation and Computation, 28(3), 793–819.
Landsman, Z. M., & Valdez, E. A. (2003). Tail conditional expectation for elliptical distributions. North American Actuarial Journal, 7(4), 55–71.
Mafusalov, A., Shapiro, A., & Uryasev, S. (2018). Estimation and asymptotics for buffered probability of exceedance. European Journal of Operational Research, 270(3), 826–836.
Mafusalov, A., & Uryasev, S. (2018). Buffered probability of exceedance: Mathematical properties and optimization. SIAM Journal on Optimization, 28(2), 1077–1103.
Norton, M., Mafusalov, A., & Uryasev, S. (2017). Soft margin support vector classification as buffered probability minimization. The Journal of Machine Learning Research, 18(1), 2285–2327.
Norton, M., & Uryasev, S. (2016). Maximization of AUC and buffered AUC in binary classification. Mathematical Programming, 174(1–2), 575–612.
Rockafellar, R., & Royset, J. (2010). On buffered failure probability in design and optimization of structures. Reliability Engineering & System Safety, 95, 499–510.
Rockafellar, R., & Uryasev, S. (2000). Optimization of conditional value-at-risk. The Journal of Risk, 2(3), 21–41.
Rockafellar, R. T., & Royset, J. O. (2014). Random variables, monotone relations, and convex analysis. Mathematical Programming, 148(1–2), 297–331.
Rockafellar, R. T., & Uryasev, S. (2002). Conditional value-at-risk for general loss distributions. Journal of Banking & Finance, 26(7), 1443–1471.
Sgouropoulos, N., Yao, Q., & Yastremiz, C. (2015). Matching a distribution by matching quantiles estimation. Journal of the American Statistical Association, 110(510), 742–759.
Shang, D., Kuzmenko, V., & Uryasev, S. (2018). Cash flow matching with risks controlled by buffered probability of exceedance and conditional value-at-risk. Annals of Operations Research, 260(1–2), 501–514.
Uryasev, S. (2014). Buffered probability of exceedance and buffered service level: Definitions and properties. Department of Industrial and Systems Engineering, University of Florida, research report 3.
Funding
This research was supported by the Naval Postgraduate School’s Research Initiation Program.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Proofs
Proofs
1.1 Proposition 1
Proof
First, note that \(q_\alpha (X) = \frac{-\ln (1 - \alpha ) }{\lambda }\) for exponential RVs with rate parameter \(\lambda \). We then have,
Since \(\int \ln (y)dy = y \ln (y) - y + C\), we have
We can then see that
\(\square \)
1.2 Corollary 1
Proof
We know that X, being exponential, has CDF given by \(P(X \ge x) = 1 - e^{-\lambda x}\). From Proposition 1, we know that
Then, since \(\mu =\frac{1}{\lambda }\), it follows that \({\bar{p}}_x (X) = e^{-\lambda ( x - \mu )} = 1 - P(X \le x - \mu ) = P(X > x - \mu )\). The equality for CVaR follows easily from Proposition 1 since \(q_\alpha (X) = \frac{-\ln (1 - \alpha ) }{\lambda }\).
\(\square \)
1.3 Proposition 2
Proof
First, note that the conditional distribution of a Pareto, conditioned on the event that the random value is larger than some \(\gamma \), is simply another Pareto with parameters \(a,\gamma \). This implies that \(E[ X | X > \gamma ] = \frac{ a \gamma }{ a - 1}\) if \(a\ge 1\); otherwise the expectation is \(\infty \). Also, \(1- F(\gamma ) = \left( \frac{x_m}{\gamma } \right)^a\). Since,
we will have that,
This gives us bPOE formula,
Since \(a>1\), the maximization objective is concave over the range \(\gamma \in (0,\infty )\) which contains the range \([x_m,x)\), so we just need to take the gradient of function \(g(\gamma )=\frac{\gamma ^{a-1} (x-\gamma )(a-1) }{x_m^a}\) and set it to zero to find the optimal \(\gamma \) as follows:
Plugging this value of \(\gamma \) into the objective of our bPOE formula yields,
CVaR is then equal to the value of x which solves the equation \(1-\alpha = {\bar{p}}_x(X)\) or,
which has solution,
\(\square \)
1.4 Corollary 2
Proof
It follows by simply comparing the formulas from Proposition 1 and the CDF and quantile formulas for a Pareto RV. \(\square \)
1.5 Proposition 3
Proof
For these results, we rely on the fact that if \(X \sim GPD(\mu ,s,\xi )\), then \(X-\gamma | X> \gamma \sim GPD(0,s+\xi (\gamma -\mu ),\xi )\), meaning that the excess distribution of a GPD random variable is also GPD. Now, note also that if \(\xi <1\), then \(E[X]=\mu + \frac{s}{1-\xi }\). This gives us,
which further implies that,
Plugging in the values of the quantile functions yields the final formulas. Using the formulas we just found for \({\bar{q}}_\alpha (X)\), it is straightforward to solve for \({\bar{p}}_x(X)\) which equals \(1-\alpha \) such that \(\alpha \) solves the equation \(x={\bar{q}}_\alpha (X)\). \(\square \)
1.6 Proposition 4
Proof
To get the superquantile, we begin with the integral representation:
To evaluate the integral, we use simple substitution as well as the identity \(\int \ln (y)dy = y \ln (y) - y + C\). After simplifying, we see that with \(\alpha < .5\) the integral evaluates to,
Similarly, we find that with \(\alpha \ge .5\) the integral evaluates to,
For bPOE, first assume that threshold \(x \ge \mu +b\). Using our formula for CVaR, we see that \({\bar{q}}_{.5} (X) = \mu + b\). Thus, \(x \ge \mu +b\) implies that \(1-{\bar{p}}_x(X) \ge .5\) implying that
Assume contrarily that \(x< \mu + b\). Since \({\bar{q}}_{.5} (X) = \mu + b\), we have that \(1-{\bar{p}}_x(X) < .5\) which implies that
Letting \(z=\frac{x-\mu }{b}\), we must now find \(\alpha \) which solves the equation \(\left(\frac{\alpha }{1-\alpha } \right)(1 - \ln (2 \alpha ))= z\). We do so as follows:
where the final step follows from the definition of the Lambert-\({\mathcal {W}}\) function which is given by the relation \(xe^x=y \iff {\mathcal {W}}(y)=x\). Thus, \(\frac{-z}{\alpha } = {\mathcal {W}}(-2ze^{-z-1}) \implies {\bar{p}}_x(X) = 1-\alpha =1 + \frac{z}{{\mathcal {W}}( -2e^{-z-1}z)} \). \(\square \)
1.7 Proposition 5
Proof
It is well known that if \(X \sim {\mathcal {N}}(0,1)\), then the conditional expectation is given by the inverse Mills Ratio, \(E[ X | X > \gamma ] = \frac{f(\gamma )}{1 - F(\gamma ) }\). It follows then that \({\bar{q}}_\alpha (X) = E[ X | X > q_\alpha (X) ] = \frac{f(q_\alpha (X))}{1 - F(q_\alpha (X)) }= \frac{f(q_\alpha (X))}{1 - \alpha }\). \(\square \)
1.8 Proposition 6
Proof
Note that for a standard normal random variable, the tail expectation beyond any threshold \(\gamma \) is given by the inverse Mills Ratio,
Note also that for any threshold \(\gamma \) and any random variable we have,
Using the Mills ratio gives us,
Plugging this result into the minimization formula for bPOE yields the final formula. \(\square \)
1.9 Proposition 7
Proof
This follows from the fact that \({\bar{q}}_\alpha (X) = E[ X | X > q_\alpha (X) ] = \frac{f(q_\alpha (X))}{1 - F(q_\alpha (X)) }\) and the optimization formula of bPOE given in the previous proposition for normally distributed variables. \(\square \)
1.10 Proposition 8
Proof
To derive the integral representation, simply plug in the formula for \(E[ X - \gamma ]^+\), then utilize the definition of the PDF and CDF. The gradient calculation is a standard calculus exercise. \(\square \)
1.11 Proposition 9
Proof
We simply evaluate the integral of the quantile function as follows.
\(\square \)
1.12 Proposition 10
Proof
To obtain the superquantile, we have
Utilizing simple substitution as well as the identity \(\int \ln (y)dy = y \ln (y) - y + C\), we get
To get bPOE, we simply follow the bPOE definition, needing to find \(\alpha \) which solves \( \mu + \frac{s}{1-\alpha } H( \alpha )=x\). The transformed system arises from combining logarithms within the superquantile formula and applying exponential transformations. \(\square \)
1.13 Proposition 11
Proof
This follows from the fact that \(E[X-\gamma ]^+ = \int _{\gamma }^{\infty } (1-F(t))dt\). Evaluating this integral for \(X \sim Logistic(\mu ,s)\) yields, \(E[X-\gamma ]^+= s\ln ( 1+e^{-(\frac{\gamma -\mu }{s}) } )\) which can then be plugged into the minimization formula for bPOE. The second part of the proposition follows from the fact that the gradient of the objective function w.r.t. \(\gamma \) is given by,
Setting this gradient to zero and simplifying yields the stated optimality condition. \(\square \)
1.14 Proposition 12
Proof
Since there is no closed-form expression for the quantile, we utilize the representation of the superquantile given by \(\frac{1}{1-\alpha } \int _{q_\alpha (X)}^\infty t f(t) dt\). To evaluate this integral, we first take the derivative of the PDF, giving
Rearranging yields,
We can then integrate both sides,
Integrating by parts gives us the following form of the middle term;
Then, finally, after substituting this new expression for the middle term and simplifying, we get
Taking the definite integral yields,
It is easy to see that the second limit goes to one and, after applying l’Hôpital’s rule where necessary, that the first limit goes to zero. This leaves
where the final step comes from writing the non-standardized quantile \(q_\alpha (X)\) and PDF f(x) in their standardized form. Then, finally, dividing by \(1-\alpha \) yields the formula
\(\square \)
1.15 Proposition 13
Proof
To calculate the superquantile, we utilize the integral representation (1), which is
To put this integral into the form of the upper incomplete gamma function, make the change of variable \(y= -\ln (1-p)\). This gives \(e^y = \frac{1}{1-p}\) and \(dp = (1-p) dy = e^{-y} dp\) with new lower limit of integration \( -\ln (1-\alpha )\) and upper limit of integration \( \infty \). Applying to the integral yields
\(\square \)
1.16 Proposition 14
Proof
To calculate the superquantile, we utilize the integral representation as follows:
Now, note first that for \(X \sim LogLogistic(a,b)\), we have \(E[X] =a\frac{\pi }{b} csc\left( \frac{\pi }{b} \right)\). Next, for the incomplete beta function, letting \(A_1 = \frac{1}{b}+1\) and \(A_2=1 - \frac{1}{b}\), we can see that
Using these two facts, we have,
\(\square \)
1.17 Proposition 15
Proof
Assume we have \(\xi =0\). Then, we have
Assume now that \(\xi \ne 0\). Then, we have that,
\(\square \)
Rights and permissions
About this article
Cite this article
Norton, M., Khokhlov, V. & Uryasev, S. Calculating CVaR and bPOE for common probability distributions with application to portfolio optimization and density estimation. Ann Oper Res 299, 1281–1315 (2021). https://doi.org/10.1007/s10479-019-03373-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-019-03373-1