Lecture 25: The Dispersion Parameter

Sometimes the exponential family is written in the form fY (y; , ) = exp y B() + C(y, ) , (1)

where B() and C(, ) are known functions, and the range of Y does not depend on or . In this formulation, we call the canonical parameter, and the dispersion parameter. If the distribution is parameterized in terms of the mean of Y , , so that g() for some function g, then g() is the canonical link. If is known, then (1) agrees with the usual denition of the 1-parameter exponential family in canonical form. NOTE: For a random variable Y with distribution of the form (1), E[Y ] = B () and Var[Y ] = B () V (). Here V is called the variance function. Thus, the variance function is equal to B () for exponential families. NOTE: The dispersion parameter and variance function are unique only up to a constant, since Var[Y ] = V () = (c)[V ()/c] = V () for any constant c, where is now considered the dispersion parameter, and V () is now considered the variance function. However, by convention, we would take both and V to be positive, since theyre contributing to the variance, which is always positive. Example: Normal distribution The density of a random variable Y with N(, 2 ) distribution can be written as fY (y) =
(y)2 1 e 22 2 2 y 2 /2 1 2 2 = exp [y / + log(2 2 )] . 2 2

So, B() = 2 /2 and C(y, ) = 1 [y 2 / 2 + log(2 2 )]. The dispersion parameter is 2 . The 2 canonical parameter is , and E[Y ] = and Var[Y ] = 2 . By convention (and in S-PLUS), we take V () = 1 and = 2 .

Example: Gamma distribution The density of a random variable Y with Gamma(, ) distribution, 0 < , , y < , can be written as fY (y) = y 1 ey () = exp {y + log + ( 1) log y log ()} y(/) [ log ] = exp + ( 1) log y log () . 1/

Letting / and 1/, fY (y) = exp y [ log()] log()/ + (1/ 1) log y log (1/)

Therefore, the Gamma distribution is in the exponential family with B() = log() and dispersion parameter 1/. This denition of is conventional, and is used by S-PLUS. Since 1 E[Y ] = B () = and Var[Y ] = B () = 2 = 2 , 2 this denition then implies that V () = . The easiest way to nd the canonical link for the Gamma distribution is to parameterize it in terms of its mean. We can compute Thus,
y 1 log() fY (y) = exp log()/ + (1/ 1) log y log (1/) .
1 Therefore, the canonical link function is g() = .

Example: Poisson distribution The distribution of a random variable Y with a Poisson() distribution can be written as fY (y) = e y y! = exp {y log log y!} y log log y! . = exp 1

So, the canonical parameter is log . In this case, the dispersion parameter is 1. It is for this reason that S-PLUS gives the message 2

(Dispersion Parameter for Poisson family taken to be 1 ) in the summary output when we t a Poisson GLM. Example: Binomial distribution The distribution of a random variable Y with Binomial(n, p) distribution can be written as fY (y) = n y py (1 p)ny p + n log(1 p) + log 1p
p 1p

= exp y log = exp

n y n y

+ n log(1 p) 1

+ log

p So, the canonical parameter is log 1p . Like in the Poisson case, in this case, the dispersion parameter is also 1. It is for this reason that S-PLUS gives the message

(Dispersion Parameter for Binomial family taken to be 1 ) in the summary output when we t a Binomial GLM. Extending the GLM Framework to Allow for a Dispersion Parameter In most situations, is unknown. In this case, the distribution (1) does not t into the usual GLM framework (where we assume that the distribution is in the 1-parameter exponential family). However, it turns out that we can still use the GLM framework to model observations with distribution (1). In particular, we treat as known and common to all observations. Then, we let (and hence ) vary among observations in the usual way, i.e. for observations Y1 , . . . , Yn , we assume that
p

g(i ) =
j=1

xij j .

The dierence between this model and usual GLMs is that, in addition to estimating the j s, we will now need to estimate as well. We can obtain such an estimate using the Pearson chi-squared statistic and its asymptotic properties. Estimating the Dispersion Parameter We have dened the Pearson chi-squared statistic in the special case where Yi has a Poisson or binomial distribution. In general, this statistic is dened as X2 = (Yi i )2 , V (i ) i=1 3
n

where Var[Yi ] = V (i ). Exercise: Check that this denition is consistent with that given for Poisson or binomial data. The scaled Pearson chi-squared statistic is dened as
2 Xs

X2 = .

It turns out that, if the model is specied correctly,

asymptotically, where n is the sample size and p is the number of unknown regression coefcients (the j s) in the model.
2 Since the mean of a 2 random variable is np, we can use the approximation Xs np, np and hence the estimator

X2 . np

Note that this is not the MLE of (it is actually a moment estimator). However, it has some nice properties not shared by the MLE. Example: Normal distribution (cont.) For independent observations Y1 , . . . , Yn with Yi N(i , 2 ), Var[Yi ] = V (i ) 1 2 so V (i ) 1 and X2 = Therefore, = 2 =
n i=1 n i=1 (Yi

(Yi i )2 . i )2 . np

This is the usual unbiased estimator of 2 . (The MLE, which has n rather than n p in the denominator, is biased.) Example: Gamma distribution (cont.)

For independent observations Y1 , . . . , Yn with Yi Gamma(i , ), Var[Yi ] = B () = 2 2 i so V (i ) 2 and i (Yi i )2 X = . 2 i i=1

Therefore, =

(Yi i )2 . 2 i=1 i (n p)

Since moment estimators also have the invariance property (like MLEs), we can estimate by 1 = .