Regression Techniques

Class Test 1
Max. Marks: 10
December 18, 2020

(1) Consider the model Y = β0 + β1 X1 + β2 X2 + β3 X3 + β4 X4 + . If it is suggested to you that the two

variables Z1 = X1 + X3 and Z2 = X2 + X4 might be adequate to represent the data, what hypothesis,
in the form Cβ = 0, would you need to test? Given n observations from this model, find the test under
the assumption of normality.
(6)
2
(2) Consider the model Y = Xβ+, with X is n×p, p < n matrix and has full rank. Given that Var[Y] = σ In
and Y
b = Xβ,b where βb is the LSE of β, prove that
n
X
Var[Ybi ] = σ 2 p.
i=1

(4)

1
 Regression Techniques
Class Test 2
Max. Marks: 15
January 5, 2021

(1) Given the general linear model Yi = β0 + β1 xi1 + · · · βp−1 xi,p−1 + i , i = 1, . . . , n (n > p). Test whether
or not the regression on the regression variables is significant. Write your test statistic in terms of the
multiple correlation coefficient, R.
(5)
(2) In linear regression with two covariates X1 and X2 , what can you say about the LSE of the linear
parameters corresponding to these covariates?
(2)
(3) Please indicate right/wrong with justification. No credit will be given for answer lacking justification.
(a) When R2 = 1, all the residuals must be zero.
RSS
(b) In full rank linear regression model, S 2 = n−p is an unbiased estimator of the error variance σ 2 even
if one considers an overfitted model.
(c) In full rank linear regression model with incetcept, if the ith data point is a high-leverage point, then
the Mahalanobis distance corresponding to ith observation will be small and ith hat matrix diagonal
1
will be close to n.
(1+3+4)

1
 Regression Techniques
Class Test 3
Max. Marks: 15
January 19, 2021

(1) Consider the scaled and centered multiple regression model such that the regression matrix is a full rank
matrix. Write down the jth variance inflation factor in terms of the eigen values of the sample correlation
matrix of the explanatory variables. What can you say if the minimum eigen value is close to zero?
(5)
(2) Consider fitting a full rank multiple regression model y = Xβ +  by the least squares method. Let hi be
the ith diagonal entry of the hat matrix H.
(a) Your friend says that the difference between the LSE of β and ith deleted LSE of β does not depend
on the ith response. Do you agree? Give justification of your answer.
(b) Show that the ith deleted residual ei(−i) = yi − xTi β
b
(−i) can be expressed as
ei
ei(−i) =
1 − hi
where xi is the ith row of X (as a column vector), ei is the ith residual from the full model fit, and
β
b
(−i) is the ith deleted LSE of β.
(2+3)
(3) Let Z1 , Z2 , . . . , Zn be independent random variables and let Zi have a N (iθ, i2 σ 2 ) distribution for i =
1, 2, . . . , n. Find the weighted least squares estimator of θ and its variance.
(5)

1
 Regression Techniques
Class Test 4
Max. Marks: 15
February 18, 2021

(1) Please indicate right/wrong with justification. No credit will be given for answer lacking justification.

(a) The ridge estimator can be expressed in terms of the least squares estimator and is therefore unbiased.

(b) If the coefficient of determination in case of the regression of the k-th covariate on other covariates
is large, then the k-th variance inflation factor will be small.

(c) In linear regression with double exponential error, the maximum likelihood estimators of the regres-
sion coefficients are the lease absolute deviation estimators.

(d) The finite-sample breakdown point for least squares estimation method is 12 .

(e) A point will have a large Cook’s Distance if it has a large Studentized residual or is a high-leverage
point.
(10)
(2) Let Yi = βxi + ui , xi > 0 (i = 1, 2, . . . , n), where ui = ρui−1 + i and the i are independently distributed
as N (0, σ 2 ) and are independent of uj for j < i. If βb is the ordinary least squares estimate of β, prove
that var[β]
b is inflated when ρ > 0.
(5)

1
 India Statistical Institute, Delhi Centre
Master of Statistics (NB-stream)
Regression Techniques
End-Semester Examination
March 17, 2021
Max. Marks: 50 Time: 10:00 AM - 12:50 PM

1. Consider the centered multiple regression model

yi = α0 + β1 (x1i − x̄1 ) + · · · + β1 (xp−1,i − x̄p ) + i , i = 1, . . . , n

n
1X b 0 (x0 − x̄);
where x̄k = xki . The aim is to predict y at x0 = (x10 , x20 , . . . , xp−1,0 )0 using yb = ȳ + β c
n i=1
b 0 = (βb1 , . . . , βbp−1 ), ȳ = 1 Pn yi ; βbi is the least squares estimate of βi , i =
x̄ = (x̄1 , . . . , x̄p−1 )0 , β c n i=1
1, . . . , p − 1. Prove that the variance of the predictor yb at x0 depends on how outlying x0 is. Then
explain how collinearity affects the prediction. (8)

2. Consider a regression with regression matrix X, response vector Y and LSE β.

b Suppose that a new
case with data (x, Y ) is added. Show that the residual sum of squares is increased by an amount
e2 /(1 + x0 (X0 X)x), where e = Y − x0 β.
b

Hint: Let A be m × m non-singular matrix and u and v be m−vectors. Then

A−1 uv0 A−1 A−1 uv0 A−1
(A + uv0 )−1 = A−1 − , (A − uv0 )−1 = A−1 + .
1 + v0 A−1 u 1 − v0 A−1 u
(8)

3. (a) Show that the first step in forward selection is equivalent to selecting the variable most highly
correlated with the response.
(b) Consider a regression with K explanatory variables. Let Fp be the F statistic for testing that
some specified subset of r variables can be deleted from the model, where r = K +1−p. Show that
the value of Mallow’s Cp for this reduced model with the r variables deleted is Cp = r(Fp − 1) + p.
(3+5)

4. In a multiple linear regression model, Ridge estimate is the solution of a constrained least squares
problem. Explain. (4)

5. In a multiple linear regression model Y = Xβ +  where X is n × p with rank p, let H = ((hij )) be

the hat matrix or the projection matrix on the column space of X. Let β
b be the LSE of β and Yb be
the predicted value of Y.

(a) Show that

X
(1 − hii )2 + h2ij = (1 − hii ).
j6=i

(b) If the i-th observation is an outlier and a high-leverage point, then Ybi will be highly affected by
Yi . Explain. (2+3)

6. Let Yi be the sample proportion of successes for the ni trials such that E[Yi ] = πi . There is a
p
X
linear predictor xij βj which is related to π using a logit link. Obtain the likelihood equations for
j=1
βj , j = 1, . . . , p. (8)

1
7. (a) In linear regression, the coefficient of determination is the correlation of the squares of the re-
sponses and the fitted values.
(b) Variance of the least squares estimator of the intercept in a simple linear regression model increases
as the square of the coefficient of variation of the explanatory variable increases.
(c) In logistic regression, if the difference in deviances of full model and model under consideration
is small, then the latter is about as good as a fit as the full model.
(d) In case of Poisson loglinear model, the mean at xj +1 equals the mean at xj , multiplied by exp(βj )
where xj is the j th predictor and βj is the corresponding regression coefficient.
(e) The garrote estimates of the regression coefficients shrink the least squares estimates and some
are even forced to zero.
(f) Akaike’s Information Criterion (AIC) is based on prediction error. (2+2+2+2+1+1)

2
 MStat 1st Year
Analysis 1
Mid Semestral Exams

November 8, 2021

The question paper has 5 questions, all questions carry equal weight. State clearly all
results that you are using (you can use any theorem proved in class by stating it clearly,
for using any other result, you need to supply a proof of it).

1. Show that given any 2 real numbers x and y, with x < y, there always exists a
irrational number z such that x < z < y.

2. Suppose {xn } is a bounded sequence. Show that {xn } is not convergent if and only if
there exist two different proper subsequences of {xn } which converge to two different
limits.
√ √
3. Suppose 0 < a < b for real numbers a and b. Then show that a < b.

4. Suppose an is a sequence converging to a and bn is a sequence converging to b. Then

show that if an ≤ bn ultimately then a ≤ b. What if an ≤ bn frequently, then will
the result still hold? Explain.

5. Suppose an and bn are increasing sequences converging to a and b respectively.

Consider the sequence cn = max{an , bn }. Show that cn is an increasing sequence
converging to c = max{a, b}.

1
 Probability Theory
Midterm Examination
11 November 2021
Time: 2 hours Total Marks: 30

1. Suppose that the characteristic function φX (t) of a random varaible X is

real valued. Show that X is a symmetric random variable. [5]

2. Let X and Y be i.i.d. random variables which are uniformly distributed

on the interval (−2, 1).

(a) Find the density of |Y |. [3]

(b) Use the convolution formula to find the density of |X| − |Y |. [7]

3. (a) Let X be a random variable having a Poisson distribution with pa-

rameter λ > 0. Find the charachteristic function of X. [3]
(b) For every n ≥ 1, let Yn have a Poisson distribution with parameter
n. Show that
√
lim P (Yn ≤ n + x n) = Φ(x) ∀x∈R
n→∞

where Φ denotes the cumulative distribution function of a stan-

dard normal random variable. [7]

4. Let F and G be two distribution functions. Define H by

H(x) = F (x)G(x) ∀ x ∈ R.

Show that H is also a distribution function. [5]

 Indian Statistical Institute, Delhi Center
Master of Statistics (NB-stream)
Linear Algebra and Linear Models
Midterm Examination

10 November 2021, 10:00am – 12:15pm Total points: 25

State any results you use (you may be penalized otherwise). You may assume any results
proven in class, as long as you state them clearly.

Instructions

1. Please follow the general instructions given to you earlier. In particular, keep your video on
throughout the examination, and ensure that both your face and your answerscript are visible.

2. This is an open note and open book examination. However, no electronic devices (including
laptops and phones) may be used during the examination.

3. You may use the first ten minutes (i.e., 10:00AM - 10:10AM) to copy the questions. Beyond
that time, you may not use any electronic device to access the question paper.

4. For any questions or clarifications, contact the instructor through phone or WhatsApp at +91-
9810660673 instead of asking on zoom.

5. You must upload your scanned answerscripts to moodle by 12:15PM, beyond which you will not
be able to upload. Please plan accordingly.

6. In case there are any technical difficulties in accessing moodle, or if you are not sure whether your
upload has been successful, you may email the scanned answerscript to <deepayan@isid.ac.in>.

7. Retain the original answerscript with you till the end of the semester.
Question 1: Let A be a n × n real-valued matrix. Show that if tr(AB) = 0 for any n × n matrix [4]
B, then A must be the zero matrix.

Question 2: Let A be a m × n real-valued matrix. Show that < x, y >= x 0 A 0 Ay is an inner [4]
n
product on R if and only if A has rank n.

Question 3: Let W be a finite subset of a vector space V and S = span(W ). Let x ∈ S. If x can [4]
be expressed uniquely as a linear combination of elements of W , then show that W is a basis for S.

Question 4: Let V = {(x1 , x2 , . . . ) : xi ∈ R} be the vector space of all infinite real-valued sequences [4]
under elementwise vector addition and elementwise scalar-vector multiplication. Consider the set
B = {ei , i = 1, 2, . . . }, where ei ∈ V has all elements zero, except the i-th element which is 1. Does
B form a basis for V ? Justify your answer.

Question 5: The angles θ1 , θ2 , and θ3 of a triangular field (so θ1 + θ2 + θ3 = 180◦ ) were measured in [4]
an aerial survey and the observations y1 , y2 , and y3 were obtained as their respective measurements.
Assume that yi -s are independent and yi ∼ N (θi , σ 2 ). Is this a linear model? If yes, what are y, X,
and β?

Question 6: Suppose you have bivariate data (xi , yi ), i = 1, 2, . . . , n. Which of the following models [5]
for the data are linear models? Justify. Here α, β, γ, etc., are unknown parameters.

1. E(yi ) = 5 + βxi

2. E(yi ) = α + βxi + γx2i

3. E(yi ) = α + βxi + γ sin(xi )

4. E(yi ) = α + βxi + sin(γxi )


0 if xi < 0
5. E(yi ) =
βx
iif x ≥ 0
i

6. E(yi ) = αj + βj xi , where j = [xi ].

Good luck!
 Indian Statistical Institute, Delhi Centre
Master of Statistics (NB-stream)
Regression Techniques
Midterm Examination

November 9, 2021 Total Marks: 30

Time: 10:00 A.M. - 12:10 P.M.

Note:

• Two hours ten minutes includes time to scan and upload.

• This is a closed book, closed notes exam.

• Respect the sanctity of the exam.

• Marks will be deducted for writing irrelevant things.

• Clearly state any result you use and properly define all the notation.

• The filename should be of the form RT-YOURNAME-midterm.pdf where

YOURNAME should be replaced by your name in capital letters without space.

(1) Consider the full rank multiple linear regression model y = Xβ +  with intercept term
and the method used to estimate the unknown parameters is the least squares. Indicate
whether the following statements are true or false with justification. No credit will be
given for answers lacking justification.
(a) Average of hat matrix diagonals is p.
(b) The vector of fitted values y b is always orthogonal to the vector of residuals b.
(c) In linear regression, an underfitted model always overestimates the regression pa-
rameters.
(d) Variance of the least squares estimator of the intercept in a simple linear regression
model increases as the square of the coefficient of variation of the explanatory variable
increases.
(e) In scaled and centered model, the variances of the least squares estimator of the
regression coefficients are always less than the error variance.
(1 + 1 + 2 + 2 + 1 12 )
1
2

(2) For a multiple regression model, yi = βo + β1 xi,1 + · · · + βp−1 xi,p−1 + i , i = 1, . . . , n, where

i are independent N (0, σ 2 ).
(a) The aim is to test whether the regression on expalnatory variables is significant.
Your friend says that the test statistic can be written as
1 − R2 n − p − 1
,
R2 p
where R is the multiple correlation coefficient. Is it correct? Work out by stating
all the assumptions required to test this hypothesis and state the distribution of the
test statistic.
(b) Prove that the F -statistic for testing the hypothesis H : βq = βq+1 = · · · = βp−1 = 0
(0 < q ≤ p − 1) is unchanged if a constant c is substracted from each Yi .
(4+4)
(3) Let Yi = βxi +ui , xi > 0 (i = 1, 2, ..., n) and the dispersion matrix of u = (u1 , u2 , . . . , un )>
is  
1 ρ ρ2 · · · ρn−1
 ρ 1 ρ · · · ρn−2 
 
.
 . .. .. .. .. 
 .
 . . . . . 
ρn−1 ρn−2 ρn−3 · · · 1
If βb is the ordinary least squares estimator of β, prove that var[β]
b is inflated when ρ > 0.
(5)
(4) Consider fitting a full rank multiple regression model y = Xβ +  by least squares
method. Let hi be the i-th diagonal element of the hat matrix H. Suppose you estimate
β by deleting the i-th observation. Then write down the i-th deleted residual ei(−i) =
yi − xTi β(i)
b in terms of the i-th residual ei from the full model. Here xi is the i-th row
of X ( as a column vector), β(i)
b is the LSE of β obtained by excluding i-th observation.
(5)
(5) (a) There are two possible models, M1 and M2 , fitted for a given dataset. The estimated
2 2 2 2
error variances are σbM 1
and σ
bM 2
. You observe that σ
bM 1
<σ
bM 2
. If the criterion for
2
model selection is the adjusted R (R: multiple correlation coefficient), which model
do you consider? Why?
(b) In analyzing a given dataset using linear regression, suppose that the j th hat matrix
diagonal is 0.98. Then the j th predicted value of the response largely depends on the
j th response - Explain. What can you say about the location of the j th data point?
(2 21 + 3)
 Statistical Inference I, MStat I
First Semester, 2021

Mid-Semester Examination
Date: 12/11/2021
Time: 2 hours
Total Points: 30

• READ THE FOLLOWING INSTRUCTIONS CAREFULLY.

• All questions will be graded. Read questions carefully.

• Write your NAME and ROLL NUMBER on top of your answer sheet.

• Write your answers neatly on a plain white sheet, with no background lines/markings. Answer
each question on a fresh page with clear numbering.

• Scan and upload your answers in Moodle with file name: rollno-firstname-stinf-mid21.pdf.
Your answer file should be in pdf format. I have created a dummy midterm assignment for
this purpose.

• Each question has partial markings for a correct approach.

• Do NOT copy! You will get a zero, without any consideration.

• You get maximum 10 minutes beyond the 2hr time for uploading answer sheets.

1
 Questions

1. A random vector X = (X1 , X2 , X3 )0 takes values in the sample space X = {−1, 0, 1}3 . Also,
X has a p.m.f. which can be expressed in the canonical one-parameter exponential family
form, with

T (x) = median{x1 , x2 , x3 } and h(x) = 1, for all x = (x1 , x2 , x3 )0 ∈ X .

(a) Find the natural parameter space for this family of p.m.f.’s and write the p.m.f. of X in
the canonical exponential family form. [8 pts]

(b) Find the expectation of T (X), in terms of the natural canonical parameter. [2 pts]

2. Assume, {X1 , . . . , Xn } is a sequence of independent Poisson random variables, with E(Xi ) =

λi (> 0), for each i = 1, . . . , n. Also assume, {t1 , . . . , tn } is a sequence of fixed (real-valued)
constants such that they are related to λi , in the following manner: log (λi ) = α+βti , for each
i = 1, . . . , n, where α, β ∈ R, are unknown parameters. Based on this sample, find minimal
sufficient statistics for the underlying family of joint distributions. [7 pts]

3. State if the following statements are true or false. Provide your answer with clear
justification.

(a) Let {Z1 , . . . , Zn } be i.i.d. standard Normal random variables. Then,

Z1
kZk and will be dependent,
kZk
p
where kZk = Z12 + . . . + Zn2 . [8 pts]

(b) Assume {X1 , . . . , Xn } are i.i.d. random variables with p.d.f.

2x
f (x : θ) = · 1(0 < x < θ), for all θ > 0.
θ2

Then, T = X(n) will be complete and sufficient for this family. [5 pts]