ECE 302 Spring 2012. Practice problems: Counting, discrete random variables, expectation, variance.

Ilya Pollak
These problems have been constructed over many years, using many dierent sources. If you nd that some problem is incorrectly attributed, please let me know at ipollak@ecn.purdue.edu. Suggested reading: Sections 1.62.4 in the recommended text [3]. Equivalently, Sections 2.3, 2.6 (except for the multinomial law), 3.1, 3.4 (discrete random variables only), 3.5, and 3.6 in the Leon-Garcia text [7]. Problem 1. Gambling in Florence. (Ilya Pollak, after the discussion at the beginning of Chapter 4 of [8].) Consider the game of independently throwing three fair six-sided dice. There are six ways in which the three resulting numbers can sum up to nine, and also six ways in which they can sum up to 10: 9 = 1+2+6 = 1+3+5 = 1+4+4 = 2+3+4= 2+2+5 = 3+3+3 10 = 1 + 3 + 6 = 1 + 4 + 5 = 2 + 2 + 6 = 2 + 3 + 5 = 2 + 4 + 4 = 3 + 3 + 4. This seems to suggest that the chances of throwing nine and 10 should be equal. Yet, a certain gambler in Florence in the early XVII century (most likely, the Grand Duke of Tuscany Cosimo II de Medici) noticed that in practice it is more likely to throw 10 than nine. Show that the probability to get a total of 10 is, in fact, larger than the probability to get a total of nine. (This problem was solved for the duke by Galileo Galilei.) Problem 2. Benefits of independent retesting. (Ilya Pollak, after Chapter 4 of [1].) Suppose your company is trying to decide whether or not to buy a certain software package. The vendor has given you a copy of the software for testing. You ask two people, Pat and Sam, to independently test the software. After the testing is done, Pat reports 20 aws and Sam 30. (For example, a aw could be either an outright bug, or the lack of a feature that your company needs.) Estimate the total number of aws in the software (a) if the total number of aws they detect is 40 (i.e., their lists of aws have 10 aws in common); (b) if the total number of aws they detect is 49 (i.e., their lists of aws have one aw in common). Assume that Pat and Sams detections of aws can each be modeled as a sequence of f independent Bernoulli trials where f is the actual number of aws, with probabilities of success (i.e., detection of a aw) p and s, respectively. Also assume that neither Pat nor Sam produce any false alarmsi.e., that they do not report a aw where there is none. Estimate p in terms of f as the relative frequency of the event Pat detects a aw in the f Bernoulli trials. Similarly estimate s and ps, then solve for f . Problem 3. How to construct multiple-choice tests. (Ilya Pollak). Suppose 200 students take an exam consisting of M multiple-choice questions, each with N possible 1

answers. All students decide to answer questions randomly and independently, i.e., each students probability to answer any question correctly is 1/N , and all the answers are independent. Each question is worth one point, and a score of M 3 or more is an A. What is the probability that at least one student will get an A? Express your answer in terms of M and N . Evaluate this expression for two cases: 20 true-false questions (i.e., M = 20 and N = 2), and 20 questions with 10 options each (i.e., M = 20 and N = 10.) Problem 4. How to rate mutual funds? (Ilya Pollak). There are over 8000 mutual funds in the United States. Suppose we measure the performance of mutual funds by whether they overperform or underperform some market average (say, the S&P 500 index). Let us consider the possibility that all the funds are independent, and have equal chances to be above or below the market in any given year. Assume that the total number of funds is exactly 8000. Based on these assumptions, (a) what is the probability that at least one mutual fund out of 8000 will perform above the market average for 10 consecutive years, starting with the next year? (b) what is the probability that at least one fund will perform above average for 15 consecutive years, starting with the next year? (c) what is the probability that there will be at least one mutual fund that outperforms the market for each of 15 consecutive calendar years during some 15-year interval during the next 50 years? To answer this question, let p(s, n) be the probability that one mutual fund will achieve a streak of s years during the next n years, and go through the following steps. (i) Argue that the probability that we are looking for is 1 (1 p(15, 50))8000 .

(ii) Argue that p(s, n) = 0 for n < s and that p(s, n) = (1/2)s for s = n.

(iii) Argue that, for n > s, p(s, n) = p(s, n 1) + (1 p(s, n s 1)) (1/2)s+1 . Hint. Dene the following events: A = a streak of length s happens during years 1 through n B = a streak of length s happens during years 1 through n 1 C = a streak of length s happens in years n s + 1, n s + 2, . . . , n D = year n s is a below-average year E = a streak of length s does not happen at any time during years 1 through n s 1 Argue that events B and C D E form a partition of the event A. Further argue that the events C , D , and E are independent. Then argue that P(A) = p(s, n), P(B ) = p(s, n 1), P(C ) = (1/2)s , P(D ) = 1/2, and P(E ) = 1 p(s, n s 1). (iv) Use the following Matlab code to compute the nal answer: n=50; s=15; probs = zeros(1,n); probs(s) = (1/2)^s; probs(s+1) = (1/2)^s + (1/2)^(s+1); for k=s+2:n probs(k) = probs(k-1) + (1-probs(k-s-1))*(1/2)^(s+1); end; 2

% After this calculation, probs(50) is p(15,50). answer = 1-(1-probs(50))^8000 [If you are not interested in nance feel free to skip the next two paragraphs. They are not needed to solve the problem.] This problem is inspired by the notorious 15-year streak of Legg Mason Value Trust mutual fund (LMVTX on Yahoo Finance) which has been discussed in many articles and books, e.g., [8, 12]. It beat the S&P 500 index (GSPC on Yahoo Finance) for each calendar year from 1991 to 2005. Its cumulative return from 1991 to 2005 was about 480% compared to about 278% for S&P 500. Since then, however, LMVTX lost 41% while S&P 500 gained 7.7%, as of close on Friday, February 3, 2012. The cumulative result is that a dollar invested in LMVTX just before the beginning of 1991 would become only $3.42 on 2/3/12, whereas a dollar invested in S&P 500 would become $4.07. This is something to keep in mind when you look for a mutual fund to invest in. The fact that someone has consistently beaten the market over the last few years just might be the result of a random uctuation, simply due to the fact that there are thousands of mutual funds out there. If in fact the past performance was solely due to random chance then it is irrelevant in trying to predict the future performance. (In fact, even if the past good performance was due to the great skills of the fund manager rather than luck, there are still many compelling reasons why the future may still be unpredictable, such as (a) the particular market anomaly that the fund manager has been exploiting may be discovered by many other market participants and therefore cease to be protable; (b) the fund manager may get another job and leave, and be replaced by a less-competent person.) Problem 5. On the importance of default dependencies in the valuation of mortgagebacked securities. (Ilya Pollak). [If you are not interested in nance feel free to skip this paragraph. It is not needed to solve the problem.] A typical mortgage-backed security (MBS) pools many mortgages together. The security has several tranches with dierent levels of seniority. The most junior tranche, usually called the equity tranche, is the riskiest and has the highest rate of return, in order to compensate the holders of this tranche for the risk they are taking. The senior tranche is the least risky but has the smallest rate of return. The interest payments are made to the senior tranche rst. If there is anything left over after the promised rate of return for the senior tranche is satised, the next tranche receives its interest payments, etc. If the underlying mortgages default and cause the loss of capital, the equity tranche usually suers losses rst: the capital of the senior tranche is untouched until all the tranches below it are completely wiped out. If you are interested in more specics on how mortgage-backed securities work, please ask questions through the forum or during the oce hours or the help session. You do not actually need to know anything about mortgage-backed securities in order to solve this problem. This problem explores the importance of correctly accounting for the dependencies among the underlying mortgages in order to be able to properly value the MBS that they comprise. Suppose an MBS of duration one year is based on 10 1-year underlying mortgages, $200,000 each. Assume that, for each mortgage, one of two things can happen: 3

 Either the mortgage is paid back in full with 5% interest at the end of the year, or it defaults with 100% loss. For each mortgage, the probability of default is 0.1. Suppose that the MBS consists of two tranches: the senior tranche which has 60% of the capital and the equity tranche which has 40% of the capital. The senior tranche earns 3% interest, the equity tranche earns 8%. (a) What is the probability that the senior tranche will lose some capital, if the defaults of individual mortgages are independent? Note that the event that the senior tranche will lose some capital is the event that ve or more out of the ten mortgages will default. (b) What is the probability that the senior tranche will lose some capital if the defaults of individual mortgages are perfectly dependent (i.e., either all ten default or none of them defaults)? (c) What is the probability that the senior tranche is completely wiped out if the defaults are independent? Note that this happens when all ten mortgages default. (d) What is the probability that the senior tranche is completely wiped out if the defaults are perfectly dependent? Problem 6. (Ilya Pollak, after Gilbert and Mosteller [6].) Suppose you are interviewing people for the position of ECE 302 instructor. You have four candidates who are interviewed consecutively: 1, 2, 3, and 4. If you interview any two of them, you will be able to determine which one of the two is better. The problem is, you are required to tell the decision to each candidate at the end of his/her interview: you either reject the candidate and go on to the next candidate, or you hire the candidate and reject all the remaining candidates without interviewing them. If you reject the rst three candidates, you must hire the fourth. Before the interviews start, you have no information about the relative strength of the candidates, and so you assume that the best candidate is equally likely to be candidate 1, 2, 3 or 4; the second-best candidate is also equally likely to be candidate 1, 2, 3, or 4, etc. (a) Suppose your strategy is to hire the candidate 1, regardless of what happens during his/her interview. What is the probability that you will hire the best candidate? (b) Suppose you interview and reject candidate 1, and then continue the interviews and hire the rst candidate you see who is better than candidate 1. I.e., if candidate 2 is better than candidate 1, you hire candidate 2; if candidate 2 is worse than candidate 1 then you interview candidate 3. If you interview candidate 3, then: if he is better than candidate 1, you hire candidate 3, otherwise you hire candidate 4. What is the probability that you will hire the best candidate? (c) Suppose you interview and reject rst two candidates, and then hire candidate 3 if he is better than both candidates 1 and 2, and hire candidate 4 otherwise. What is the probability that you will hire the best candidate? Problem 7. (After Drake [4], Problem 1.23.) In the communication network of Fig. 1, link failures are independent, and each link has a probability 4

A
e

d f g

Figure 1: Communication network for Problem 7. of failure of p. Consider the physical situation before you write anything. A can communicate with B as long as they are connected by at least one path which contains only in-service links. (Note: a, b, c, d, e, f , and g, are just the names of the links, NOT probabilities.) (a) Given that exactly four links have failed, determine the conditional probability that A can communicate with B . (b) Given that exactly four links have failed, determine the probability that either g or f (but not both) is still operating properly. (c) Given that a, b, and c have failed (but no information about the condition of other links), determine the probability that A can communicate with B . Problem 8. (Bertsekas-Tsitsiklis supplementary problems to Chapter 1, Problem 19 [2].) An internet access provider (IAP) owns two servers. Each server has a 50% chance of being down independently of the other. Fortunately, only one server is necessary to allow the IAP to provide service to its customers, i.e., only one server is needed to keep the IAPs system up. Suppose a customer tries to access the internet on four dierent occasions, which are suciently spaced apart in time, so that we may assume that the states of the system corresponding to these four occasions are independent. What is the probability that the customer will only be able to access the internet on three out of the four occasions? Problem 9. (Rozanov [11] Chapter 1, Example 5.) A subway train made up of n cars is boarded by r passengers (r n), each passenger choosing a car at random (using the discrete uniform probability law) and independently of other passengers. What is the probability of the passengers all ending up in dierent cars? Problem 10. (Rozanov [11] Chapter 1, Problem 2.) A wooden cube with painted faces is sawed up into 1000 little cubes, all of the same size. The little cubes are then mixed up, and one is chosen at random. What is the probability of its having exactly two painted faces? Problem 11. (Rozanov [11] Chapter 1, Problem 6.) Suppose n people sit down at random and independently of each other in an auditorium containing n + k seats. What is the probability that m seats specied in advance (m n) will be occupied? 5

Problem 12. (Ilya Pollak and Bin Ni.) In blackjack, the objective is to get as close as possible to 21 without going over 21. The cards are valued as follows: An Ace can count as either 1 or 11, whichever makes the best hand. The cards from 2 through 10 are valued as indicated. Jacks, Queens, and Kings are all valued at 10. Suppose we use a 52-card deck. (a) We draw two cards at random from a full deck, so that each pair of cards is equally likely to be drawn. What is the probability to get a blackjack (i.e., a total of 21)? (b) Suppose we started with a full deck and drew the 9 of clubs and the 9 of diamonds. What is the conditional probability that, if we draw another card, we bust (i.e., go over 21)? What is the conditional probability that this third card we draw will be a 3? Assume that, after the 9 of clubs and the 9 of diamonds are drawn, each of the remaining 50 cards is equally likely to be drawn. (c) Suppose we started with a full deck and drew the 7 of clubs and the 8 of spades. What is the conditional probability that the next card leads to a bust? Assume that, after the 7 of clubs and the 8 of spades are drawn, each of the remaining 50 cards is equally likely to be drawn. (d) Suppose we started with a full deck and drew the 5 of hearts and the 10 of clubs. What is the conditional probability that the next card leads to a bust? Assume that, after the 5 of hearts and the 10 of clubs are drawn, each of the remaining 50 cards is equally likely to be drawn. Problem 13. (Ilya Pollak.) An important milestone in the development of the theory of probability was the exchange of letters between Blaise Pascal (1623-1662) and Pierre Fermat (1601-1665), two French mathematicians. One motivation for these letters were problems proposed to Pascal by his friend Chevalier de M er e, in particular the problem of points, described in Problem 38 of Chapter 1 in [3]. Another of de M er es problems is Problem 49 in Chapter 1 [3], and here is yet another one ( [5], page 56). De M er e bet that at least one 6 would appear during a total of four independent rolls of a fair six-sided die. From past experience, he knew that he was more successful than not with this game of chance. Tired of his approach, he decided to change the game. He bet that a double 6 would appear at least once during twenty-four independent rolls of two dice. Soon he realized that his old approach to the game resulted in more money. Please help him gure out why his new approach was not as protable. (Hint. First, count the total number of outcomes of the rst game. How many of them result in no 6s?) Problem 14. (Mosteller [9], Problem 1(a).) A drawer contains red socks and black socks. When two socks are drawn at random, the probability

that both are red is 1/2. What is the smallest possible number of socks in the drawer? Problem 15. (Drake [4], Problem 1.28.) Companies A, B, C, D and E each send three delegates to a conference. A committee of four delegates, selected by a lot, is formed. Determine the probability that: (a) Company A is not represented on the committee. (b) Company A has exactly one representative on the committee. (c) Neither company A nor company E is represented on the committee. Problem 16.(Drake [4], Problem 1.30.) The Jones family household includes Mr. and Mrs. Jones, four children, two cats and three dogs. Every six hours there is a Jones family stroll. The rules for a Jones family stroll are: Exactly ve beings (people + dogs + cats) go on each stroll. Each stroll must include at least one parent and at least one pet. There can never be a dog and a cat on the same stroll unless both parents go. All acceptable stroll groupings are equally likely. Given that exactly one parent went on the 6pm stroll, what is the conditional probability that Rover, the oldest dog, also went? Problem 17. (Ilya Pollak and Bin Ni.) A poker hand consists of ve cards. The dierent possible hands, from the lowest to the highest, are: One pair: two cards of dierent suits which have the same denominations, e.g., two Queens or two 10s etc. Two pair: e.g., two 5s and two 10s. Three of a kind: e.g., three 10s. Straight: ve consecutive cards, which are not all of the same suit. An Ace can play either high or low, as in A-2-3-4-5 or 10-J-Q-K-A, but not bothi.e., for example, Q-K-A-2-3 is not a straight. Flush: ve cards of the same suit which are not all consecutive. Full house: three of a kind and one pair. Four of a kind. Straight ush: ve consecutive cards in the same suit. Suppose we draw ve cards at random from a deck of 52 cards. (a) What is the probability to get four of a kind? 7

(b) What is the probability to get a full house? (c) What is the probability to get three of a kind? (Hint. Keep in mind that a hand like 10-10-10K-K is not three of a kind: it is a full house. Similarly, 10-10-10-10-K is not three of a kind, but four of a kind.) (d) What is the probability to get a straight ush? (e) What is the probability to get one pair? (Hint. First, note that a hand like 10-10-2-2-3 is not a pair but two pair, 10-10-2-2-2 is a full house, 10-10-10-2-3 is three of a kind, and 10-10-10-10-2 is four of a kind. Conclude therefore that, on order for a hand to be one pair, the pair and each of the three remaining cards must have four distinct denominations. Count the number of ways to choose these four denominations out of 13 possible denominations; then count the number of ways to choose a pair in one of the denominations and a single card in each of the remaining three.) (f) What is the probability to get two pair? (g) What is the probability to get a straight? (Hint. Keep in mind that, for example, 2-3-4-5-6 of clubs is not a straight but a straight ush.) (h) What is the probability to get a ush? (Hint. Again, keep in mind that, for example, 2-3-4-5-6 of clubs is not a ush but a straight ush.) Problem 18. (Deepan Palguna and Ilya Pollak.) The letters of the word PROBABILITY are shued such that each distinct permutation is equally likely. An n-letter sub-word is dened as a contiguous sequence of n letters, read left to right. What is the probability that the seven-letter sub-word ABILITY occurs in such a shuing? Problem 19. (After Feller [5], Chapter II, Section 5, page 38.) Suppose N and K are two positive integers such that N > K . Suppose N identical balls are distributed among K dierent boxes, and let nk be the number of balls in the k-th box, for k = 1, . . . , K . Let (n1 , . . . , nK ) be the outcome of this experiment. (a) Consider the set S of all outcomes such that each box has at least one ball (in other words, such that nk 1 for k = 1, . . . , K ). How many outcomes are in this set S ? Your answer will be an expression involving K and N . (b) Now consider the set T of all possible outcomes, including all outcomes in which some of the boxes are empty. How many outcomes are in this set T ? Problem 20. (Drake [4], Problem 2.01.) The geometric PMF for discrete random variable X is dened to be pX (k) = P(X = k) = C (1 P )k1 if k = 1, 2, 3, . . . 0 for all other values of k,

where P is a positive real number which is less than 1. 8

(a) Determine the value of C . (b) Let N be a positive integer. Determine the probability that an experimental value of X will be greater than N . (c) Given that an experimental value of random variable X is greater than integer N , what is the conditional probability that it is also larger than 2N ? (d) What is the probability that an experimental value of X is equal to an integer multiple of 3? Problem 21. (After Problem 16 from Chapter 2 of [3].) Let X be a random variable with PMF pX (x) = (a) Find C . (b) Find E [X ]. (c) Find the PMF of the random variable Z = (X E [X ])2 . (d) Using Part (c), compute the variance of X . (e) Compute the variance of X using the identity var(X ) =
x

0,

| x| C ,

if x = 4, 3, 2, 1, 0, 1, 2, 3, 4 otherwise.

(x E [X ])2 pX (x).

Problem 22. (Mosteller [9], Problem 4.) (a) On the average, how many independent throws of a fair six-sided die must be performed until one gets a 6? (b) On the average, how many independent tosses of a fair coin must be performed until one gets an H? Problem 23. Suppose that X is a Bernoulli random variable, with pX (0) = 1 p and pX (1) = p. Suppose further that E [X ] = 3var(X ). Find p. Problem 24. (Ross [10], Chapter 4, Example 3d.) A class of 120 students are driven in 3 buses to a concert. There are 36 students in one of the buses, 40 in another, and 44 in the third bus. When the buses arrive, one of the 120 students is chosen, according to a discrete uniform probability law. Let X denote the number of students on the bus of that randomly chosen student. Find E [X ]. Problem 25. (After Ross [10], Chapter 4, Example 6g.) In a U.S. presidential election the candidate who gains the maximum number of votes in a state is awarded the total number of electoral college votes allocated to that state. The number of electoral college votes of a given state is roughly proportional to the population of the state. Let us determine the average power in a close presidential election of a citizen in a state with n voters and c electoral 9

votes, where by average power of a voter we mean the expected number of electoral votes he/she will aect. Assume that there are two presidential candidates in the election. Assume that there are n = 2k + 1 voters in a state: Joe and 2k other voters. The objective of this problem is to calculate Joes average power. (a) Assuming that each of the other 2k voters acts independently and is equally likely to vote for either candidate, nd the probability that Joes vote will be the deciding vote. In other words, nd the probability that the 2k remaining votes are split evenly between the two candidates. (Hint. The number of votes for one of the candidates is a binomial random variable with parameter p = 1/2.) (b) Simplify the expression obtained in Part (a) using Stirlings approximation: m! mm+1/2 em 2 (this approximation is valid for any large integer m.) (c) If Joes vote is the deciding vote in the state, then it will aect c electoral votes; otherwise, it will not aect any electoral votes. Find Joes average poweri.e., the expected number of electoral votes that his vote will aect. You can use n/2 k. Calculate an approximate numerical answer for c = 25 (the number of Floridas electoral votes in 2000) and n = 6, 000, 000 (approximate number of people who voted in the presidential elections in Florida in 2000). Problem 26. (Drake [4], Problem 2.02.) The probability that any particular bulb will burn out during its K -th month of use is given by the PMF for K, 1 4 k 1 , k = 1, 2, 3, . . . . pK (k) = 5 5 Four bulbs are life-tested simultaneously. Their lifetimes are independent. Determine the probability that (a) None of the four bulbs fails during its rst month of use. (b) Exactly two bulbs have failed by the end of the third month. (c) Exactly one bulb fails during each of the rst three months. (d) Exactly one bulb has failed by the end of the second month, and exactly two bulbs are still working at the start of the fth month. Problem 27. (Rozanov [11], Chapter 4, Problem 12.) Balls are drawn from an urn containing w white balls and b black balls until a white ball appears. Find the mean value m and variance 2 of the number of black balls drawn, assuming each ball is placed back into the urn after being drawn. (In other words, for each attempt we have w white balls and b black balls in the urn.) Assume that all draws are independent. Problem 28. (After Mosteller [9], Problem 7.) American roulette wheels have 38 equally likely numbers. If the players number comes up, he is paid 35 times his stake and gets his original stake back; otherwise he loses his stake. 10

Al always bets a dollar on the number 13 at roulette. Bob bets Al $20 that Al will be behind at the end of 36 roulette plays. In other words, if Al is losing to the casino after 36 roulette plays, he must pay $20 to Bob; if Als prot against the casino is zero or positive after 36 roulette plays, he gets $20 from Bob. Assume that all roulette plays are independent. (a) Calculate Als expected gain for the 36 roulette plays only, without Bobs $20 bet. (If Al is expected to lose money, his expected gain will be a negative number.) (b) Calculate the probability that Al is behind at the end of 36 roulette plays. Here and for the remaining parts of this problem, assume that all the roulette plays are independent. (Hint. Can he be behind if he won at least once in the 36 plays?) (c) Calculate Als expected gain in his bet against Bob. (d) Using the linearity of expectation and your results from Parts (a) and (c), calculate Als overall expected gain from roulette and the bet.

References
[1] J.D. Barrow. One Hundred Essential Things You Didnt Know You Didnt Know: Math Explains Your World, W.W. Norton & Company, 2009. [2] D.P. Bertsekas and J.N. Tsitsiklis. Introduction to Probability: Supplementary Problems, Athena Scientic, Belmont, Massachusetts, 2005, http://www.athenasc.com/prob-supp.html. [3] D.P. Bertsekas and J.N. Tsitsiklis. Introduction to Probability, 2nd Edition, Athena Scientic, Belmont, Massachusetts, 2008. [4] A. Drake. Fundamentals of Applied Probability Theory, McGraw-Hill, Inc., 1988. [5] W. Feller. An Introduction to Probability Theory and Its Applications, Volume I, 3rd Edition, John Wiley & Sons, 1968. [6] J.P. Gilbert and F. Mosteller. Recognizing the Maximum of a Sequence. Journal of the American Statistical Association, 61(313):3573, March, 1966. [7] A. Leon-Garcia. Probability and Random Processes for Electrical Engineering, 2nd Edition, Addison Wesley Longman, 1994. [8] L. Mlodinow. The Drunkards Walk: How Randomness Rules Our Lives, Vintage Books, 2009. [9] F. Mosteller. Fifty Challenging Problems in Probability with Solutions, Dover, 1965. [10] S. Ross. A First Course in Probability, 6th Edition, Prentice-Hall, 2002. [11] Y.A. Rozanov. Probability Theory: A Concise Course, Revised English Edition, translated and edited by R.A. Silverman. Dover, 1977. [12] J. Zweig. The Long Climb and Steep Descent of Legg Masons Top Stock Picker. The Wall Street Journal, November 18, 2011. online.wsj.com/article/SB10001424052970204517204577044570430299472.html

11