1.5.6 Averages in involving multiple variables

Let’s go back to the coin game above where $1 if it lands heads and you have to pay $1 if it lands tails. You throw the coin twice. The first thing to consider is a probability distribution involving two variables. We’ll choose the first variable $x$ to be the amount won (or lost) in the first flip. The second variable $T$ is the total amount won for both flips. We want to know for example, the probability that you make $1 on the first flip but make $ 0 overall. Generalizing, we want to know $P(x,T)$, the probability of making $x$ $’s on the first flip and $T$ $’s overall.

Let’s determine $P(x,T)$. There are 3 possible values of $T$ -2, 0, and 2. There are two possible values of $x$, 1 and -1. There are four possible outcomes for these two flips. So what’s $P(-1,-2)$? Well there’s only one way of getting that, you have to get tails on both flips. So $P(-1,-2)=1/4$. Similarly $P(1,2)=1/4$. How about $P(-1,2)$? Well you can’t lose on the first flip and make money on both flips, so $P(-1,2)=0$. Similarly $P(1,-2)=0$. How about $P(-1,0)$? There’s only one way of getting that, you’ve got to make $1 on the second flip, so $P(-1,0)=1/4$. Similarly $P(1,0)=1/4$. So we’ve determined the values of this two dimensional probability distribution.

P(x,T) P -2 0 2 -1 1/4 1/4 0 1 0 1/4 1/4

(1.22)

Now let’s ask for the average value of various quantities. First, what’s the average value of $x$? You already should know the answer, but how do you calculate it from here?

$$⟨x⟩=\sum _x\sum _{T}xP(x,T)$$

(1.23)

Here we’re summing over all possible values of $x$ and $T$. To do this sum, we can do the sum over $T$ first.

$$⟨x⟩=\sum _{x}x\sum _{T}P(x,T)$$

(1.24)

From the table above, when you do that sum you add up all the entries in each row. Both of these are $1/2$. So

$$⟨x⟩=\sum _{x}x\frac{1}{2}=(-1+1)\frac{1}{2}=0$$

(1.25)

as it should be.

Now let’s do a more tricky one. What’s $⟨xT⟩$?

$$⟨xT⟩=\sum _x\sum _{T}xTP(x,T)$$

(1.26)

We’ll do this term by term

x T P(x,T) P -2 0 2 -1 (-1)(-2)/4 (-1)0/4 (-1)(2) 0 1 (1)(-2)0 (1)(0)/4 (1)(2)1/4

(1.27)

You see the only nonzero entries are on the upper left hand and lower right hand corners. So adding up all these terms, you get

$$⟨xT⟩=1.$$

(1.28)

Suppose you want to compute the average of something like height (h) $\times$ weight (w), for a population of chipmunks. This involves two separate variables. You might have the distribution $P(h,w)$ which is a probability density. $P(h,w)dhdw$ is the probability of finding a chipmunk of height between $h$ and $h+dh$, and weight between $w$ and $w+dw$. In this case we can then define the average as

$$⟨hw⟩=\int \int hwP(h,w)𝑑h𝑑w$$

(1.29)

This is for a continuous distribution For a discrete one, you’d replace the integrals by summation, as we saw in above in the one dimensional case.

Suppose that you have two variables that are independent. The case of height and weight we used above, is not a good example of this, because you might expect that a taller chipmunk would tend to weigh more. However we can think of something else, let’s take two blades of grass at opposite sides of a lawn. Call the length of the first blade $x$ and the second one $y$. You’d expect their lengths to be quite independent, That means $P(x,y)dxdy=(P(x)dx)(P(y)dy)$. Now we see that

	$⟨xy⟩={\displaystyle \int \int xyP(x,y)𝑑x𝑑y}={\displaystyle \int \int xyP(x)P(y)𝑑x𝑑y}$		(1.30)
	$={\displaystyle \int xP(x)𝑑x\int yP(y)𝑑y}=⟨x⟩⟨y⟩$

This says that for independent variables, the average of the product is the product of the average. This property makes independent variables very important conceptually in the study of probability.

Let’s see how we can apply it to examples of problems where the variables aren’t independent. Consider the example we went through above of the two dice in section 1.5.6. There we calculated $⟨xT⟩$: the average of the result on the first flip times the total. Note that these two variables are not independent. If the first coin lands tails, you pay $1. That will effect the total amount of money that you make.

The trick is to find a set of variables that are independent. In this case that’s not too hard. Call the result of the second flip $y$. The results for the first and second flip are independent. So now we express the total in terms of these independent variables: $T=x+y$. So

$$⟨xT⟩=⟨x(x+y)⟩=⟨x^2+xy⟩=⟨x^2⟩+⟨xy⟩$$

(1.31)

Now since $x$ can only take values $\pm 1$, $⟨x^2⟩=⟨1⟩=1$. In the second term we can use the independence of $x$ and $y$ to say $⟨xy⟩=⟨x⟩⟨y⟩=0$. So

$$⟨xT⟩=1$$

(1.32)

as we obtained before.

1. Consider what happens when you throw two dice. You can ask the question ”what’s the probability of getting a total of 8 when the first one is 5?. Or in more generality: what’s the probability of getting a total of $T$ when the first one is $m$?. The answer is a function of two variables, $T$ and $m$. You could call the result $P(m,T)$. How do you define averages in this case? For example, what’s the average value of $mT$.

You could determine $⟨mT⟩$ by throwing the dice repeatedly taking the number of the first die and multiplying by the total. What you get should be equivalent to

$$⟨mT⟩=\sum _m\sum _{T}mTP(m,T)$$

(1.33)

but that’s a sum over a lot of possibilities, all possible values of $m$ and $T$. Instead use independence to solve this problem in analogy to what we just did for the two dice.

Now we’re going to analyze an important example a couple of different ways: what’s the average value for a variable distributed according to the binomial distribution?

Where does the binomial distribution come from? Flipping a coin $n$ times and saying that the probability of a head is $p$, and a tail is $1-p$. You can assign a variable $x_i$ to the ith flip, saying that if it lands tails, $x_i=0$, you make nothing but if it lands heads, $x_i=1$, that is you make $1. So the sum of the variables $x_i$ is just the number of times it landed heads. Earlier we called that sum $X$, see eqn 1.14. We know the distribution P(X) is the binomial distribution, eqn. 1.9. So

	$⟨X⟩={\displaystyle \sum _{X=0}^n}XP(X)={\displaystyle \sum _{X=0}^n}X(\begin{array}{c}n\\ X\end{array}){(p)}^X{(1-p)}^{n-X}=$		(1.36)
	${\displaystyle \sum _{X=0}^n}X{\displaystyle \frac{n!}{(n-X)!X!}}{(p)}^X{(1-p)}^{n-X}$

You can do this sum using some differentiation trick, but this is not the best way to find out the average. Let’s think about going back to our earlier approach.

First consider just one trial, say $x_5$. You flip a coin. What’s the probability of it coming up heads? That’s just the definition of $p$. So what’s the average value of $x_5$? It’s $p$ too, if you want to see this more mathematically just start with the definition of an average eqn. 1.16. In this case you have its $0P(0)+1P(1)=0P(0)+1p=p$. This is how much you expect to make in one trial. Now we’re interested in the total amount you make after $n$ trials. Well as we reasoned before, you can just multiply the result for one toss by $n$, so the answer is $np$. Saying this in mathematical notation:

$$⟨X⟩=⟨\sum _{i=1}^n{x}_i⟩=\sum _{i=1}^n⟨x_i⟩=\sum _{i=1}^{n}p=np$$

(1.37)

This uses the fact that the average of the sum is the sum of the average as we discovered before. So now we know the mean for the binomial distribution. It’s just (number of trials) $\times$ (the mean for one trial).

Now some of you probably wish I’d done it the longer way using the calculus trick. Try it out if you’re so inclined, it’s not that bad. However it’s always best to try to come up with the most elegant and intuitive derivations of answers that you can. You understand the nature of the problem much better that way. Once you can do that you’ll have a shot and understanding and solving really hard problems, something that you won’t be able to do if you don’t follow this advice.