The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation, or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu,
OK.
So let us start.
All right.
So today we're starting a new unit in this class.
We have covered, so far, the basics of probability theory--
the main concepts and tools, as far as just probabilities
are concerned.
But if that was all that there is in this subject, the
subject would not be rich enough.
What makes probability theory a lot more interesting and
richer is that we can also talk about random variables,
which are ways of assigning numerical results to the
outcomes of an experiment.
So we're going to define what random variables are, and then
we're going to describe them using so-called probability
mass functions.
Basically some numerical values are more likely to
occur than other numerical values, and we capture this by
assigning probabilities to them the usual way.
And we represent these in a compact way using the
so-called probability mass functions.
We're going to see a couple of examples of random variables,
some of which we have already seen but with different
terminology.
And so far, it's going to be just a couple of definitions
and calculations of the type that you
already know how to do.
But then we're going to introduce the one new, big
concept of the day.
So up to here it's going to be mostly an exercise in notation
and definitions.
But then we got our big concept which is the concept
of the expected value of a random variable, which is some
kind of average value of the random variable.
And then we're going to also talk, very briefly, about
close distance of the expectation, which is the
concept of the variance of a random variable.
OK.
So what is a random variable?
It's an assignment of a numerical value to every
possible outcome of the experiment.
So here's the picture.
The sample space is this class, and we've got lots of
students in here.
This is our sample space, omega.
I'm interested in the height of a random student.
So I'm going to use a real line where I record height,
and let's say this is height in inches.
And the experiment happens, I pick a random student.
And I go and measure the height of that random student,
and that gives me a specific number.
So what's a good number in inches?
Let's say 60.
OK.
Or I pick another student, and that student has a height of
71 inches, and so on.
So this is the experiment.
These are the outcomes.
These are the numerical values of the random variable that we
call height.
OK.
So mathematically, what are we dealing with here?
We're basically dealing with a function from the sample space
into the real numbers.
That function takes as argument, an outcome of the
experiment, that is a typical student, and produces the
value of that function, which is the height of that
particular student.
So we think of an abstract object that we denote by a
capital H, which is the random variable called height.
And that random variable is essentially this particular
function that we talked about here.
OK.
So there's a distinction that we're making here--
H is height in the abstract.
It's the function.
These numbers here are particular numerical values
that this function takes when you choose one particular
outcome of the experiment.
Now, when you have a single probability experiment, you
can have multiple random variables.
So perhaps, instead of just height, I'm also interested in
the weight of a typical student.
And so when the experiment happens, I
pick that random student--
this is the height of the student.
But that student would also have a weight, and I could
record it here.
And similarly, every student is going to have their own
particular weight.
So the weight function is a different function from the
sample space to the real numbers, and it's a different
random variable.
So the point I'm making here is that a single probabilistic
experiment may involve several interesting random variables.
I may be interested in the height of a random student or
the weight of the random student.
These are different random variables
that could be of interest.
I can also do other things.
Suppose I define an object such as H bar, which is 2.58.
What does that correspond to?
Well, this is the height in centimeters.
Now, H bar is a function of H itself, but if you were to
draw the picture, the picture would go this way.
60 gets mapped to 150, 71 gets mapped to, oh, that's
too hard for me.
OK, gets mapped to something, and so on.
So H bar is also a random variable.
Why?
Once I pick a particular student, that particular
outcome determines completely the numerical value of H bar,
which is the height of that student but measured in
centimeters.
What we have here is actually a random variable, which is
defined as a function of another random variable.
And the point that this example is trying to make is
that functions of random variables
are also random variables.
The experiment happens, the experiment determines a
numerical value for this object.
And once you have the numerical value for this
object, that determines also the numerical
value for that object.
So given an outcome, the numerical value of this
particular object is determined.
So H bar is itself a function from the sample space, from
outcomes to numerical values.
And that makes it a random variable according to the
formal definition that we have here.
So the formal definition is that the random variable is
not random, it's not a variable, it's just a function
from the sample space to the real numbers.
That's the abstract, right way of thinking about them.
Now, random variables can be of different types.
They can be discrete or continuous.
Suppose that I measure the heights in inches, but I round
to the nearest inch.
Then the numerical values I'm going to get here would be
just integers.
So that would make it an integer
valued random variable.
And this is a discrete random variable.
Or maybe I have a scale for measuring height which is
infinitely precise and records your height to an infinite
number of digits of precision.
In that case, your height would be just a
general real number.
So we would have a random variable that takes values in
the entire set of real numbers.
Well, I guess not really negative numbers, but the set
of non-negative numbers.
And that would be a continuous random variable.
It takes values in a continuous set.
So we will be talking about both discrete and continuous
random variables.
The first thing we will do will be to devote a few
lectures on discrete random variables, because discrete is
always easier.
And then we're going to repeat everything in
the continuous setting.
So discrete is easier, and it's the right place to
understand all the concepts, even those who may appear to
be elementary.
And then you will be set to understand what's going on
when we go to the continuous case.
So in the continuous case, you get all the complications of
calculus and some extra math that comes in there.
So it's important to have been down all the concepts very
well in the easy, discrete case so that you don't have
conceptual hurdles when you move on to
the continuous case.
Now, one important remark that may seem trivial but it's
actually very important so that you don't get tangled up
between different types of concepts--
there's a fundamental distinction between the random
variable itself, and the numerical
values that it takes.
Abstractly speaking, or mathematically speaking, a
random variable, x, or H in this example, is a function.
OK.
Maybe if you like programming the words "procedure" or
"sub-routine" might be better.
So what's the sub-routine height?
Given a student, I take that student, force them on the
scale and measure them.
That's the sub-routine that measures heights.
It's really a function that takes students as input and
produces numbers as output.
The sub-routine we denoted by capital H.
That's the random variable.
But once you plug in a particular student into that
sub-routine, you end up getting a particular number.
This is the numerical output of that sub-routine or the
numerical value of that function.
And that numerical value is an element of the real numbers.
So the numerical value is a real number, where this
capital X is a function from omega to the real numbers.
So they are very different types of objects.
And the way that we keep track of what we're talking about at
any given time is by using capital letters for random
variables and lower case letters for numbers.
OK.
So now once we have a random variable at hand, that random
variable takes on different numerical values.
And we want to describe to say something about the relative
likelihoods of the different numerical values that the
random variable can take.
So here's our sample space, and here's the real line.
And there's a bunch of outcomes that gave rise to one
particular numerical value.
There's another numerical value that arises if we have
this outcome.
There's another numerical value that arises if we have
this outcome.
So our sample space is here.
The real numbers are here.
And what we want to do is to ask the question, how likely
is that particular numerical value to occur?
So what we're essentially asking is, how likely is it
that we obtain an outcome that leads to that particular
numerical value?
We calculate that overall probability of that numerical
value and we represent that probability using a bar so
that we end up generating a bar graph.
So that could be a possible bar graph
associated with this picture.
The size of this bar is the total probability that our
random variable took on this numerical value, which is just
the sum of the probabilities of the different outcomes that
led to that numerical value.
So the thing that we're plotting here, the bar graph--
we give a name to it.
It's a function, which we denote by lowercase b, capital
X. The capital X indicates which random variable we're
talking about.
And it's a function of little x, which is the range of
values that our random variable is taking.
So in mathematical notation, the value of the PMF at some
particular number, little x, is the probability that our
random variable takes on the numerical value, little x.
And if you want to be precise about what this means, it's
the overall probability of all outcomes for which the random
variable ends up taking that value, little x.
So this is the overall probability of all omegas that
lead to that particular numerical
value, x, of interest.
So what do we know about PMFs?
Since there are probabilities, all these entries in the bar
graph have to be non-negative.
Also, if you exhaust all the possible values of little x's,
you will have exhausted all the possible outcomes here.
Because every outcome leads to some particular x.
So the sum of these probabilities
should be equal to one.
This is the second relation here.
So this relation tell us that some little
x is going to happen.
They happen with different probabilities, but when you
consider all the possible little x's together, one of
those little x's is going to be realized.
Probabilities need to add to one.
OK.
So let's get our first example of a non-trivial bar graph.
Consider the experiment where I start with a coin and I
start flipping it over and over.
And I do this until I obtain heads for the first time.
So what are possible outcomes of this experiment?
One possible outcome is that I obtain heads at the first
toss, and then I stop.
In this case, my random variable takes the value 1.
Or it's possible that I obtain tails and then heads.
How many tosses did it take until heads appeared?
This would be x equals to 2.
Or more generally, I might obtain tails for k minus 1
times, and then obtain heads at the k-th time, in which
case, our random variable takes the value, little k.
So that's the experiment.
So capital X is a well defined random variable.
It's the number of tosses it takes until I see heads for
the first time.
These are the possible outcomes.
These are elements of our sample space.
And these are the values of X depending on the outcome.
Clearly X is a function of the outcome.
You tell me the outcome, I'm going to tell you what X is.
So what we want to do now is to calculate the PMF of X. So
Px of k is, by definition, the probability that our random
variable takes the value k.
For the random variable to take the value of k, the first
head appears at toss number k.
The only way that this event can happen is if we obtain
this sequence of events.
T's the first k minus 1 times, tails, and
heads at the k-th flip.
So this event, that the random variable is equal to k, is the
same as this event, k minus 1 tails followed by 1 head.
What's the probability of that event?
We're assuming that the coin tosses are independent.
So to find the probability of this event, we need to
multiply the probability of tails, times the probability
of tails, times the probability of tails.
We multiply k minus one times, times the probability of
heads, which puts an extra p at the end.
And this is the formula for the so-called geometric PMF.
And why do we call it geometric?
Because if you go and plot the bar graph of this random
variable, X, we start at 1 with a certain
number, which is p.
And then at 2 we get p(1-p).
At 3 we're going to get something smaller, it's p
times (1-p)-squared.
And the bars keep going down at the rate of geometric
progression.
Each bar is smaller than the previous bar, because each
time we get an extra factor of 1-p involved.
So the shape of this PMF is the graph
of a geometric sequence.
For that reason, we say that it's the geometric PMF, and we
call X also a geometric random variable.
So the number of coin tosses until the first head is a
geometric random variable.
So this was an example of how to compute the
PMF of a random variable.
This was an easy example, because this event could be
realized in one and only one way.
So to find the probability of this, we just needed to find
the probability of this particular outcome.
More generally, there's going to be many outcomes that can
lead to the same numerical value.
And we need to keep track of all of them.
For example, in this picture, if I want to find this value
of the PMF, I need to add up the probabilities of all the
outcomes that leads to that value.
So the general procedure is exactly what
this picture suggests.
To find this probability, you go and identify which outcomes
lead to this numerical value, and add their probabilities.
So let's do a simple example.
I take a tetrahedral die.
I toss it twice.
And there's lots of random variables that you can
associate with the same experiment.
So the outcome of the first throw, we can call it F.
That's a random variable because it's determined once
you tell me what happens in the experiment.
The outcome of the second throw is
another random variable.
The minimum of the two throws is also a random variable.
Once I do the experiment, this random variable takes on a
specific numerical value.
So suppose I do the experiment and I get a 2 and a 3.
So this random variable is going to take the numerical
value of 2.
This is going to take the numerical value of 3.
This is going to take the numerical value of 2.
And now suppose that I want to calculate the PMF of this
random variable.
What I will need to do is to calculate Px(0), Px(1), Px(2),
Px(3), and so on.
Let's not do the entire calculation then, let's just
calculate one of the entries of the PMF.
So Px(2)--
that's the probability that the minimum of the two throws
gives us a 2.
And this can happen in many ways.
There are five ways that it can happen.
Those are all of the outcomes for which the smallest of the
two is equal to 2.
That's five outcomes assuming that the tetrahedral die is
fair and the tosses are independent.
Each one of these outcomes has probability of 1/16.
There's five of them, so we get an answer, 5/16.
Conceptually, this is just the procedure that you use to
calculate PMFs the way that you construct this
particular bar graph.
You consider all the possible values of your random
variable, and for each one of those random variables you
find the probability that the random variable takes on that
value by adding the probabilities of all the
possible outcomes that leads to that
particular numerical value.
So let's do another, more interesting one.
So let's revisit the coin tossing
problem from last time.
Let us fix a number n, and we decide to flip a coin n
consecutive times.
Each time the coin tosses are independent.
And each one of the tosses will have a probability, p, of
obtaining heads.
Let's consider the random variable, which is the total
number of heads that have been obtained.
Well, that's something that we dealt with last time.
We know the probabilities for different numbers of heads,
but we're just going to do the same now
using today's notation.
So let's, for concreteness, n equal to 4.
Px is the PMF of that random variable, X. Px(2) is meant to
be, by definition, it's the probability that a random
variable takes the value of 2.
So this is the probability that we have, exactly two
heads in our four tosses.
The event of exactly two heads can happen in multiple ways.
And here I've written down the different
ways that it can happen.
It turns out that there's exactly six
ways that it can happen.
And each one of these ways, luckily enough, has the same
probability--
p-squared times (1-p)-squared.
So that gives us the value for the PMF evaluated at 2.
So here we just counted explicitly that we have six
possible ways that this can happen, and this gave rise to
this factor of 6.
But this factor of 6 turns out to be the same as
this 4 choose 2.
If you remember definition from last time, 4 choose 2 is
4 factorial divided by 2 factorial, divided by 2
factorial, which is indeed equal to 6.
And this is the more general formula that
you would be using.
In general, if you have n tosses and you're interested
in the probability of obtaining k heads, the
probability of that event is given by this formula.
So that's the formula that we derived last time.
Except that last time we didn't use this notation.
We just said the probability of k heads is equal to this.
Today we introduce the extra notation.
And also having that notation, we may be tempted to also plot
a bar graph for the Px.
In this case, for the coin tossing problem.
And if you plot that bar graph as a function of k when n is a
fairly large number, what you will end up obtaining is a bar
graph that has a shape of something like this.
So certain values of k are more likely than others, and
the more likely values are somewhere in the
middle of the range.
And extreme values--
too few heads or too many heads, are unlikely.
Now, the miraculous thing is that it turns out that this
curve gets a pretty definite shape, like a so-called bell
curve, when n is big.
This is a very deep and central fact from probability
theory that we will get to in a couple of months.
For now, it just could be a curious observation.
If you go into MATLAB and put this formula in and ask MATLAB
to plot it for you, you're going to get an interesting
shape of this form.
And later on we will have to sort of understand where this
is coming from and whether there's a nice, simple formula
for the asymptotic form that we get.
All right.
So, so far I've said essentially nothing new, just
a little bit of notation and this little conceptual thing
that you have to think of random variables as functions
in the sample space.
So now it's time to introduce something new.
This is the big concept of the day.
In some sense it's an easy concept.
But it's the most central, most important concept that we
have to deal with random variables.
It's the concept of the expected
value of a random variable.
So the expected value is meant to be, let's speak loosely,
something like an average, where you interpret
probabilities as something like frequencies.
So you play a certain game and your rewards are going to be--
use my standard numbers--
your rewards are going to be one dollar
with probability 1/6.
It's going to be 2 dollars with probability 1/2, and four
dollars with probability 1/3.
So this is a plot of the PMF of some random variable.
If you play that game and you get so many dollars with this
probability, and so on, how much do you expect to get on
the average if you play the game a zillion times?
Well, you can think as follows--
one sixth of the time I'm going to get one dollar.
One half of the time that outcome is going to happen and
I'm going to get two dollars.
And one third of the time the other outcome happens, and I'm
going to get four dollars.
And you evaluate that number and it turns out to be 2.5.
OK.
So that's a reasonable way of calculating the average payoff
if you think of these probabilities as the
frequencies with which you obtain the different payoffs.
And loosely speaking, it doesn't hurt to think of
probabilities as frequencies when you try to make sense of
various things.
So what did we do here?
We took the probabilities of the different outcomes, of the
different numerical values, and multiplied them with the
corresponding numerical value.
Similarly here, we have a probability and the
corresponding numerical value and we added up over all x's.
So that's what we did.
It looks like an interesting quantity to deal with.
So we're going to give a name to it, and we're going to call
it the expected value of a random variable.
So this formula just captures the calculation that we did.
How do we interpret the expected value?
So the one interpretation is the one that I
used in this example.
You can think of it as the average that you get over a
large number of repetitions of an experiment where you
interpret the probabilities as the frequencies with which the
different numerical values can happen.
There's another interpretation that's a little more visual
and that's kind of insightful, if you remember your freshman
physics, this kind of formula gives you the center of
gravity of an object of this kind.
If you take that picture literally and think of this as
a mass of one sixth sitting here, and the mass of one half
sitting here, and one third sitting there, and you ask me
what's the center of gravity of that structure.
This is the formula that gives you the center of gravity.
Now what's the center of gravity?
It's the place where if you put your pen right underneath,
that diagram will stay in place and will not fall on one
side and will not fall on the other side.
So in this thing, by picture, since the 4 is a little more
to the right and a little heavier, the center of gravity
should be somewhere around here.
And that's what for the math gave us.
It turns out to be two and a half.
Once you have this interpretation about centers
of gravity, sometimes you can calculate
expectations pretty fast.
So here's our new random variable.
It's the uniform random variable in which each one of
the numerical values is equally likely.
Here there's a total of n plus 1 possible numerical values.
So each one of them has probability 1 over (n + 1).
Let's calculate the expected value of this random variable.
We can take the formula literally and consider all
possible outcomes, or all possible numerical values, and
weigh them by their corresponding probability, and
do this calculation and obtain an answer.
But I gave you the intuition of centers of gravity.
Can you use that intuition to guess the answer?
What's the center of gravity infrastructure of this kind?
We have symmetry.
So it should be in the middle.
And what's the middle?
It's the average of the two end points.
So without having to do the algebra, you know that's the
answer is going to be n over 2.
So this is a moral that you should keep whenever you have
PMF, which is symmetric around a certain point.
That certain point is going to be the expected value
associated with this particular PMF.
OK.
So having defined the expected value, what is there that's
left for us to do?
Well, we want to investigate how it behaves, what kind of
properties does it have, and also how do you calculate
expected values of complicated random variables.
So the first complication that we're going to start with is
the case where we deal with a function of a random variable.
OK.
So let me redraw this same picture as before.
We have omega.
This is our sample space.
This is the real line.
And we have a random variable that gives rise to various
values for X. So the random variable is capital X, and
every outcome leads to a particular numerical value x
for our random variable X. So capital X is really the
function that maps these points into the real line.
And then I consider a function of this random variable, call
it capital Y, and it's a function of my
previous random variable.
And this new random variable Y takes numerical values that
are completely determined once I know the numerical value of
capital X. And perhaps you get a diagram of this kind.
So X is a random variable.
Once you have an outcome, this determines the value of x.
Y is also a random variable.
Once you have the outcome, that determines
the value of y.
Y is completely determined once you know X. We have a
formula for how to calculate the expected value of X.
Suppose that you're interested in calculating the expected
value of Y. How would you go about it?
OK.
The only thing you have in your hands is the definition,
so you could start by just using the definition.
And what does this entail?
It entails for every particular value of y, collect
all the outcomes that leads to that value of y.
Find their probability.
Do the same here.
For that value, collect those outcomes.
Find their probability and weight by y.
So this formula does the addition over this line.
We consider the different outcomes and add things up.
There's an alternative way of doing the same accounting
where instead of doing the addition over those numbers,
we do the addition up here.
We consider the different possible values of x, and we
think as follows--
for each possible value of x, that value is going to occur
with this probability.
And if that value has occurred, this is how much I'm
getting, the g of x.
So I'm considering the probability of this outcome.
And in that case, y takes this value.
Then I'm considering the probabilities of this outcome.
And in that case, g of x takes again that value.
Then I consider this particular x, it happens with
this much probability, and in that case, g of x takes that
value, and similarly here.
We end up doing exactly the same arithmetic, it's only a
question whether we bundle things together.
That is, if we calculate the probability of this, then
we're bundling these two cases together.
Whereas if we do the addition up here, we do a separate
calculation--
this probability times this number, and then this
probability times that number.
So it's just a simple rearrangement of the way that
we do the calculations, but it does make a big difference in
practice if you actually want to calculate expectations.
So the second procedure that I mentioned, where you do the
addition by running over the x-axis
corresponds to this formula.
Consider all possibilities for x and when that x happens, how
much money are you getting?
That gives you the average money that you are getting.
All right.
So I kind of hand waved and argued that it's just a
different way of accounting, of course one needs to prove
this formula.
And fortunately it can be proved.
You're going to see that in recitation.
Most people, once they're a little comfortable with the
concepts of probability, actually believe that this is
true by definition.
In fact it's not true by definition.
It's called the law of the unconscious statistician.
It's something that you always do, but it's something that
does require justification.
All right.
So this gives us basically a shortcut for calculating
expected values of functions of a random variable without
having to find the PMF of that function.
We can work with the PMF of the original function.
All right.
So we're going to use this property over and over.
Before we start using it, one general word of caution--
the average of a function of a random variable, in general,
is not the same as the function of the average.
So these two operations of taking averages and taking
functions do not commute.
What this inequality tells you is that, in general, you can
not reason on the average.
So we're going to see instances where this property
is not true.
You're going to see lots of them.
Let me just throw it here that it's something that's not true
in general, but we will be interested in the exceptions
where a relation like this is true.
But these will be the exceptions.
So in general, expectations are average,
something like averages.
But the function of an average is not the same as the average
of the function.
OK.
So now let's go to properties of expectations.
Suppose that alpha is a real number, and I ask you, what's
the expected value of that real number?
So for example, if I write down this expression--
expected value of 2.
What is this?
Well, we defined random variables and we defined
expectations of random variables.
So for this to make syntactic sense, this thing inside here
should be a random variable.
Is 2 --
the number 2 --- is it a random variable?
In some sense, yes.
It's the random variable that takes, always, the value of 2.
So suppose that you have some experiment and that experiment
always outputs 2 whenever it happens.
Then you can say, yes, it's a random experiment but it
always gives me 2.
The value of the random variable is
always 2 no matter what.
It's kind of a degenerate random variable that doesn't
have any real randomness in it, but it's still useful to
think of it as a special case.
So it corresponds to a function from the sample space
to the real line that takes only one value.
No matter what the outcome is, it always gives me a 2.
OK.
If you have a random variable that always gives you a 2,
what is the expected value going to be?
The only entry that shows up in this summation
is that number 2.
The probability of a 2 is equal to 1, and the value of
that random variable is equal to 2.
So it's the number itself.
So the average value in an experiment that always gives
you 2's is 2.
All right.
So that's nice and simple.
Now let's go to our experiment where age was
your height in inches.
And I know your height in inches, but I'm interested in
your height measured in centimeters.
How is that going to be related to
your height in inches?
Well, if you take your height in inches and convert it to
centimeters, I have another random variable, which is
always, no matter what, two and a half times bigger than
the random variable I started with.
If you take some quantity and always multiplied by two and a
half what happens to the average of that quantity?
It also gets multiplied by two and a half.
So you get a relation like this, which says that the
average height of a student measured in centimeters is two
and a half times the average height of a
student measured in inches.
So that makes perfect intuitive sense.
If you generalize it, it gives us this relation, that if you
have a number, you can pull it outside the expectation and
you get the right result.
So this is a case where you can reason on the average.
If you take a number, such as height, and multiply it by a
certain number, you can reason on the average.
I multiply the numbers by two, the averages
will go up by two.
So this is an exception to this cautionary statement that
I had up there.
How do we prove that this fact is true?
Well, we can use the expected value rule here, which tells
us that the expected value of alpha X, this is our g of X,
essentially, is going to be the sum over all x's of my
function, g of X, times the probability of the x's.
In our particular case, g of X is alpha times X. And we have
those probabilities.
And the alpha goes outside the summation.
So we get alpha, sum over x's, x Px of x, which is alpha
times the expected value of X.
So that's how you prove this relation formally using this
rule up here.
And the next formula that I have here also gets
proved the same way.
What does this formula tell you?
If I take everybody's height in centimeters--
we already multiplied by alpha--
and the gods give everyone a bonus of ten extra
centimeters.
What's going to happen to the average height of the class?
Well, it will just go up by an extra ten centimeters.
So this expectation is going to be giving you the bonus of
beta just adds a beta to the average height in centimeters,
which we also know to be alpha times the expected
value of X, plus beta.
So this is a linearity property of expectations.
If you take a linear function of a single random variable,
the expected value of that linear function is the linear
function of the expected value.
So this is our big exception to this cautionary note, that
we have equal if g is linear.
OK.
All right.
So let's get to the last concept of the day.
What kind of functions of random
variables may be of interest?
One possibility might be the average value of X-squared.
Why is it interesting?
Well, why not.
It's the simplest function that you can think of.
So if you want to calculate the expected value of
X-squared, you would use this general rule for how you can
calculate expected values of functions of random variables.
You consider all the possible x's.
For each x, you see what's the probability that it occurs.
And if that x occurs, you consider and see how big
x-squared is.
Now, the more interesting quantity, a more interesting
expectation that you can calculate has to do not with
x-squared, but with the distance of x from the mean
and then squared.
So let's try to parse what we've got up here.
Let's look just at the quantity inside here.
What kind of quantity is it?
It's a random variable.
Why?
X is random, the random variable, expected value of X
is a number.
Subtract a number from a random variable, you get
another random variable.
Take a random variable and square it, you get another
random variable.
So the thing inside here is a legitimate random variable.
What kind of random variable is it?
So suppose that we have our experiment and we have
different x's that can happen.
And the mean of X in this picture might be somewhere
around here.
I do the experiment.
I obtain some numerical value of x.
Let's say I obtain this numerical value.
I look at the distance from the mean, which is this
length, and I take the square of that.
Each time that I do the experiment, I go and record my
distance from the mean and square it.
So I give more emphasis to big distances.
And then I take the average over all possible outcomes,
all possible numerical values.
So I'm trying to compute the average squared
distance from the mean.
This corresponds to this formula here.
So the picture that I drew corresponds to that.
For every possible numerical value of x, that numerical
value corresponds to a certain distance from the mean
squared, and I weight according to how likely is
that particular value of x to arise.
So this measures the average squared
distance from the mean.
Now, because of that expected value rule, of course, this
thing is the same as that expectation.
It's the average value of the random variable, which is the
squared distance from the mean.
With this probability, the random variable takes on this
numerical value, and the squared distance from the mean
ends up taking that particular numerical value.
OK.
So why is the variance interesting?
It tells us how far away from the mean we expect to be on
the average.
Well, actually we're not counting distances from the
mean, it's distances squared.
So it gives more emphasis to the kind of outliers in here.
But it's a measure of how spread out the
distribution is.
A big variance means that those bars go far to the left
and to the right, typically.
Where as a small variance would mean that all those bars
are tightly concentrated around the mean value.
It's the average squared deviation.
Small variance means that we generally have small
deviations.
Large variances mean that we generally have large
deviations.
Now as a practical matter, when you want to calculate the
variance, there's a handy formula which I'm not proving
but you will see it in recitation.
It's just two lines of algebra.
And it allows us to calculate it in a somewhat simpler way.
We need to calculate the expected value of the random
variable and the expected value of the squares of the
random variable, and these two are going
to give us the variance.
So to summarize what we did up here, the variance, by
definition, is given by this formula.
It's the expected value of the squared deviation.
But we have the equivalent formula, which comes from
application of the expected value rule, to the function g
of X, equals to x minus the (expected value of X)-squared.
OK.
So this is the definition.
This comes from the expected value rule.
What are some properties of the variance?
Of course variances are always non-negative.
Why is it always non-negative?
Well, you look at the definition and your just
adding up non-negative things.
We're adding squared deviations.
So when you add non-negative things, you get something
non-negative.
The next question is, how do things scale if you take a
linear function of a random variable?
Let's think about the effects of beta.
If I take a random variable and add the constant to it,
how does this affect the amount of spread that we have?
It doesn't affect--
whatever the spread of this thing is, if I add the
constant beta, it just moves this diagram here, but the
spread doesn't grow or get reduced.
The thing is that when I'm adding a constant to a random
variable, all the x's that are going to appear are further to
the right, but the expected value also moves to the right.
And since we're only interested in distances from
the mean, these distances do not get affected.
x gets increased by something.
The mean gets increased by that same something.
The difference stays the same.
So adding a constant to a random variable doesn't do
anything to it's variance.
But if I multiply a random variable by a constant alpha,
what is that going to do to its variance?
Because we have a square here, when I multiply my random
variable by a constant, this x gets multiplied by a constant,
the mean gets multiplied by a constant, the square gets
multiplied by the square of that constant.
And because of that reason, we get this square of alpha
showing up here.
So that's how variances transform under linear
transformations.
You multiply your random variable by constant, the
variance goes up by the square of that same constant.
OK.
That's it for today.
See you on Wednesday.