The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high-quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
JOHN TSITSIKLIS: OK.
We can start.
Good morning.
So we're going to start now a new unit.
For the next couple of lectures, we will be talking
about continuous random variables.
So this is new material which is not going
to be in the quiz.
You are going to have a long break next week without any
lecture, just a quiz and recitation and tutorial.
So what's going to happen in this new unit?
Basically, we want to do everything that we did for
discrete random variables, reintroduce the same sort of
concepts but see how they apply and how they need to be
modified in order to talk about random variables that
take continuous values.
At some level, it's all the same.
At some level, it's quite a bit harder because when things
are continuous, calculus comes in.
So the calculations that you have to do on the side
sometimes need a little bit more thinking.
In terms of new concepts, there's not going to be a
whole lot today, some analogs of things we have done.
We're going to introduce the concept of cumulative
distribution functions, which allows us to deal with
discrete and continuous random variables, all
of them in one shot.
And finally, introduce a famous kind of continuous
random variable, the normal random variable.
OK, so what's the story?
Continuous random variables are random variables that take
values over the continuum.
So the numerical value of the random variable can be any
real number.
They don't take values just in a discrete set.
So we have our sample space.
The experiment happens.
We get some omega, a sample point in the sample space.
And once that point is determined, it determines the
numerical value of the random variable.
Remember, random variables are functions on the sample space.
You pick a sample point.
This determines the numerical value of the random variable.
So that numerical value is going to be some real number
on that line.
Now we want to say something about the distribution of the
random variable.
We want to say which values are more likely than others to
occur in a certain sense.
For example, you may be interested in a particular
event, the event that the random variable takes values
in the interval from a to b.
And we want to say something about the
probability of that event.
In principle, how is this done?
You go back to the sample space, and you find all those
outcomes for which the value of the random variable happens
to be in that interval.
The probability that the random variable falls here is
the same as the probability of all outcomes that make the
random variable to fall in there.
So in principle, you can work on the original sample space,
find the probability of this event, and you would be done.
But similar to what happened in chapter 2, we want to kind
of push the sample space in the background and just work
directly on the real axis and talk about
probabilities up here.
So we want now a way to specify probabilities, how
they are bunched together, or arranged, along the real line.
So what did we do for discrete random variables?
We introduced PMFs, probability mass functions.
And the way that we described the random variable was by
saying this point has so much mass on top of it, that point
has so much mass on top of it, and so on.
And so we assigned a total amount of 1 unit of
probability.
We assigned it to different masses, which we put at
different points on the real axis.
So that's what you do if somebody gives you a pound of
discrete stuff, a pound of mass in little chunks.
And you place those chunks at a few points.
Now, in the continuous case, this total unit of probability
mass does not sit just on discrete points but is spread
all over the real axis.
So now we're going to have a unit of mass that spreads on
top of the real axis.
How do we describe masses that are continuously spread?
The way we describe them is by specifying densities.
That is, how thick is the mass that's sitting here?
How dense is the mass that's sitting there?
So that's exactly what we're going to do.
We're going to introduce the concept of a probability
density function that tells us how probabilities accumulate
at different parts of the real axis.
So here's an example or a picture of a possible
probability density function.
What does that density function kind of convey
intuitively?
Well, that these x's are relatively
less likely to occur.
Those x's are somewhat more likely to occur because the
density is higher.
Now, for a more formal definition, we're going to say
that a random variable X is said to be continuous if it
can be described by a density function in
the following sense.
We have a density function.
And we calculate probabilities of falling inside an interval
by finding the area under the curve that sits
on top of that interval.
So that's sort of the defining relation for
continuous random variables.
It's an implicit definition.
And it tells us a random variable is continuous if we
can calculate probabilities this way.
So the probability of falling in this interval is the area
under this curve.
Mathematically, it's the integral of the density over
this particular interval.
If the density happens to be constant over that interval,
the area under the curve would be the length of the interval
times the height of the density, which
sort of makes sense.
Now, because the density is not constant but it kind of
moves around, what you need is to write down an integral.
Now, this formula is very much analogous to what you would do
for discrete random variables.
For a discrete random variable, how do you calculate
this probability?
You look at all x's in this interval.
And you add the probability mass function over that range.
So just for comparison, this would be the formula for the
discrete case--
the sum over all x's in the interval from a to b over the
probability mass function.
And there is a syntactic analogy that's happening here
and which will be a persistent theme when we deal with
continuous random variables.
Sums get replaced by integrals.
In the discrete case, you add.
In the continuous case, you integrate.
Mass functions get replaced by density functions.
So you can take pretty much any formula from the discrete
case and translate it to a continuous analog of that
formula, as we're going to see.
OK.
So let's take this now as our model.
What is the probability that the random variable takes a
specific value if we have a continuous random variable?
Well, this would be the case.
It's a case of a trivial interval, where the two end
points coincide.
So it would be the integral from a to itself.
So you're integrating just over a single point.
Now, when you integrate over a single point, the
integral is just 0.
The area under the curve, if you're only looking at a
single point, it's 0.
So big property of continuous random variables is that any
individual point has 0 probability.
In particular, when you look at the value of the density,
the density does not tell you the probability of that point.
The point itself has 0 probability.
So the density tells you something a little different.
We are going to see shortly what that is.
Before we get there, can the density be
an arbitrary function?
Almost, but not quite.
There are two things that we want.
First, since densities are used to calculate
probabilities, and since probabilities must be
non-negative, the density should also be non-negative.
Otherwise you would be getting negative probabilities, which
is not a good thing.
So that's a basic property that any density function
should obey.
The second property that we need is that the overall
probability of the entire real line should be equal to 1.
So if you ask me, what is the probability that x falls
between minus infinity and plus infinity, well, we are
sure that x is going to fall in that range.
So the probability of that event should be 1.
So the probability of being between minus infinity and
plus infinity should be 1, which means that the integral
from minus infinity to plus infinity should be 1.
So that just tells us that there's 1 unit of total
probability that's being spread over our space.
Now, what's the best way to think intuitively about what
the density function does?
The interpretation that I find most natural and easy to
convey the meaning of a density is to look at
probabilities of small intervals.
So let us take an x somewhere here and then x plus delta
just next to it.
So delta is a small number.
And let's look at the probability of the event that
we get a value in that range.
For continuous random variables, the way we find the
probability of falling in that range is by integrating the
density over that range.
So we're drawing this picture.
And we want to take the area under this curve.
Now, what happens if delta is a fairly small number?
If delta is pretty small, our density is not going to change
much over that range.
So you can pretend that the density is
approximately constant.
And so to find the area under the curve, you just take the
base times the height.
And it doesn't matter where exactly you take the height in
that interval, because the density doesn't change very
much over that interval.
And so the integral becomes just base times the height.
So for small intervals, the probability of a small
interval is approximately the density times delta.
So densities essentially give us
probabilities of small intervals.
And if you want to think about it a little differently, you
can take that delta from here and send it to
the denominator there.
And what this tells you is that the density is
probability per unit length for intervals of small length.
So the units of density are probability per unit length.
Densities are not probabilities.
They are rates at which probabilities accumulate,
probabilities per unit length.
And since densities are not probabilities, they don't have
to be less than 1.
Ordinary probabilities always must be less than 1.
But density is a different kind of thing.
It can get pretty big in some places.
It can even sort of blow up in some places.
As long as the total area under the curve is 1, other
than that, the curve can do anything that it wants.
Now, the density prescribes for us the
probability of intervals.
Sometimes we may want to find the probability of more
general sets.
How would we do that?
Well, for nice sets, you will just integrate the density
over that nice set.
I'm not quite defining what "nice" means.
That's a pretty technical topic in the theory of
probability.
But for our purposes, usually we will take b to be something
like a union of intervals.
So how do you find the probability of falling in the
union of two intervals?
Well, you find the probability of falling in that interval
plus the probability of falling in that interval.
So it's the integral over this interval plus the integral
over that interval.
And you think of this as just integrating over the union of
the two intervals.
So once you can calculate probabilities of intervals,
then usually you are in business, and you can
calculate anything else you might want.
So the probability density function is a complete
description of any statistical information we might be
interested in for a continuous random variable.
OK.
So now we can start walking through the concepts and the
definitions that we have for discrete random variables and
translate them to the continuous case.
The first big concept is the concept of the expectation.
One can start with a mathematical definition.
And here we put down a definition by
just translating notation.
Wherever we have a sum in the discrete case, we
now write an integral.
And wherever we had the probability mass function, we
now throw in the probability density function.
This formula--
you may have seen it in freshman physics--
basically, it again gives you the center of gravity of the
picture that you have when you have the density.
It's the center of gravity of the object sitting underneath
the probability density function.
So that the interpretation still applies.
It's also true that our conceptual interpretation of
what an expectation means is also valid in this case.
That is, if you repeat an experiment a zillion times,
each time drawing an independent sample of your
random variable x, in the long run, the average that you are
going to get should be the expectation.
One can reason in a hand-waving way, sort of
intuitively, the way we did it for the case of discrete
random variables.
But this is also a theorem of some sort.
It's a limit theorem that we're going to visit later on
in this class.
Having defined the expectation and having claimed that the
interpretation of the expectation is that same as
before, then we can start taking just any formula you've
seen before and just translate it.
So for example, to find the expected value of a function
of a continuous random variable, you do not have to
find the PDF or PMF of g(X).
You can just work directly with the original distribution
of the random variable capital X.
And this formula is the same as for the discrete case.
Sums get replaced by integrals.
And PMFs get replaced by PDFs.
And in particular, the variance of a random variable
is defined again the same way.
The variance is the expected value, the average of the
distance of X from the mean and then squared.
So it's the expected value for a random variable that takes
these numerical values.
And same formula as before, integral and F instead of
summation, and the P.
And the formulas that we have derived or formulas that you
have seen for the discrete case, they all go through the
continuous case.
So for example, the useful relation for variances, which
is this one, remains true.
All right.
So time for an example.
The most simple example of a continuous random variable
that there is, is the so-called
uniform random variable.
So the uniform random variable is described by a density
which is 0 except over an interval.
And over that interval, it is constant.
What is it meant to convey?
It's trying to convey the idea that all x's in this range are
equally likely.
Well, that doesn't say very much.
Any individual x has 0 probability.
So it's conveying a little more than that.
What it is saying is that if I take an interval of a given
length delta, and I take another interval of the same
length, delta, under the uniform distribution, these
two intervals are going to have the same probability.
So being uniform means that intervals of same length have
the same probability.
So no interval is more likely than any other to occur.
And in that sense, it conveys the idea of sort of complete
randomness.
Any little interval in our range is equally likely as any
other little interval.
All right.
So what's the formula for this density?
I only told you the range.
What's the height?
Well, the area under the density must be equal to 1.
Total probability is equal to 1.
And so the height, inescapably, is going to be 1
over (b minus a).
That's the height that makes the density integrate to 1.
So that's the formula.
And if you don't want to lose one point in your exam, you
have to say that it's also 0, otherwise.
OK.
All right?
That's sort of the complete answer.
How about the expected value of this random variable?
OK.
You can find the expected value in two different ways.
One is to start with the definition.
And so you integrate over the range of
interest times the density.
And you figure out what that integral is going to be.
Or you can be a little more clever.
Since the center-of-gravity interpretation is still true,
it must be the center of gravity of this picture.
And the center of gravity is, of course, the midpoint.
Whenever you have symmetry, the mean is always the
midpoint of the diagram that gives you the PDF.
OK.
So that's the expected value of X.
Finally, regarding the variance, well, there you will
have to do a little bit of calculus.
We can write down the definition.
So it's an integral instead of a sum.
A typical value of the random variable minus the expected
value, squared, times the density.
And we integrate.
You do this integral, and you find it's (b minus a) squared
over that number, which happens to be 12.
Maybe more interesting is the standard deviation itself.
And you see that the standard deviation is proportional to
the width of that interval.
This agrees with our intuition, that the standard
deviation is meant to capture a sense of how spread out our
distribution is.
And the standard deviation has the same units as the random
variable itself.
So it's sort of good to-- you can interpret it in a
reasonable way based on that picture.
OK, yes.
Now, let's go up one level and think about the following.
So we have formulas for the discrete case, formulas for
the continuous case.
So you can write them side by side.
One has sums, the other has integrals.
Suppose you want to make an argument and say that
something is true for every random variable.
You would essentially need to do two separate proofs, for
discrete and for continuous.
Is there some way of dealing with random variables just one
at a time, in one shot, using a sort of uniform notation?
Is there a unifying concept?
Luckily, there is one.
It's the notion of the cumulative distribution
function of a random variable.
And it's a concept that applies equally well to
discrete and continuous random variables.
So it's an object that we can use to describe distributions
in both cases, using just one piece of notation.
So what's the definition?
It's the probability that the random variable takes values
less than a certain number little x.
So you go to the diagram, and you see what's the probability
that I'm falling to the left of this.
And you specify those probabilities for all x's.
In the continuous case, you calculate those probabilities
using the integral formula.
So you integrate from here up to x.
In the discrete case, to find the probability to the left of
some point, you go here, and you add probabilities again
from the left.
So the way that the cumulative distribution function is
calculated is a little different in the continuous
and discrete case.
In one case you integrate.
In the other, you sum.
But leaving aside how it's being calculated, what the
concept is, it's the same concept in both cases.
So let's see what the shape of the cumulative distribution
function would be in the two cases.
So here what we want is to record for every little x the
probability of falling to the left of x.
So let's start here.
Probability of falling to the left of here is 0--
0, 0, 0.
Once we get here and we start moving to the right, the
probability of falling to the left of here is the area of
this little rectangle.
And the area of that little rectangle increases linearly
as I keep moving.
So accordingly, the CDF increases linearly until I get
to that point.
At that point, what's the value of my CDF?
1.
I have accumulated all the probability there is.
I have integrated it.
This total area has to be equal to 1.
So it reaches 1, and then there's no more probability to
be accumulated.
It just stays at 1.
So the value here is equal to 1.
OK.
How would you find the density if somebody gave you the CDF?
The CDF is the integral of the density.
Therefore, the density is the derivative of the CDF.
So you look at this picture and take the derivative.
Derivative is 0 here, 0 here.
And it's a constant up there, which
corresponds to that constant.
So more generally, and an important thing to know, is
that the derivative of the CDF is equal to the density--
almost, with a little bit of an exception.
What's the exception?
At those places where the CDF does not have a derivative--
here where it has a corner--
the derivative is undefined.
And in some sense, the density is also
ambiguous at that point.
Is my density at the endpoint, is it 0 or is it 1?
It doesn't really matter.
If you change the density at just a single point, it's not
going to affect the value of any
integral you ever calculate.
So the value of the density at the endpoint, you can leave it
as being ambiguous, or you can specify it.
It doesn't matter.
So at all places where the CDF has a derivative,
this will be true.
At those places where you have corners, which do show up
sometimes, well, you don't really care.
How about the discrete case?
In the discrete case, the CDF has a more peculiar shape.
So let's do the calculation.
We want to find the probability of b
to the left of here.
That probability is 0, 0, 0.
Once we cross that point, the probability of being to the
left of here is 1/6.
So as soon as we cross the point 1, we get the
probability of 1/6, which means that the size of the
jump that we have here is 1/6.
Now, question.
At this point 1, which is the correct value of the CDF?
Is it 0, or is it 1/6?
It's 1/6 because--
you need to look carefully at the definitions, the
probability of x being less than or equal to little x.
If I take little x to be 1, it's the probability that
capital X is less than or equal to 1.
So it includes the event that x is equal to 1.
So it includes this probability here.
So at jump points, the correct value of the CDF is going to
be this one.
And now as I trace, x is going to the right.
As soon as I cross this point, I have added another 3/6
probability.
So that 3/6 causes a jump to the CDF.
And that determines the new value.
And finally, once I cross the last point, I get
another jump of 2/6.
A general moral from these two examples and these pictures.
CDFs are well defined in both cases.
For the case of continuous random variables, the CDF will
be a continuous function.
It starts from 0.
It eventually goes to 1 and goes smoothly--
well, continuously from smaller to higher values.
It can only go up.
It cannot go down since we're accumulating more and more
probability as we are going to the right.
In the discrete case, again it starts from 0,
and it goes to 1.
But it does it in a staircase manner.
And you get a jump at each place where the PMF assigns a
positive mass.
So jumps in the CDF are associated with point masses
in our distribution.
In the continuous case, we don't have any point masses,
so we do not have any jumps either.
Now, besides saving us notation--
we don't have to deal with discrete
and continuous twice--
CDFs give us actually a little more flexibility.
Not all random variables are continuous or discrete.
You can cook up random variables that are kind of
neither or a mixture of the two.
An example would be, let's say you play a game.
And with a certain probability, you get a certain
number of dollars in your hands.
So you flip a coin.
And with probability 1/2, you get a reward of 1/2 dollars.
And with probability 1/2, you are led to a dark room where
you spin a wheel of fortune.
And that wheel of fortune gives you a random reward
between 0 and 1.
So any of these outcomes is possible.
And the amount that you're going to get,
let's say, is uniform.
So you flip a coin.
And depending on the outcome of the coin, either you get a
certain value or you get a value that ranges over a
continuous interval.
So what kind of random variable is it?
Is it continuous?
Well, continuous random variables assign 0 probability
to individual points.
Is it the case here?
No, because you have positive probability of
obtaining 1/2 dollar.
So our random variable is not continuous.
Is it discrete?
It's not discrete, because our random variable can take
values also over a continuous range.
So we call such a random variable a
mixed random variable.
If you were to draw its distribution very loosely,
probably you would want to draw a picture like this one,
which kind of conveys the idea of what's going on.
So just think of this as a drawing of masses that are
sitting over a table.
We place an object that weighs half a pound, but it's an
object that takes zero space.
So half a pound is just sitting on top of that point.
And we take another half-pound of probability and spread it
uniformly over that interval.
So this is like a piece that comes from mass functions.
And that's a piece that looks more like a density function.
And we just throw them together in the picture.
I'm not trying to associate any formal
meaning with this picture.
It's just a schematic of how probabilities are distributed,
help us visualize what's going on.
Now, if you have taken classes on systems and all of that,
you may have seen the concept of an impulse function.
And you my start saying that, oh, I should treat this
mathematically as a so-called impulse function.
But we do not need this for our purposes in this class.
Just think of this as a nice picture that conveys what's
going on in this particular case.
So now, what would the CDF look like in this case?
The CDF is always well defined, no matter what kind
of random variable you have.
So the fact that it's not continuous, it's not discrete
shouldn't be a problem as long as we can calculate
probabilities of this kind.
So the probability of falling to the left here is 0.
Once I start crossing there, the probability of falling to
the left of a point increases linearly with
how far I have gone.
So we get this linear increase.
But as soon as I cross that point, I accumulate another
1/2 unit of probability instantly.
And once I accumulate that 1/2 unit, it means that my CDF is
going to have a jump of 1/2.
And then afterwards, I still keep accumulating probability
at a fixed rate, the rate being the density.
And I keep accumulating, again, at a linear rate until
I settle to 1.
So this is a CDF that has certain pieces where it
increases continuously.
And that corresponds to the continuous part of our
randomize variable.
And it also has some places where it has discrete jumps.
And those district jumps correspond to places in which
we have placed a positive mass.
And by the--
OK, yeah.
So this little 0 shouldn't be there.
So let's cross it out.
All right.
So finally, we're going to take the remaining time and
introduce our new friend.
It's going to be the Gaussian or normal distribution.
So it's the most important distribution there is in all
of probability theory.
It's plays a very central role.
It shows up all over the place.
We'll see later in the class in more detail
why it shows up.
But the quick preview is the following.
If you have a phenomenon in which you measure a certain
quantity, but that quantity is made up of lots and lots of
random contributions--
so your random variable is actually the sum of lots and
lots of independent little random variables--
then invariability, no matter what kind of distribution the
little random variables have, their sum will turn out to
have approximately a normal distribution.
So this makes the normal distribution to arise very
naturally in lots and lots of contexts.
Whenever you have noise that's comprised of lots of different
independent pieces of noise, then the end result will be a
random variable that's normal.
So we are going to come back to that topic later.
But that's the preview comment, basically to argue
that it's an important one.
OK.
And there's a special case.
If you are dealing with a binomial distribution, which
is the sum of lots of Bernoulli random variables,
again you would expect that the binomial would start
looking like a normal if you have many, many-- a large
number of point fields.
All right.
So what's the math involved here?
Let's parse the formula for the density of the normal.
What we start with is the function X squared over 2.
And if you are to plot X squared over 2, it's a
parabola, and it has this shape --
X squared over 2.
Then what do we do?
We take the negative exponential of this.
So when X squared over 2 is 0, then negative
exponential is 1.
When X squared over 2 increases, the negative
exponential of that falls off, and it falls off pretty fast.
So as this goes up, the formula for the
density goes down.
And because exponentials are pretty strong in how quickly
they fall off, this means that the tails of this distribution
actually do go down pretty fast.
OK.
So that explains the shape of the normal PDF.
How about this factor 1 over square root 2 pi?
Where does this come from?
Well, the integral has to be equal to 1.
So you have to go and do your calculus exercise and find the
integral of this the minus X squared over 2 function and
then figure out, what constant do I need to put in front so
that the integral is equal to 1?
How do you evaluate that integral?
Either you go to Mathematica or Wolfram's Alpha or
whatever, and it tells you what it is.
Or it's a very beautiful calculus exercise that you may
have seen at some point.
You throw in another exponential of this kind, you
bring in polar coordinates, and somehow the answer comes
beautifully out there.
But in any case, this is the constant that you need to make
it integrate to 1 and to be a legitimate density.
We call this the standard normal.
And for the standard normal, what is the expected value?
Well, the symmetry, so it's equal to 0.
What is the variance?
Well, here there's no shortcut.
You have to do another calculus exercise.
And you find that the variance is equal to 1.
OK.
So this is a normal that's centered around 0.
How about other types of normals that are centered at
different places?
So we can do the same kind of thing.
Instead of centering it at 0, we can take some place where
we want to center it, write down a quadratic such as (X
minus mu) squared, and then take the negative
exponential of that.
And that gives us a normal density that's centered at mu.
Now, I may wish to control the width of my density.
To control the width of my density, equivalently I can
control the width of my parabola.
If my parabola is narrower, if my parabola looks like this,
what's going to happen to the density?
It's going to fall off much faster.
OK.
How do I make my parabola narrower or wider?
I do it by putting in a constant down here.
So by putting a sigma here, this stretches or widens my
parabola by a factor of sigma.
Let's see.
Which way does it go?
If sigma is very small, this is a big number.
My parabola goes up quickly, which means my normal falls
off very fast.
So small sigma corresponds to a narrower density.
And so it, therefore, should be intuitive that the standard
deviation is proportional to sigma.
Because that's the amount by which you
are scaling the picture.
And indeed, the standard deviation is sigma.
And so the variance is sigma squared.
So all that we have done here to create a general normal
with a given mean and variance is to take this picture, shift
it in space so that the mean sits at mu instead of 0, and
then scale it by a factor of sigma.
This gives us a normal with a given
mean and a given variance.
And the formula for it is this one.
All right.
Now, normal random variables have some wonderful
properties.
And one of them is that they behave nicely when you take
linear functions of them.
So let's fix some constants a and b, suppose that X is
normal, and look at this linear function Y.
What is the expected value of Y?
Here we don't need anything special.
We know that the expected value of a linear function is
the linear function of the expectation.
So the expected value is this.
How about the variance?
We know that the variance of a linear function doesn't care
about the constant term.
But the variance gets multiplied by a squared.
So we get these variance, where sigma squared is the
variance of the original normal.
So have we used so far the property that X is normal?
No, we haven't.
This calculation here is true in general when you take a
linear function of a random variable.
But if X is normal, we get the other additional fact that Y
is also going to be normal.
So that's the nontrivial part of the fact that
I'm claiming here.
So linear functions of normal random variables are
themselves normal.
How do we convince ourselves about it?
OK.
It's something that we will do formerly in about two or three
lectures from today.
So we're going to prove it.
But if you think about it intuitively, normal means this
particular bell-shaped curve.
And that bell-shaped curve could be sitting anywhere and
could be scaled in any way.
So you start with a bell-shaped curve.
If you take X, which is bell shaped, and you multiply it by
a constant, what does that do?
Multiplying by a constant is just like scaling the axis or
changing the units with which you're measuring it.
So it will take a bell shape and spread it or narrow it.
But it will still be a bell shape.
And then when you add the constant, you just take that
bell and move it elsewhere.
So under linear transformations, bell shapes
will remain bell shapes, just sitting at a different place
and with a different width.
And that sort of the intuition of why normals remain normals
under this kind of transformation.
So why is this useful?
Well, OK.
We have a formula for the density.
But usually we want to calculate probabilities.
How will you calculate probabilities?
If I ask you, what's the probability that the normal is
less than 3, how do you find it?
You need to integrate the density from minus
infinity up to 3.
Unfortunately, the integral of the expression that shows up
that you would have to calculate, an integral of this
kind from, let's say, minus infinity to some number, is
something that's not known in closed form.
So if you're looking for a closed-form formula for this--
X bar--
if you're looking for a closed-form formula that gives
you the value of this integral as a function of X bar, you're
not going to find it.
So what can we do?
Well, since it's a useful integral, we can
just tabulate it.
Calculate it once and for all, for all values of X bar up to
some precision, and have that table, and use it.
That's what one does.
OK, but now there is a catch.
Are we going to write down a table for every conceivable
type of normal distribution--
that is, for every possible mean and every variance?
I guess that would be a pretty long table.
You don't want to do that.
Fortunately, it's enough to have a table with the
numerical values only for the standard normal.
And once you have those, you can use them in a clever way
to calculate probabilities for the more general case.
So let's see how this is done.
So our starting point is that someone has graciously
calculated for us the values of the CDF, the cumulative
distribution function, that is the probability of falling
below a certain point for the standard normal
and at various places.
How do we read this table?
The probability that X is less than, let's say,
0.63 is this number.
This number, 0.7357, is the probability that the standard
normal is below 0.63.
So the table refers to the standard normal.
But someone, let's say, gives us some other numbers and
tells us we're dealing with a normal with a certain mean and
a certain variance.
And we want to calculate the probability that the value of
that random variable is less than or equal to 3.
How are we going to do it?
Well, there's a standard trick, which is so-called
standardizing a random variable.
Standardizing a random variable
stands for the following.
You look at the random variable, and you
subtract the mean.
This makes it a random variable with 0 mean.
And then if I divide by the standard deviation, what
happens to the variance of this random variable?
Dividing by a number divides the variance by sigma squared.
The original variance of X was sigma squared.
So when I divide by sigma, I end up with unit variance.
So after I do this transformation, I get a random
variable that has 0 mean and unit variance.
It is also normal.
Why is its normal?
Because this expression is a linear function of the X that
I started with.
It's a linear function of a normal random variable.
Therefore, it is normal.
And it is a standard normal.
So by taking a general normal random variable and doing this
standardization, you end up with a standard normal to
which you can then apply the table.
Sometimes one calls this the normalized score.
If you're thinking about test results, how would you
interpret this number?
It tells you how many standard deviations are you
away from the mean.
This is how much you are away from the mean.
And you count it in terms of how many standard
deviations it is.
So this number being equal to 3 tells you that X happens to
be 3 standard deviations above the mean.
And I guess if you're looking at your quiz scores, very
often that's the kind of number that you think about.
So it's a useful quantity.
But it's also useful for doing the calculation we're now
going to do.
So suppose that X has a mean of 2 and a variance of 16, so
a standard deviation of 4.
And we're going to calculate the probability of this event.
This event is described in terms of this X that has ugly
means and variances.
But we can take this event and rewrite it as
an equivalent event.
X less than 3 is this same as X minus 2 being less than 3
minus 2, which is the same as this ratio being less than
that ratio.
So I'm subtracting from both sides of the inequality the
mean and then dividing by the standard deviation.
This event is the same as that event.
Why do we like this better than that?
We like it because this is the standardized, or normalized,
version of X. We know that this is standard normal.
And so we're asking the question, what's the
probability that the standard normal is less than this
number, which is 1/4?
So that's the key property, that this is normal (0, 1).
And so we can look up now with the table and ask for the
probability that the standard normal random variable
is less than 0.25.
Where is that going to be?
0.2, 0.25, it's here.
So the answer is 0.987.
So I guess this is just a drill that you could learn in
high school.
You didn't have to come here to learn about it.
But it's a drill that's very useful when we will be
calculating normal probabilities all the time.
So make sure you know how to use the table and how to
massage a general normal random variable into a
standard normal random variable.
OK.
So just one more minute to look at the big picture and
take stock of what we have done so far
and where we're going.
Chapter 2 was this part of the picture, where we dealt with
discrete random variables.
And this time, today, we started talking about
continuous random variables.
And we introduced the density function, which is the analog
of the probability mass function.
We have the concepts of expectation and
variance and CDF.
And this kind of notation applies to both discrete and
continuous cases.
They are calculated the same way in both cases except that
in the continuous case, you use sums.
In the discrete case, you use integrals.
So on that side, you have integrals.
In this case, you have sums.
In this case, you always have Fs in your formulas.
In this case, you always have Ps in your formulas.
So what's there that's left for us to do is to look at
these two concepts, joint probability mass functions and
conditional mass functions, and figure out what would be
the equivalent concepts on the continuous side.
So we will need some notion of a joint density when we're
dealing with multiple random variables.
And we will also need the concept of conditional
density, again for the case of continuous random variables.
The intuition and the meaning of these objects is going to
be exactly the same as here, only a little subtler because
densities are not probabilities.
They're rates at which probabilities accumulate.
So that adds a little bit of potential confusion here,
which, hopefully, we will fully resolve in the next
couple of sections.
All right.
Thank you.