CS 591-06, Lecture 8

Jared Saia University of New Mexico

Class Outline

A reasonable probability is the only certainty. - E.W. Howe

• Intro to probability theory

• Some basic Probability definitions and facts

• Linearity of Expectation

• Union Bounds

Intro to Probability Theory

Defns:

• An Event Space is a set of “events” which is closed under complement and union.

• A Probability Function P is a mapping from an event space, Γ, to the real numbers such that:

– For all γ ∈ Γ, 0 ≤ P (γ) ≤ 1

– P (Ω) = 1, where Ω is the set containing all atomic events and their complements.

– For mutually disjoint events, γ_{1}, γ_{2}, …, γ_{l},

P ( l ; ⋃ ; i=1 γ_{i}) = l ; ∑ ; i=1 P (γ_{i})

Example

Consider event space when we flip a coin. Let γ_{1} be the event that the coin turns up heads.

Then Γ = {{}, γ_{1}, γ̄_{1}, {γ_{1} ⋃ γ̄_{1}}}

and:

• Every element in Γ has probability of occurence between 1 and 0.

• P (Ω) = P ({γ_{1} ⋃ γ̄_{1}}) = 1

• Probabilities of events that are disjoint (can’t occur together) add. E.g. P (γ_{1} ⋃ γ̄_{1}) = P (γ_{1}) + P (γ̄_{1})

Basic Facts

• Complement Fact: For any event γ, P (γ̄) = 1 − P (γ)

• Independence Fact A set of events {γ_{i}|i ∈ I}, is said to be independent if for any subset S of I, P (⋂_{i∈S} γ_{i}) = ∏_{i∈S} P (γ_{i})

In Class Exercise

Imagine we flip two independent coins. Let γ_{1} be the event that the first coin is heads, γ_{2} be the event that the second coin is heads.

Let P (γ_{1}) = 1/2 and P (γ_{2}) = 1/2.

• Question 1: What is the probability that both coins turn up heads? (Hint: Use the fact that γ_{1} and γ_{2} are independent).

• Question 2: What is the probability that at least one coin turns up heads? (Hint: Use the complement fact and independence of the events γ̄_{1} and γ̄_{2}).

Useful Algebraic Tricks

Some useful algebraic facts to know: • (1 − x) ≤ e^{−x}

Sailors and Cabins

• e^{x} ∗ e^{y} = e^{x+y} There are n sailors and n cabins and sailor i is assigned to cabin i.

After a night of drunken revelry, the sailors return to ship and each of them chooses uniformly at random one of n cabins to sleep in.

• Question: What is (an upper bound on) the probability that no sailor sleeps in his own cabin?

• Let γ_{i} be the event that sailor i sleeps in his own cabin.

• Note that for all i, P (γ_{i}) = 1/n

• Note that all of the events γ_{i} are independent.

• We want P (⋂_{i=1}^{n} γ̄_{i})

P ( n ; ⋂ ; i=1 γ̄_{i}) = n ; ∏ ; i=1 P (γ̄_{i})  (1)
= n ; ∏ ; i=1 (1 − P (γ_{i}))  (2)
= n ; ∏ ; i=1 (1 − 1/n)  (3)
≤ n ; ∏ ; i=1 e^{−1/n}  (4)
≤ e^{−1}  (5)

The first step holds by Independence, and the second step holds by Complement.

Random Variables

• A Random Variable is a variable which randomly takes on some value in the real numbers.

Example Random Variables:

• Let X be 1 with probability 1/2 and 0 with probability 1/2 • Let Y take on a value in the range {1, …, n} uniformly at

random.

• Let Z = X + Y.

• Let W = X ∗ Y.

Expectation

The Expectation of a random variable X (written E(X)), which takes on values in some range R, is defined as follows:

• E(X) = ∑_{r∈R}(r ∗ P (X = r))

Linearity of Expectation

Linearity of Expecation Fact: If there are random variable X, X_{1}, …, X_{n}, such that X = ∑_{i=1}^{n} X_{i}, then

• E(X) = ∑_{i=1}^{n} E(X_{i})

Example:

Let the variable X_{i} be 1 if the i-th flip of a coin is heads and 0 otherwise and say that E(X_{i}) = 1/2. Let X = ∑_{i=1}^{n} X_{i}. Then:

E(X) = n/2

More Drunken Sailors

There are n sailors and n cabins and sailor i is assigned to cabin i.

After a night of drunken revelry, the sailors return to ship and each of them chooses uniformly at random one of n cabins to sleep in.

• What is the expected number of sailors that actually sleep in their own cabin?

• Let r_{i} be an random variable which has value one of sailor i sleeps in his own cabin and 0 otherwise.

• E(r_{i}) = (1/n)

• Let R = ∑_{i=1}^{n} r_{i}

• Then we know by Linearity of Expectation:

E(R) = E( n ; ∑ ; i=1 r_{i})  (6)
= n ; ∑ ; i=1 E(r_{i})  (7)
= n ; ∑ ; i=1 1/n  (8)
= 1  (9)

Union Bound

Union Bound Fact: Let Γ be any event space, P be any probability function and γ_{1}, γ_{2}, … γ_{n} be any events in Γ. Then:

P ( n ; ⋃ ; i=1 γ_{i}) ≤ n ; ∑ ; i=1 P (γ_{i})

Example:

Let γ_{1} be the event that coin 1 is heads, γ_{2} be the event that coin 2 is heads. Then:

P (γ_{1} ⋃ γ_{2}) ≤ P (γ_{1}) + P (γ_{2})

Drunk Sailors, the next day

There are n sailors and n mugs (of coffee). The i-th mug belongs to sailor i.

The next morning, each sailor wakes up and randomly takes exactly one mug.

• Question: What is an upper bound on the probability that one of the first three sailors gets the first mug?

Drunk Sailors, the next day

• For i = {1, 2, 3}, let γ_{i} be the event that the i-th sailor gets the first mug.

• We want P (γ_{1} ⋃ γ_{2} ⋃ γ_{3})

• By the Union Bound, P (γ_{1} ⋃ γ_{2} ⋃ γ_{3}) ≤ P (γ_{1})+P (γ_{2})+P (γ_{3})

• So P (γ_{1} ⋃ γ_{2} ⋃ γ_{3}) ≤ 3/n

Another Union Bound Example

Imagine an Attack-Resistant Peer-to-peer network over n peers.

• Assume that for any fixed set of n/2 peers, the probability that deletion of that set foils the network is p_{f}.

• There are n choose n/2 sets of size n/2

• Hence the probability that no set exists such that its deletion will foil the network is no more than p_{f} * (n choose n/2).