Events

Uncertainty refers to a situation that is not completely determined. Probability theory is a mathematical formalism used to describe and quanity uncertainty.

1. Sample Spaces and Events

We consider a random experiment whose range of possible outcomes can be described by set , called the sample space. An event is a subset of the sample space, ; it is a collection of some possible outcomes. For example, tossing heads or rolling an even number . Extreme events are and . Singleton subsets of are called the elementary events of .

If we perform a random experiment, the outcome is a single element . Then for any , has occurred iff . If has not occurred, , so has occurred, which can be read as not . The smallest event that could have occurred is , so for any other event, . Thus, for any sample space :

• The null event will never occur.
• The universal event will always occur.
• It is only for events for which there is uncertainty.

Consider a set of events :

• The event will occur iff at least one of the events occurs. So can be read as or .
• The event will occur iff all of the events occur. So can be read as and .
• Events are said to be mutually exclusive if . At most one of the events can occur.

2. Probability

When defining a probabilty function on we simultaneously agree on a collection of subsets of that we wish to assign a probability to, . must be:

• Non-empty: .
• Closed under complements: .
• Closed under countable unions: .

Such a collection of sets is known as -algebra.

A probability measure on the pair is a mapping satisfying the following axioms for all subsets of on which it is defined:

3. Countably additive. For mutually exclusive , we have .

From the axioms, it is easy to derive the following:

• .
• .
• For any events , : .

3. Joint Events

A joint event , where and occur at the same time. The events are independent if . More generally, a set of events are independent if for any finite subset , , where is any set of distinct indices.

(!) Proposition

If and are independent, then and are independent:

• is a disjoint union.
• by axiom .
• .

(1) Proposition

.

• We know from set theory that .
• and are disjoint.
• by axiom .

4. Conditional Probability

For events and in , where , the conditional probability of occuring given is:

If and are independent, then .

defines a valid probability measure, obeying the axioms of probability on the restricted sample space .

For three events , , and , and are conditionally independent given iff .

The law of total probability states that for a set of events that form a partition of , we have that for any event :

(!) Proof of Law of Total Probability

• .
• Hence, by axiom .
• So, .

Note that for any and in , form a partition of . So, by law of total probability, .

For any events and in , we have . This gives us Bayes' Theorem: