[MUSIC PLAYING]
BRIAN YU: All right, welcome back, everyone, to an Introduction
to Artificial Intelligence with Python.
And last time we took a look at how it is that AI inside of our computers
can represent knowledge.
We represented that knowledge in the form of logical sentences
in a variety of different logical languages,
and the idea was we wanted our AI to be able to represent knowledge
or information and somehow use those pieces of information
to be able to derive new pieces of information via inference,
to be able to take some information and deduce
some additional conclusions based on the information
that it already knew for sure.
But in reality, when we think about computers and we think about AI,
very rarely are our machines going to be able to know things for sure.
Oftentimes there's going to be some amount of uncertainty
in the information that our AIs or our computers
are dealing with where it might believe something with some probability,
as we'll soon discuss what probability is all about and what it means,
but not entirely for certain.
And we want to use the information that it has some knowledge about, even if it
doesn't have perfect knowledge, to still be able to make inferences, still
be able to draw conclusions.
So you might imagine, for example, in the context of a robot that
has some sensors and is exploring some environment,
it might not know exactly where it is or exactly what's around it,
but it does have access to some data that can allow it to draw inferences
with some probability.
There's some likelihood that one thing is true or another,
or you can imagine in context where there is a little bit more
randomness and uncertainty, something like predicting the weather, where
you might not be able to know for sure what tomorrow's weather is
with 100% certainty, but you can probably
infer with some probability what tomorrow's weather is
going to be based on maybe today's webinar and yesterday's weather
and other data that you might have access to as well.
And so oftentimes we can distill this in terms of just possible events
that might happen and what the likelihood of those events are.
This comes a lot in games, for example, where
there's an element of chance inside of those games.
So you imagine rolling the dice.
You're not sure exactly what the die roll is going to be,
but you know it's going to be one of these possibilities from one to six,
for example.
And so here, now, we introduce the idea of probability theory.
And what we'll take a look at today is beginning
by looking at the mathematical foundations of probability theory,
getting an understanding for some of the key concepts within probability,
and then diving into how we can use probability and the ideas
that we look at mathematically to represent some ideas in terms of models
that we can put into our computers in order to program an AI that
is able to use information about probability to draw inferences,
to make some judgments about the world with some probability
or likelihood of being true.
So probability ultimately boils down to this idea
that there are possible worlds that we're here
representing using this little Greek letter omega,
and the idea of a possible world is that, when I roll a die,
there are six possible worlds that could result from it.
I can roll a 1 or 2 or 3 or a 4 or a 5 or a 6,
and each of those or a possible world, and each of those possible worlds
has some probability of being true, the probability that I do roll a 1 or a 2
or a 3 or something else.
And we represent that probability like this,
using the capital letter P and then, in parentheses, what
it is that we want the probability of.
So this right here would be the probability of some possible world
as represented by the little letter omega.
Now, there are a couple of basic axioms of probability
that become relevant as we consider how we deal with probability
and how we think about it.
First and foremost, every probability value
must range between zero and one inclusive.
So the smallest value any probability can
have is the number zero, which is an impossible event, something
like I roll a die and the die is a seven is the roll that I get.
If the die only has numbers one through six,
the event that I roll a seven is impossible,
so it would have probability zero.
And on the other end of the spectrum, probability
can range all the way up to the positive number one,
meaning an event is certain to happen, that I roll a die and the number
is less than 10, for example.
That is an event that is guaranteed to happen if the only sides on my die
are one through six, for instance.
And then there can range through any real number in between these two values
where, generally speaking, a higher value for the probability
means an event is more likely to take place
and a lower value for the probability means
the event is less likely to take place.
And the other key rule for probability looks a little bit like this.
This sigma notation, if you haven't seen it before,
refers to summation, the idea that we're going to be adding up
a whole sequence of values.
And this sigma notation's going to come up a couple of times
today, because as we deal with probability,
oftentimes we're adding up a whole bunch of individual values
or individual probabilities to get some other value.
So we'll see this come up a couple of times.
But what this notation means is that if I sum up all of the possible world's
omega that are in big Omega, which represents the set of all
the possible worlds, meaning I take for all of the worlds
in the set of possible worlds and add up all of their probabilities, what
I ultimately get is the number one.
So if I take all the possible worlds, add up what each of their probabilities
is, I should get the number one at the end,
meaning all probabilities just need to sum to one.
So for example, if I take dice, for example,
if you imagine I have a fair die with numbers one through six
and I roll the die, each one of these rolls
has an equal probability of taking place,
and the probability is one over six, for example.
So each of these probabilities is between zero and one, zero meaning
and possible and one meaning for certain.
And if you add up all of these probabilities
for all of the possible worlds, you get the number one.
And we can represent any one of those probabilities like this.
The probability that we roll the number two, for example, is just one over six.
Every six times we roll the die, we'd expect that one time, for instance,
the die might come up as a two.
Its probability is not certain, but it's a little more than nothing,
for instance.
And so this is all fairly straightforward for just a single die.
But things get more interesting as our models of the world
get a little bit more complex.
Let's imagine now that we're not just dealing with a single die,
but we have two dice, for example.
I have a red die here and a blue die there,
and I care not just about what the individual roll is,
but I care about the sum of the two rolls.
In this case, the sum of the two rolls is the number three.
How do I begin to now reason about, what does
the probability look like if, instead of having one die, I now have two dice?
Well, what we might imagine is that we could first consider,
what are all of the possible worlds?
And in this case, all of the possible worlds
are just every combination of the red and blue die that I could come up with.
For the red die, it could be a 1 or a 2 or a 3 or a 4 or a 5 or a 6,
and for each of those possibilities, the blue die,
likewise, could also be either 1 or 2 or 3 or 4 or 5 or 6.
And it just so happens that, in this particular case,
each of these possible combinations is equally likely.
Equally likely are all of these various different possible worlds.
That's not always going to be the case.
As you imagine more complex models that we could try to build
and things that we could try to represent in the real world,
it's probably not going to be the case that every single possible world is
always equally likely.
But in the case of fair dice where, in any given die roll,
any one number has just as good a chance of coming up as any other number,
we can consider all of these possible worlds to be equally likely.
But even though all of the possible worlds are equally likely,
that doesn't necessarily mean that their sums are equally likely.
So if we consider what the sum is of all of these two-- so 1 plus 1, that's a 2.
2 plus 1 is a 3--
and consider for each of these possible pairs of numbers
what their sum ultimately is, we can notice
that there are some patterns here where it's not entirely the case
that every number comes up equally likely.
If you consider seven, for example, what's
the probability that when I roll two dice their sum is seven,
there are several ways this can happen.
There are six possible worlds where the sum is seven.
It could be a one and a six or a two and a five or a three
and a four, a four and a three, and so forth.
But if you instead consider, what's the probability that I
roll two dice and the sum of those two die rolls is 12, for example, well,
looking at this diagram, there's only one possible world
in which that can happen, and that's the possible world
where both the red die and the blue die both come up at sixes
to give us the sum total of 12.
So based on just taking a look at this diagram,
we see that some of these probabilities are likely different.
The probability that the sum is a seven must be greater than the probability
that the sum is a 12.
And we can represent that even more formally by saying, OK,
the probability that we sum to 12 is one out of 36.
Out of the 36 equally likely possible worlds,
six squared because we have six options for the red die
and six options for the blue die, out of those 36 options, only one of them
sums to 12, whereas, on the other hand, the probability
that if we take two dice rolls and they sum up to the number seven,
well, out of those 36 possible worlds, there
were six worlds where the sum was seven, and so we get six over 36,
which we can simplify as a fraction to just one over six.
So here, now, we're able to represent these different ideas of probability,
representing some events that might be more likely
and then other events that are less likely, as well.
And these sorts of judgments where we're figuring out,
just in the abstract, what is the probability that this thing takes
place, are generally known as unconditional probabilities,
some degree of belief we have in some proposition, some fact about the world
in the absence of any other evidence without knowing
any additional information.
If I roll a die, what's the chance it comes up as a two,
or if I roll two dice, what's the chance that the sum of those two
die rolls is a seven?
But usually when we're thinking about probability, especially when we're
thinking about training in AI to intelligently be able to know something
about the world and make predictions based on that information,
it's not unconditional probability that our AI is dealing with,
but, rather, conditional probability, probability
where rather than having no original knowledge,
we have some initial knowledge about the world and how the world actually works.
So conditional probability is the degree of belief
in a proposition given some evidence that has already been revealed to us.
So what does this look like?
Well, it looks like this in terms of notation.
We're going to represent conditional probability as probability of a
and then this vertical bar and then b.
And the way to read this is the thing on the left-hand side of the vertical bar
is what we want the probability of.
Here, now, I want the probability that a is true, that it is the real world,
that it is the event that actually does take place.
And then on the right side of the vertical bar
is our evidence, the information that we already
know for certain about the world--
for example, that b is true.
So the way to read this entire expression
is, what is the probability of a given b,
the probability that a is true given that we already know that b is true?
And this type of judgment, conditional probability,
the probability of one thing given some other fact, comes up quite a lot
when we think about the types of calculations
we might want our AI to be able to do.
For example, we might care about the probability of rain
today given that we know that it rained yesterday.
We could think about the probability of rain today just in the abstract.
What is the chance that today it rains?
But usually we have some additional evidence.
I know for certain that it rained yesterday,
and so I would like to calculate the probability that it rains today
given that I know that it rained yesterday,
or you might imagine that I want to know the probability
that my optimal route to my destination changes
given the current traffic conditions.
So whether or not traffic conditions change,
that might change the probability that this route is actually
the optimal route, or you might imagine in a medical context
I want to know the probability that a patient has a particular disease given
some results of some tests that have been performed on that patient,
and I have some evidence, the results of that test,
and I would like to know the probability that a patient has
a particular disease.
So this notion of conditional probability
comes up everywhere as we begin to think about what
we would like to reason about, but being able to reason
a little more intelligently by taking into account evidence
that we already have.
We're more able to get an accurate result for what
is the likelihood that someone has this disease
if we know this evidence, the results of the test,
as opposed to if we were just calculating
the unconditional probability of saying, what is the probability they have
the disease without any evidence to try and back up our result one
way or the other?
So now that we've got this idea of what conditional probability is,
the next question we have to ask is, all right,
how do we calculate conditional probability?
How do we figure out, mathematically, if I have an expression like this,
how do I get a number from that?
What does conditional probability actually mean?
Well, the formula for conditional probability
looks a little something like this--
the probability of a given b, the probability
that a is true given that we know that b is true,
is equal to this fraction-- the probability
that a and b are true divided by just the probability that b is true.
And the way to intuitively try to think about this
is that if I want to know the probability that a is true given that b
is true, well, I want to consider all the ways they could both be
true out of the only worlds that I care about
are the worlds where b is already true.
I can sort of ignore all the cases where b isn't true
because those aren't relevant to my ultimate computation.
They're not relevant to what it is that I want to get information about.
So let's take a look at an example.
Let's go back to that example of rolling two dice and the idea
that those two dice might sum up to the number 12.
We discussed earlier that the unconditional probability
that if I roll two dice and they sum to 12 is one out of 36,
because out of the 36 possible worlds that I might care about,
in only one of them is the sum of those two dice 12.
It's only when red is six and blue is also six.
But let's say now that I have some additional information.
I now want to know, what is the probability
that the two dice sum to 12 given that I know that the red die was a six?
So I already have some evidence.
I already know the red die is a six.
I don't know what the blue die is.
That information isn't given to me in this expression.
But given the fact that I know that the red die
rolled a six, what is the probability that we sum to 12?
And so we can begin to do the math using that expression from before.
Here, again, are all of the possibilities,
all of the possible combinations of red die being one through six
and blue die being one through six.
And I might consider, first, all right, what
is the probability of my evidence, my b variable where
I want to know what is the probability that the red die is a six?
Well, the probability that the red die is a six is just one out of six.
So these one out of six options are really the only worlds
that I care about here now.
All the rest of them are irrelevant to my calculation
because I already have this evidence that the red die was a six,
so I don't need to care about all of the other possibilities that could result.
So now, in addition to the fact that the red die rolled as a six
and the probability of that, the other piece
of information I need to know in order to calculate
this conditional probability is the probability that both of my variables,
a and b, are true, the probability that both the red die is a six
and they all sum to 12.
So what is the probability that both of these things happen?
Well, it only happens in one possible case, in one out of these 36 cases,
and it's the case where both the red and the blue die are equal to six.
This is a piece of information that we already knew.
And so this probability is equal to one over 36.
And so to get the conditional probability
that the sum is 12 given that I know that the red dice is equal to six,
well, I just divide these two values together, and 1/36 divided by 1/6
gives us this probability of 1/6.
Given that I know that the red die rolled a value of six,
the probability that the sum of the two dice is 12 is also one over six.
And that probably makes intuitive sense for you,
too, because if the red die is a six, the only way for me to get to a 12
is if the blue die also rolls a six.
And we know that the probability of the blue die rolling a six is one over six.
So in this case, the conditional probability
seems fairly straightforward.
But this idea of calculating a conditional probability
by looking at the probability that both of these events take place
is an idea that's going to come up again and again.
This is the definition, now, of conditional probability,
and we're going to use that definition as we
think about probability more generally to be able to draw conclusions
about the world.
This, again, is that formula.
The probability of a given b is equal to the probability
that a and b take place divided by the probability of b.
And you'll see this formula sometimes written in a couple of different ways.
You could imagine, algebraically, multiplying both sides of this equation
by probability of b to get rid of the fraction,
and you'll get an expression like this.
The probability of a and b, which is this expression over here,
is just the probability of b times the probability of a given b,
or you could represent this equivalently since a and b, in this expression,
are interchangeable. a and b is the same thing as b and a.
You could imagine also representing the probability of a and b
as the probability of a times the probability of b given a, just
switching all of the a's and b's.
These three are all equivalent ways of trying to represent
what joint probability means.
And so you'll sometimes see all of these equations,
and they might be useful to you as you begin to reason about probability
and to think about what values might be taking place in the real world.
Now, sometimes when we deal with probability,
we don't just care about a Boolean event.
Like, did this happen or did this not happen?
Sometimes we might want the ability to represent variable values
in a probability space where some variable might take
on multiple different possible values.
And in probability, we call a variable in probability theory
a random variable.
A random variable in probability is just some variable in probability theory
that has some domain of values that it can take on.
So what do I mean by this?
Well, what I mean is I might have a random variable that
is just called Roll, for example, that has six possible values.
Roll is my variable, and the possible values,
the domain of values that it can take on, are 1, 2, 3, 4, 5, and 6.
And I might like to know the probability of each.
In this case, they happen to all be the same.
But in other random variables, that might not be the case.
For example, I might have a random variable
to represent the weather, for example, where
the domain of values it could take on are things like sun or cloudy
or rainy or windy or snowy, and each of those
might have a different probability, and I care about knowing,
what is the probability that the weather equals sun
or that the weather equals clouds, for instance,
and I might like to do some mathematical calculations based on that information.
Other random variables might be something like traffic.
What are the odds that there is no traffic or light traffic
or heavy traffic?
Traffic, in this case, is my random variable,
and the values that that random variable can take on are here.
It's either none or light or heavy.
And I, the person doing these calculations, I, the person encoding
these random variables into my computer, need
to make the decision as to what these possible values actually are.
You might imagine, for example, for a flight,
if I care about whether or not I make it to a flight on time,
my flight has a couple of possible values that it could take on.
My flight could be on time.
My flight could be delayed.
My flight could be canceled.
So flight, in this case, is my random variable,
and these are the values that it can take on.
And often I'll want to know something about the probability
that my random variable takes on each of those possible values.
And this is what we then call a probability distribution.
A probability distribution takes a random variable
and gives me the probability for each of the possible values in its domain.
So in the case of this flight, for example, my probability distribution
might look something like this.
My probability distribution says, the probability
that the random variable Flight is equal to the value on time is 0.6,
or, otherwise, put into more English, human-friendly terms, the likelihood
that my flight is on time is 60%, for example.
And in this case, the probability that my flight is delayed is 30%.
The probability that my flight is canceled is 10%, or 0.1.
And if you sum up all of these possible values, the sum is going to be 1.
If you take all of the possible worlds, here
are my three possible worlds for the value of the random variable Flight.
Add them all up together.
The result needs to be the number one per that axiom of probability theory
that we've discussed before.
So this now is one way of representing this probability distribution
for the random variable Flight.
Sometimes you'll see it represented a little bit more
concisely, that this is pretty verbose for really just trying
to express three possible values.
And so often you'll instead see this same notation
representing using a vector.
And all a vector is is a sequence of values.
As opposed to just a single value, I might have multiple values.
And so I could extend, instead, represent this idea this way--
bold P-- so a larger P--
generally meaning the probability distribution of this variable flight
is equal to this vector represented in angle brackets.
The probability distribution is 0.6, 0.3, and 0.1,
and I would just have to know that this probability distribution is
an order of on time or delayed and canceled
to know how to interpret this vector to mean the first value in the vector
is the probability that my flight is on time, the second value in the vector
is the probability that my flight is delayed,
and the third value in the vector is the probability that my flight is canceled.
And so this is just an alternate way of representing
this idea a little more verbosely.
But oftentimes you'll see us just talk about a probability
distribution over a random variable.
And whenever we talk about that, what we're really doing
is trying to figure out the probabilities of each
of the possible values that that random variable can take on,
but this notation is just a little bit more succinct,
even though it can sometimes be a little confusing depending
on the context in which you see it.
So we'll start to look at examples where we use this sort of notation
to describe probability and to describe events that might take place.
A couple of other important ideas to know with
regards to probability theory--
one is this idea of independence, and independence refers to the idea
that the knowledge of one event doesn't influence
the probability of another event.
So for example, in the context of my two dice
rolls where I had the red die and the blue die, the probability
that I roll the red die and the blue die,
those two events, red die and blue die, are independent.
Knowing the result of the red die doesn't change the probabilities
for the blue die.
It doesn't give me any additional information
about what the value of the blue die is ultimately going to be.
But that's not always going to be the case.
You might imagine that in the case of weather, something like clouds
and rain, those are probably not independent, that if it is cloudy,
that might increase the probability that later in the day it's going to rain.
So some information informs some other event or some other random variable.
So independence refers to the idea that one event doesn't influence the other.
And if they're not independent, then there might be some relationship.
So mathematically, formally, what does independence actually mean?
Well, recall this formula from before, that the probability of a and b
is the probability of a times the probability of b given a.
And the more intuitive way to think about this
is that to know how likely it is that a and b happen,
well, let's first figure out the likelihood that a happens,
and then given that we know that a happens,
let's figure out the likelihood that b happens
and multiply those two things together.
But if a and b were independent, meaning knowing a doesn't change anything
about the likelihood that b is true, well,
then the probability of b given a, meaning the probability that b is true
given that I know a is true, well, that I know a is true
shouldn't really make a difference if these two things are independent,
that a shouldn't influence b at all.
So the probability of b given a is really just the probability of b,
if it is true that a and b are independent.
And so this right here is one example of a definition for what
it means for a and b to be independent.
The probability of a and b is just the probability of a times
the probability of b.
Any time you find two events a and b where this relationship holds,
then you can say that a and b are independent.
So an example of that might be the dice that we were taking a look at before.
Here, if I wanted the probability of red being a six and blue being a six,
well, that's just the probability that red is a six multiplied
by the probability that blue is a six.
Both equal to one over 36.
So I can say that these two events are independent.
What wouldn't be independent, for example, would be an example--
so this, for example, has a probability of one
over 36, as we talked about before.
But what wouldn't be independent would be a case like this--
the probability that the red die rolls a six and the red die rolls a four.
If you just naively took, OK, red die six, red die four,
well, if I'm only rolling the die once, you
might imagine the naive approach is to say, well, each of these
has a probability of one over six.
So multiply them together, and the probability is one over 36.
But, of course, if you're only rolling the red die once,
there's no way you could get two different values for the red die.
It couldn't both be a six and a four.
So the probability should be zero.
But if you were to multiply probability of red six times probability
of red four, well, that would equal one over 36.
But, of course, that's not true because we
know that there is no way, probability zero, that when we roll the red die
once we get both a six and a four because only one of those possibilities
can actually be the result.
And so we can say that the event that red roll is six and the event
that red roll is four, those two events are not independent.
If I know that the red roll is a six, I know that the red roll cannot possibly
be a four.
So these things are not independent.
And instead, if I wanted to calculate the probability,
I would need to use this conditional probability,
as is the regular definition of the probability of two events taking place.
And the probability of this, now, well, the probability of the red
roll being a six, that's one of six.
But what's the probability that the roll is a four given that the roll is a six?
Well, this is just zero, because there's no way for the red roll
to be a four given that we already know the red roll is a six.
And so the value, if we do all that multiplication,
is we get the number zero.
So this idea of conditional probability is
going to come up again and again, especially
as we begin to reason about multiple different random variables that
might be interacting with each other in some way.
And this gets us to one of the most important rules in probability theory,
which is known as Bayes' rule.
And it turns out that just using the information we've already
learned about probability and just applying a little bit of algebra,
we can actually derive Bayes' rule for ourselves.
But it's a very important rule when it comes to inference
and thinking about probability in the context of what
it is that a computer can do, or what a mathematician could do,
by having access to information about probability.
So let's go back to these equations to be
able to derive Bayes' rule ourselves.
We know the probability of a and b, the likelihood that a and b take place,
is the likelihood of b and then the likelihood of a given
that we know that b is already true.
And likewise, the probability of a given a and b
is the probability of a times the probability of b given
that we know that a is already true.
This is sort of a symmetric relationship where
it doesn't matter the order of a and b and b and a mean the same thing.
And so in these equations, we can just swap out a and b
to be able to represent the exact same idea.
So we know that these two equations are already true.
We've seen that already.
And now let's just do a little bit of algebraic manipulation of this stuff.
Both of these expressions on the right-hand side
are equal to the probability of a and b.
So what I can do is take these two expressions on the right-hand side
and just set them equal to each other.
If they're both equal to the probability of a and b,
then they both must be equal to each other.
So probability of a times probability of b
given a is equal to the probability of b times the probability of a given b.
And now all we're going to do is do a little bit of division.
I'm going to divide both sides by P of a, and now I get what is Bayes' rule.
The probability of b given a is equal to the probability of b
times the probability of a given b divided by the probability of a.
And sometimes in Bayes' rule you'll see the order
of these two arguments switched.
So instead of b times a given b, it'll be a given b times b.
That ultimately doesn't matter because in multiplication you
can switch the order of the two things you're multiplying
and it doesn't change the result.
But this here right now is the most common formulation of Bayes' rule.
The probability of b given a is equal to the probability of a given
b times the probability of b divided by the probability of a.
And this rule, it turns out, is really important
when it comes to trying to infer things about the world
because it means you can express one conditional probability,
the conditional probability of b given a, using knowledge
about the probability of a given b, using the reverse
of that conditional probability.
So let's first do a little bit of an example with this,
just to see how we might use it, and then explore what
this means a little bit more generally.
So we're going to construct a situation where I have some information.
There are two events that I care about--
the idea that it's cloudy in the morning and the idea
that it is rainy in the afternoon.
Those are two different possible events that
could take place-- cloudy in the morning, or the AM, rainy in the PM.
And what I care about is, given clouds in the morning, what
is the probability of rain in the afternoon, a reasonable question
I might ask.
In the morning, I look outside, or an AI's camera
looks outside, and sees that there are clouds in the morning,
and we want to conclude, we want to figure out,
what is the probability that in the afternoon there is going to be rain?
Of course, in the abstract, we don't have access
to this kind of information, but we can use data
to begin to try and figure this out.
So let's imagine, now, that I have access to some pieces of information.
I have access to the idea that 80% of rainy afternoons
start out with a cloudy morning.
And you might imagine that I could have gathered this data just
by looking at data over a sequence of time,
that I know that 80% of the time when it's raining in the afternoon it
was cloudy that morning.
I also know that 40% of days have cloudy mornings,
and I also know that 10% of days have rainy afternoons.
And now, using this information, I would like
to figure out, given clouds in the morning, what
is the probability that it rains in the afternoon?
I want to know the probability of afternoon rain given morning clouds,
and I can do that, in particular, using this fact, the probability of--
so if I know that 80% of rainy afternoon start with cloudy mornings,
then I know the probability of cloudy mornings given rainy afternoon.
So using sort of the reverse conditional probability, I can figure that out.
Expressed in terms of Bayes' rule, this is what that would look like--
probability of rain given clouds is the probability
of clouds given rain times the probability of rain
divided by the probability of clouds.
Here I'm just substituting in for the values of a and b
from that equation and Bayes' rule from before.
And then I can just do the math.
I have this information.
I know that 80% of the time, if it was raining, then
there were clouds in the morning-- so 0.8 here.
Probability of rain is 0.1 because 10% of days were raining and 40% of days
were cloudy.
I do the math and I can figure out the answer is 0.2.
So the probability that it rains in the afternoon given that it was cloudy
in the morning is 0.2 in this case.
And this, now, is an application of Bayes' rule,
the idea that using one conditional probability,
we can get the reverse conditional probability.
And this is often useful when one of the conditional probabilities
might be easier for us to know about or easier for us to have data about,
and using that information, we can calculate
the other conditional probability.
So what does this look like?
Well, it means that knowing the probability of cloudy mornings given
rainy afternoons, we can calculate the probability of rainy afternoons given
cloudy mornings, or, for example, more generally,
if we know the probability of some visible effect, some effect
that we can see and observe given some unknown cause that we're not
sure about, well, then we can calculate the probability of that unknown cause
given the visible effect.
So what might that look like?
Well, in the context of medicine, for example,
I might know the probability of some medical test result given a disease.
Like, I know that if someone has a disease, then x percent of the time
the medical test result will show up as this, for instance.
And using that information, then I can calculate,
what is the probability that, given I know the medical test
result, what is the likelihood that someone has the disease?
This is the piece of information that is usually easier to know, easier
to immediately have access to data for.
And this is the information that I actually want to calculate.
Or I might want to know, for example--
if I know that some probability of counterfeit bills
have blurry text around the edges, because counterfeit printers
aren't nearly as good at printing text precisely.
So I have some information about given that something
is a counterfeit bill, x percent of counterfeit bills have blurry text,
for example.
And using that information, then I can calculate some piece of information
that I might want to know, like, given that I
know there's blurry text on a bill, what is the probability that that bill is
counterfeit?
So given one conditional probability, I can
calculate the other conditional probability as well.
And so now that we've taken a look at a couple
of different types of probability.
We've looked at unconditional probability
where I just look at what is the probability of this event occurring
given no additional evidence that I might have access to,
and we've also looked at conditional probability
where I have some sort of evidence, and I would like to, using that evidence,
be able to calculate some other probability as well.
The other kind of probability that will be important for us to think about
is joint probability, and this is when we're
considering the likelihood of multiple different events simultaneously.
And so what do we mean by this?
Well, for example, I might have probability distributions
that look a little something like this, like I want to know the probability
distribution of clouds in the morning, and that distribution looks like this.
40% of the times, C, which is my random variable here,
is equal to it's cloudy, and 60% of the time it's not cloudy.
So here is just a simple probability distribution
that is effectively telling me that 40% of the time it's cloudy.
I might also have a probability distribution for rain in the afternoon
where 10% of the time, or with probability 0.1,
it is raining in the afternoon and with probability 0.9
it is not raining in the afternoon.
And using just these two pieces of information,
I don't actually have a whole lot of information about how these two
variables relate to each other.
But I could if I had access to their joint probability,
meaning for every combination of these two things--
meaning morning cloudy and afternoon rain, morning cloudy and afternoon
not rain, morning not cloudy and afternoon rain, and morning
not cloudy and afternoon not raining--
if I had access to values for each of those four,
I'd have more information-- so information that'd
be organized in a table like this.
And this, rather than just a probability distribution,
is a joint probability distribution.
It tells me the probability distribution of each
of the possible combinations of values that these random variables
can take on.
So if I want to know, what is the probability that on any given day
it is both cloudy and rainy, well, I would say,
all right, we're looking at cases where it is cloudy
and cases where it is raining and the intersection of those two,
that row and that column, is 0.08.
So that is the probability that it is both cloudy and rainy
using that information.
And using this conditional probability table,
using this joint probability table, I can
begin to draw other pieces of information
about things like conditional probability.
So I might ask a question like, what is the probability distribution of clouds
given that I know that it is raining, meaning
I know for sure that it's raining.
Tell me the probability distribution over whether it's cloudy
or not given that I know already that it is, in fact, raining.
And here I'm using C to stand for that random variable.
I'm looking for a distribution, meaning the answer to this
is not going to be a single value.
It's going to be two values, a vector of two values where the first value is
probability of clouds, the second value is probability that it is not cloudy,
but the sum of those two values is going to be one,
because when you add up the probabilities of all
of the possible worlds, the result that you get must be the number one.
And, well, what do we know about how to calculate a conditional probability?
Well, we know that the probability of a given b is the probability of a and b
divided by the probability of b.
So what does this mean?
Well, it means that I can calculate the probability of clouds
given that it's raining as the probability of clouds
and raining divided by the probability of rain.
And this comma here for the probability distribution of clouds and rain,
this comma sort of stands in for the word "and."
You'll sort of see the logical operator AND and the comma used interchangeably.
This means the probability distribution over the clouds
and knowing the fact that it is raining divided by the probability of rain.
And the interesting thing to note here and what we'll often
do in order to simplify our mathematics is
that dividing by the probability of rain, the probability of rain
here is just some numerical constant.
It is some number.
Dividing by probability of rain is just dividing
by some constant or, in other words, multiplying
by the inverse of that constant.
And it turns out that oftentimes we can just
not worry about what the exact value of this is
and just know that it is, in fact, a constant value,
and we'll see why in a moment.
So instead of expressing this as this joint probability divided
by the probability of rain, sometimes we'll just represent it as alpha times
the numerator here, the probability distribution of C, this variable,
and that we know that it is raining, for instance.
So all we've done here is said this value of one
over the probability of rain, that's really just a constant that we're
going to divide by or equivalently multiply by the inverse of at the end.
We'll just call it alpha for now and deal with it a little bit later.
But the key idea here now-- and this is an idea that's going to come up again--
is that the conditional distribution of C given rain
is proportional to, meaning just some factor multiplied by,
the joint probability of C and rain being true.
And so how do we figure this out?
Well, this is going to be the probability that it is cloudy
given that it's raining, which is 0.08, and the probability that it's not
cloudy given that it's raining, which is 0.02.
And so we get alpha times--
here now is that probability distribution.
0.08 is clouds and rain.
0.02 is not cloudy and rain.
But, of course, 0.08 and 0.02 don't sum up to the number one.
And we know that in a probability distribution,
if you consider all of the possible values,
they must sum up to a probability of one.
And so we know that we just need to figure out some constant to normalize,
so to speak, these values, something we can multiply or divide by
to get it so that all of these probabilities sum up to one.
And it turns out that if we multiply both numbers by 10,
then we can get that result of 0.8 and 0.2.
The proportions are still equivalent, but now 0.8
plus 0.2, those sum up to the number 1.
So take a look at this and see if you can understand,
step by step, how it is we're getting from one point to another.
But the key idea here is that by using the joint probabilities,
these probabilities that it is both cloudy and rainy and that it is not
cloudy and rainy, I can take that information and figure out
the conditional probability-- given that it's raining,
what is the chance that it's cloudy versus not cloudy--
just by multiplying by some normalization constant, so to speak.
And this is what a computer can begin to use
to be able to interact with these various different types
of probabilities.
And it turns out there are a number of other probability rules
that are going to be useful to us as we begin to explore how we can actually
use this information to encode into our computers
some more complex analysis that we might want to do about probability
and distributions and random variables that we might be interacting with.
So here are a couple of those important probability rules.
One of the simplest rules is just this negation rule.
What is the probability of not event a?
So a is an event that has some probability,
and I would like to know, what is the probability that a does not occur?
And it turns out it's just one minus P of a, which makes sense
because if those are the two possible cases,
either a happens or a doesn't happen, then when you add up those two cases,
you must get one, which means P of not a must just be one minus P of a
because P of a and P of not a must sum up to the number one.
They must include all of the possible cases.
We've seen an expression for calculating the probability of a and b.
We might also reasonably want to calculate the probability of a or b.
What is the probability that one thing happens or another thing happens?
So for example, I might want to calculate,
what is the probability that if I roll two dice,
a red die and a blue die, what is the likelihood that a is a six or b
is a six, one or the other?
And what you might imagine you could do and the wrong way to approach it
would be just to say, all right, well, a comes up as a six,
the red die comes up as a six with probability one over six.
The same for the blue die.
It's also one over six.
Add them together and you get 2/6, otherwise known as 1/3.
But this suffers from the problem of over counting,
that we've double counted the case where both a and b, both
the red die and the blue die, both come up as a six roll,
and I've counted that instance twice.
So to resolve this, the actual expression
for calculating the probability of a or b
uses what we call the inclusion-exclusion formula.
So I take the probability of a, add it to the probability of b.
That's all same as before.
But then I need to exclude the cases that I've double counted.
So I subtract from that the probability of a and b, and that
gets me the result for a or b.
I consider all the cases where a is true and all the cases where b is true.
And if you imagine this is like a Venn diagram of cases
where a is true, cases where b is true, I just
need to subtract out the middle to get rid of the cases
that I have over counted by double counting them inside of both
of these individual expressions.
One other rule that's going to be quite helpful
is a rule called marginalization.
Some marginalization is answering the question
of how do I figure out the probability of a using some other variable that I
might have access to, like b?
Even if I don't know additional information about it,
I know that b, some event, can have two possible states.
Either b happens or b doesn't happen, assuming it's a Boolean, true or false.
And well, what that means is that for me to be
able to calculate the probability of a, there are only two cases.
Either a happens and b happens or a happens and b doesn't happen.
And those are two disjoint, meaning they can't both happen together--
either b happens or b doesn't happen.
They're disjoint or separate cases.
And so I can figure out the probability of a just by adding up those two cases.
The probability that a is true is the probability
that a and b is true plus the probability that a is true
and b isn't true.
So by marginalizing, I've looked at the two possible cases
that might take place.
Either b happens or b doesn't happen.
And in either of those cases, I look at, what's the probability that a happens,
and if I add those together, well, then I get the probability
that a happens as a whole.
So take a look at that rule.
It doesn't matter what b is or how it's related to a.
So long as I know these joint distributions,
I can figure out the overall probability of a.
And this can be a useful way, if I have a joint distribution,
like the joint distribution of a and b, to just figure out
some unconditional probability, like the probability of a,
and we'll see examples of this soon, as well.
Now, sometimes these might not just be variables
that are events that are they happened or they didn't happen, like b is here.
They might be some broader probability distribution where
there are multiple possible values.
And so here, in order to use this marginalization rule,
I need to sum up not just over b and not b, but for all of the possible values
that the other random variable could take on.
And so here we'll see a version of this rule for random variables,
and it's going to include that summation notation
to indicate that I'm summing up, adding up, a whole bunch of individual values.
So here's the rule.
Looks a lot more complicated, but it's actually
the equivalent, exactly the same rule.
What I'm saying here is that if I have two random variables one called x
and one called y, well, the probability that x is equal to some value x sub i--
this is just some value that this variable takes on--
how do I figure it out?
Well, I'm going to sum up over j, where j
is going to range over all of the possible values that y can take on.
Well, let's look at the probability that x equals xi and y equals yj.
So the exact same rule-- the only difference here
is now I'm summing up over all of the possible values
that y can take on, saying let's add up all of those possible cases
and look at this joint distribution, this joint probability
that x takes on the value I care about given all of the possible values for y.
And if I add all those up, then I can get this unconditional probability
of what x is equal to, whether or not x is equal to some value x sub i.
So let's take a look at this rule because it
does look a little bit complicated.
Let's try and put a concrete example to it.
Here, again, is that same joint distribution from before.
I have cloud, not cloudy, rainy, not rainy.
And maybe I want to access some variable.
I want to know, what is the probability that it is cloudy?
Well, marginalization says that if I have this joint distribution
and I want to know, what is the probability that it is cloudy, well,
I need to consider the other variable, the variable that's not here,
the idea that it's rainy.
And I consider the two cases, either it's raining or it's not raining,
and I just sum up the values for each of those possibilities.
In other words, the probability that it is cloudy
is equal to the sum of the probability that it's cloudy
and it's raining and the probability that it's cloudy and it is not raining.
And so these, now, are values that I have access to.
These are values that are just inside of this joint probability table.
What is the probability that it is both cloudy and rainy?
Well, it's just the intersection of these two here, which is 0.08,
and the probability that it's cloudy and not raining is-- all right,
here's cloudy, here's not raining--
it's 0.32.
So it's 0.08 plus 0.32, which just gives us equal to 0.4.
That is the unconditional probability that it is, in fact, cloudy.
And so marginalization gives us a way to go
from these joint distributions to just some individual probability
that I might care about.
And you'll see a little bit later why it is that we care about that
and why that's actually useful to us as we
begin doing some of these calculations.
Last rule we'll take a look up before transitioning into something a little
bit different is this rule of conditioning--
very similar to the marginalization rule, but it says that, again,
if I have two events a and b--
but instead of having access to their joint probabilities,
I have access to their conditional probabilities,
how they relate to each other.
Well, again, if I want to know the probability that a happens and I know
that there's some other variable b, either b happens or b doesn't happen,
and so I can say that the probability of a
is the probability of a given b times the probability of b,
meaning b happened, and given that I know b happened,
what's the likelihood that a happened?
And then I consider the other case, that b didn't happen.
So here is the probability that b didn't happen,
and here's the probability that a happens given
that I know that b didn't happen.
And this is really the equivalent rule, just using conditional probability
instead of joint probability where I'm saying,
let's look at both of these two cases and condition on b.
Look at the case where b happens and look at the case where b doesn't happen
and look at what probabilities I get as a result.
And just as in the case of marginalization
where there was an equivalent rule for random variables
that could take on multiple possible values in a domain of possible values,
here, too, conditioning has the same equivalent rule.
Again, there's a summation to mean I'm summing over all of the possible values
that some random variable y could take on.
But if I want to know, what is the probability that x takes on this value,
then I'm going to sum up over all the values j
that y could take on and say, all right, what's
the chance that y takes on that value, yj, and multiply it
by the conditional probability that x takes on this value given
that y took on that value yj--
so equivalent rule just using conditional probabilities
instead of joint probabilities.
And using the equation we know about joint probabilities,
we can translate between these two.
All right, we've seen a whole lot of mathematics,
and we've just sort of laid the foundation for mathematics.
And no need to worry if you haven't seen probability in too much
detail up until this point.
These are sort of the foundations of the ideas
that are going to come up as we begin to explore how we can now
take these ideas from probability and begin to apply them to represent
something inside of our computer, something inside of the AI agent
we're trying to design that is able to represent information and probabilities
and the likelihoods between various different events.
So there are a number of different probabilistic models
that we can generate, but the first of the models we're going to talk about
are what are known as Bayesian networks.
And a Bayesian network is just going to be
some network of random variables, connected random variables,
that are going to represent the dependence
between these random variables.
And odds are most random variables in this world
are not independent from each other, that there's
some relationship between things that are happening that we care about.
If it is raining today, that might increase the likelihood
that my flight or my train gets delayed, for example.
There is some dependence between these random variables,
and a Bayesian network is going to be able to capture those dependencies.
So what is a Bayesian network?
What is its actual structure, and how does it work?
Well, a Bayesian network is going to be a directed graph.
And again, we've seen directed graphs before.
They are individual nodes with arrows or edges
that connect one node to another node, pointing in a particular direction.
And so this directed graph is going to have nodes,
as well, where each node in this directed graph
is going to represent a random variable, something like the weather or something
like whether my train was on time or delayed.
And we're going to have an arrow from a node x to a node y
to mean that x is a parent of y.
So that'll be our notation.
If there's an arrow from x to y, x is going to be considered a parent of y.
And the reason that's important is because each of these nodes
is going to have a probability distribution that we're
going to store along with it, which is the distribution of x given
some evidence, given the parents of x.
So the way to more intuitively think about this
is the parents are going to be thought of as sort of causes for some effect
that we're going to observe.
And so let's take a look at an actual example of a Bayesian network
and think about the types of logic that might be involved
in reasoning about that network.
Let's imagine, for a moment, that I have an appointment out of town
and I need to take a train in order to get to that appointment.
So what are the things I might care about?
Well, I care about getting to my appointment on time.
Either I make it to my appointment and I'm
able to attend it or I miss the appointment.
And you might imagine that that's influenced by the train,
that the train is either on time or it's delayed, for example.
But that train itself is also influenced.
Whether the train is on time or not depends maybe on the rain.
Is there no rain?
Is it light rain?
Is there heavy rain?
And it might also be influenced by other variables, too.
It might be influenced, as well, by whether
or not there's maintenance on the train track, for example.
If there is maintenance on the train track,
that probably increases the likelihood that my train is delayed.
And so we can represent all of these ideas using a Bayesian network that
looks a little something like this.
Here I have four nodes representing four random variables
that I would like to keep track of.
I have one random variable called Rain that
can take on three possible values in its domain, either none or light or heavy
for no rain, light rain, or heavy rain.
I have a variable called Maintenance for whether or not
there is maintenance on the train track, which it has two possible values, just
either yes or no.
Either there is maintenance or there is no maintenance happening on the track.
Then I have a random variable for the train indicating whether or not
the train was on time or not.
That random variable has two possible values in its domain.
The train is either on time or the train is delayed.
And then, finally, I have a random variable
for whether I make it to my appointment.
For my appointment down here, I have a random variable called Appointment
that itself has two possible values, attend and miss.
And so here are the possible values.
Here are my four nodes, each of which represents
a random variable, each of which has a domain of possible values
that it can take on.
And the arrows, the edges pointing from one node to another,
encode some notion of dependence inside of this graph,
that whether I make it to my appointment or not
is dependent upon whether the train is on time or delayed.
And whether the train is on time or delayed is dependent on two things,
given by the two arrows pointing at this node.
It is dependent on whether or not there was maintenance on the train track,
and it is also dependent upon whether or not
it was raining, or whether it is raining.
And just to make things a little complicated, let's
say, as well, that whether or not there's maintenance on the track,
this too might be influenced by the rain.
Then if there's heavier rain, well, maybe it's
less likely that there's going to be maintenance on the train track
that day because they're more likely to want
to do maintenance on the track on days when it's not raining, for example.
And so these nodes might have different relationships between them.
But the idea is that we can come up with a probability
distribution for any of these nodes based only upon its parents.
And so let's look node by node at what this probability distribution might
actually look like.
And we'll go ahead and begin with this root node, this Rain node here, which
is at the top and has no arrows pointing into it,
which means its probability distribution is not
going to be a conditional distribution.
It's not based on anything.
I just have some probability distribution over the possible values
for the Rain random variable.
And that distribution might look a little something like this.
None, light, and heavy-- each have a possible value.
Here I'm saying the likelihood of no rain
is 0.7, of light rain is 0.2, of heavy rain is 0.1, for example.
So here is a probability distribution for this root node in this Bayesian
network.
And let's now consider the next node in the network, Maintenance.
Track maintenance is yes or no.
And the general idea of what this distribution
is going to encode, at least in this story,
is the idea that the heavier the rain is, the less likely
it is that there's going to be maintenance on the track
because the people that are doing maintenance on the track
probably want to wait until a day when it's not as rainy in order to do
the track maintenance, for example.
And so what might that probability distribution look like?
Well, this now is going to be a conditional probability
distribution, that here are the three possible values for the Rain
random variable, which I'm here just going to abbreviate to R, either
no rain, light rain, or heavy rain.
And for each of those possible values, either there is yes track maintenance
or no track maintenance, and those have probabilities associated with them,
that I see here that if it is not raining,
then there is a probability 0.4 that there's track maintenance
and a probability of 0.6 that there isn't.
But if there's heavy rain, then here the chance
that there is track maintenance is 0.1 and the chance
that there is not track maintenance is 0.9.
Each of these rows is going to sum up to one because each of these
represent different values of whether or not
it's raining, the three possible values that that random variable can take on,
and each is associated with its own probability distribution.
That is ultimately all going to add up to the number one.
So that there is our distribution for this random variable called Maintenance
about whether or not there is maintenance on the train track.
And now let's consider the next variable.
Here we have a node inside of our Bayesian network
called Train that has two possible values, on time and delayed.
And this node is going to be dependent upon the two nodes that
are pointing towards it, that whether or not the train is on time
or delayed it depends on whether or not there is track maintenance,
and it depends on whether or not there is rain,
that heavier rain probably means more likely that my train is delayed.
And if there is track maintenance, that also
probably means it's more likely that my train is delayed as well.
And so you could construct a larger probability distribution,
a conditional probability distribution, that
instead of conditioning on just one variable, as was the case here,
is now conditioning on two variables, conditioning
both on rain, represented by R, and on maintenance, represented by yes.
Again, each of these rows has two values that sum up to the number one,
one for whether the train is on time, one for whether the train is delayed.
And here I can say something like, all right,
if I know there was light rain and track maintenance-- well, OK,
that would be R is light and M is yes--
well, then there is a probability of 0.6 that my train is on time
and a probability of 0.4 the train is delayed.
And you can imagine gathering this data just by looking at real-world data,
looking at data about, all right, if I knew that it was light rain
and there was track maintenance, how often
was a train delayed or not delayed, and you
could begin to construct this thing.
But the interesting thing is, intelligently,
being able to try to figure out, how might
you go about ordering these things?
What things might influence other nodes inside of this Bayesian network?
And the last thing I care about is whether or not
I make it to my appointment.
So did I attend or miss the appointment?
And ultimately, whether I attend or miss the appointment,
it is influenced by track maintenance because it's indirectly this idea
that, all right, if there is track maintenance, well,
then my train might more likely be delayed,
and if my train is more likely to be delayed,
then I'm more likely to miss my appointment.
But what we encode in this Bayesian network
are just what we might consider to be more direct relationships.
So the train has a direct influence on the appointment.
And given that I know whether the train is on time or delayed,
knowing whether there's track maintenance
isn't going to give me any additional information that I didn't already have,
that if I know train, these other nodes that are up above
isn't really going to influence the result.
And so here we might represent it using another conditional probability
distribution that looks a little something like this, that
train can take on two possible values.
Either my train is on time or my train is delayed.
And for each of those two possible values,
I have a distribution for what are the odds
that I'm able to attend the meeting, and what are the odds
that I missed the meeting?
And obviously, if my train is on time, I'm
much more likely to be able to attend the meeting
than if my train is delayed, in which case
I'm more likely to miss that meeting.
So all of these nodes put altogether here
represent this Bayesian network, this network of random variables
whose values I ultimately care about and that
have some sort of relationship between them,
some sort of dependence where these arrows from one node to another
indicate some dependence, that I can calculate the probability of some node
given the parents that happen to exist there.
So now that we've been able to describe the structure of this Bayesian network
and the relationships between each of these nodes,
by associating each of the node in the network with a probability
distribution, whether that's an unconditional probability
distribution in the case of this root node here,
like Rain, and a conditional probability distribution,
in the case of all of the other nodes whose probabilities are
dependent upon the values of their parents,
we can begin to do some computation and calculation using the information
inside of that table.
So let's imagine, for example, that I just
wanted to compute something simple, like the probability of light rain.
How would I get the probability of light rain?
Well, light rain-- rain here is a root node.
And so if I wanted to calculate that probability,
I could just look at the probability distribution for rain
and extract from it the probability of light rain.
It's just a single value that I already have access to.
But we could also imagine wanting to compute
more complex joint probabilities, like the probability
that there is light rain and also no track maintenance.
This is a joint probability of two values, light rain and no track
maintenance.
And the way I might do that is first by starting by saying, all right, well,
let me get the probability of light rain, but now
I also want the probability of no track maintenance.
But, of course, this node is dependent upon the value of rain.
So what I really want is the probability of no track maintenance given
that I know that there was light rain.
And so the expression for calculating this idea
that the probability of light rain and no track maintenance
is really just the probability of light rain and the probability
that there is no track maintenance given that I know that there already
is light rain.
So I take the unconditional probability of light rain,
multiply it by the conditional probability of no track maintenance
given that I know there is light rain.
And you can continue to do this again and again for every variable
that you want to add into this joint probability
that I might want to calculate.
If I wanted to know the probability of light rain
and no track maintenance and a delayed train,
well, that's going to be the probability of light rain multiplied
by the probability of no track maintenance
given light rain multiplied by the probability of a delayed train given
light rain and no track maintenance, because whether the train is on time
or delayed is dependent upon both of these other two variables,
and so I have two pieces of evidence that
go into the calculation of that conditional probability.
And each of these three values is just a value
that I can look up by looking at one of these individual probability
distributions that is encoded into my Bayesian network.
And if I wanted a joint probability over all four of the variables,
something like the probability of light rain
and no track maintenance and a delayed train and I missed my appointment,
well, that's going to be multiplying four different values, one
from each of these individual nodes.
It's going to be the probability of light rain,
then of no track maintenance given light rain,
then of a delayed train given light rain and no track maintenance.
And then, finally, for this node here for whether I make it to my appointment
or not, it's not dependent upon these two variables given that I know
whether or not the train is on time.
I only need to care about the conditional probability
that I miss my appointment given that the train happens to be delayed.
And so that's represented here by four probabilities, each of which
is located inside of one of these probability distributions
for each of the nodes, all multiplied together.
And so I can take a variable like that and figure out
what the joint probability is by multiplying
a whole bunch of these individual probabilities
from the Bayesian network.
But, of course, just as with last time where what I really wanted to do
was to be able to get new pieces of information,
here, too, this is what we're going to want to do with our Bayesian network.
In the context of knowledge, we talked about the problem of inference.
Given things that I know to be true, can I
draw conclusions, make deductions about other facts about the world
that I also know to be true?
And what we're going to do now is apply the same sort of idea to probability.
Using information about which I have some knowledge,
whether some evidence or some probabilities, can
I figure out not other variables for certain,
but can I figure out the probabilities of other variables taking
on particular values?
And so here we introduce the problem of inference
in a probabilistic setting in a case where variables might not necessarily
be true for sure, but they might be random variables
that take on different values with some probability.
So how do we formally define what exactly this inference problem actually
is?
Well, the inference problem has a couple of parts to it.
We have some query, some variable x that we
want to compute the distribution for.
Maybe I want the probability that I missed my train
or I want the probability that there is track maintenance, something
that I want information about.
And then I have some evidence variables.
Maybe it's just one piece of evidence.
Maybe it's multiple pieces of evidence.
But I've observed certain variables for some sort of event.
So for example, I might have observed that it is raining.
This is evidence that I have.
I know that there is light rain or I know that there is heavy rain,
and that is evidence I have.
And using that evidence, I want to know, what is the probability
that my train is delayed, for example?
And that is a query that I might want to ask based on this evidence.
So I have a query, some variable, evidence,
which are some other variables that I have observed inside
of my Bayesian network, and of course that
does leave some hidden variables, y.
These are variables that are not evidence variables and not
query variables.
So you might imagine in the case where I know whether or not it's raining
and I want to know whether my train is going to be delayed or not,
the hidden variable, the thing I don't have access to, is something like,
is there maintenance on the track, or am I
going to make or not make my appointment, for example?
These are variables that I don't have access to.
They're hidden because they're not things I observed,
and they're also not the query, the thing that I'm asking.
And so ultimately what we want to calculate
is I want to know the probability distribution of x given
e, the event that I observed.
So given that I observed some event, I observed that it is raining,
I would like to know, what is the distribution over the possible values
of the Train random variable?
Is it on time?
Is it delayed?
What is the likelihood it's going to be there?
And it turns out we can do this calculation just using
a lot of the probability rules that we've already seen in action.
And ultimately, we're going to take a look at the math
at a little bit of a high level, at an abstract level,
but ultimately we can allow computers and programming
libraries that already exist to begin to do some of this math for us.
But it's good to get a general sense for what's actually happening when
this inference process takes place.
Let's imagine, for example, that I want to compute the probability
distribution of the Appointment random variable
given some evidence, given that I know that there was light rain and no track
maintenance.
So there's my evidence, these two variables that I observed the value of.
I observe the value of rain.
I know there's light rain.
And I know that there is no track maintenance going on today.
And what I care about knowing, my query, is this random variable Appointment.
I want to know the distribution of this random variable Appointment.
What is the chance that I am able to attend
my appointment, what is the chance that I miss my appointment
given this evidence?
And the hidden variable, the information that I don't have access to,
is this variable Train.
This is information that is not part of the evidence that I see,
not something that I observe.
But it is also not the query that I am asking for.
And so what might this inference procedure look like?
Well, if you recall back from a when we were defining conditional probability
and doing math with conditional probabilities,
we know that a conditional probability is
proportional to the joint probability.
And we remember this by recalling that the probability of a given b
is just some constant factor alpha multiplied
by the probability of a and b.
That constant factor alpha turns up and you're
dividing over the probability of b, but the important thing
is that it's just some constant multiplied
by the joint distribution, the probability
that all of these individual things happen.
So in this case, I can take the probability
of the Appointment random variable given light rain and no track maintenance
and say that is just going to be proportional, some constant alpha,
multiplied by the joint probability, the probability of a particular value
for the appointment random variable, and light rain and no track maintenance.
Well, all right, how do I calculate this, probability
of appointment and light rain and no track maintenance,
when what I really care about is knowing--
I need all four of these values to be able to calculate a joint distribution
across everything, because, then, a particular appointment
depends upon the value of train.
Well, in order to do that, here I can begin
to use that marginalization trick, that there are only
two ways I can get any configuration of an appointment, light rain,
and no track maintenance.
Either this particular setting of variables
happens and the train is on time or this particular setting of variables happens
and the train is delayed.
Those are two possible cases that I would want to consider.
And if I add those two cases up, well, then I
get the result just by adding up all of the possibilities for the hidden
variable, or variables if there are multiple.
But since there's only one hidden variable here,
Train, all I need to do is iterate over all the possible values for that hidden
variable Train and add up their probabilities.
So this probability expression here becomes
probability distribution over Appointment, light, no rain, and train
is on time, and the probability distribution
over the Appointment, light rain, no track maintenance, and the train
is delayed, for example.
So I take both of the possible values for train, go ahead and add them up.
These are just
Joint probabilities that we saw earlier how to calculate just
by going parent, parent, parent, parent and calculating those probabilities
and multiplying them together.
And then you'll need to normalize them at the end, speaking at a high level
to make sure that everything adds up to the number one.
So the formula for how you do this and a process known
as inference by enumeration looks a little bit complicated,
but ultimately it looks like this.
And let's now try to distill what it is that all of these symbols
actually mean.
Let's start here.
What I care about knowing is the probability of x, my query variable,
given some sort of evidence.
What do I know about conditional probabilities?
Well, a conditional probability is proportional to the joint probability.
So we had some alpha, some normalizing constant,
multiplied by this joint probability of x and evidence.
And how do I calculate that?
Well, to do that, I'm going to marginalize over
all of the hidden variables.
All the variables that I don't directly observe the values for,
I'm basically going to iterate over all of the possibilities
that it could happen and just sum them all up.
And so I can translate this into a sum over all y, which
ranges over all the possible hidden variables and the values
that they could take on, and adds up all of those possible individual
probabilities.
And that is going to allow me to do this process of inference by enumeration.
And ultimately, it's pretty annoying if we as humans have
to do all of this math for ourselves.
But it turns out this is where computers and AI can be particularly helpful,
that we can program a computer to understand a Bayesian network to be
able to understand these inference procedures
and to be able to do these calculations.
And using the information you've seen here,
you could implement a Bayesian network from scratch yourself.
But turns out there are a lot of libraries, especially written
in Python, that allow us to make it easier
to do this sort of probabilistic inference
to be able to take a Bayesian network and do these sorts of calculations
so that you don't need to know and understand all of the underlying math,
though it's helpful to have a general sense for how it works.
But you just need to be able to describe the structure of the network
and make queries in order to be able to produce the result.
And so let's take a look at an example of that right now.
It turns out that there are a lot of possible libraries
that exist in Python for doing this sort of inference.
It doesn't matter too much which specific library you use.
They all behave in fairly similar ways.
But the library I'm going to use here is one known as pomegranate.
And here inside of model.py, I have defined a Bayesian network
just using the structure and the syntax that the pomegranate library expects.
And what I'm effectively doing is just, in Python,
creating nodes to represent each the nodes of the Bayesian network
that you saw me describe a moment ago.
So here on line four, after I've imported pomegranate,
I'm defining a variable called rain that is going to represent
a node inside of my Bayesian network.
It's going to be a node that follows this distribution where
there are three possible values--
none for no rain, light for light rain, heavy for heavy rain.
And these are the probabilities of each of those taking place.
0.7 is the likelihood of no rain, 0.2 for light rain, 0.1 for heavy rain.
Then, after that, we go to the next variable,
the variable for track maintenance, for example, which
is dependent upon that rain variable.
And this, instead of being an unconditional distribution,
is a conditional distribution, as indicated by a conditional probability
table here.
And the idea is that this is conditional on the distribution of rain.
So if there is no rain, then the chance that there is yes
track maintenance is 0.4.
If there's no rain, the chance that there is no track maintenance is 0.6.
Likewise, for light rain, I have a distribution.
For heavy rain, I have a distribution, as well.
But I'm effectively encoding the same information
you saw represented graphically a moment ago,
but I'm telling this Python program that the maintenance
node obeys this particular conditional probability distribution.
And we do the same thing for the other random variables, as well.
Train was a node inside my distribution that was a conditional probability
table with two parents.
It was dependent not only on rain, but also on track maintenance.
And so here I'm saying something like, given that there is no rain and yes
track maintenance, the probability that my train is on time is 0.8,
and the probability that it's delayed is 0.2.
And likewise, I can do the same thing for all
of the other possible values of the parents of the train node
inside of my Bayesian network by saying, for all of those possible values,
here is the distribution that the train node should follow.
And I do the same thing for an appointment
based on the distribution of the variable Train.
Then, at the end, what I do is actually construct this network
by describing what the states of the network are
and by adding edges between the dependent nodes.
So I create a new Bayesian network, add states to it--
one for rain, one for maintenance, one for train, one for the appointment--
and then I add edges connecting the related pieces.
Rain has an arrow to maintenance because rain influences track maintenance,
rain also influences the train, maintenance also influences the train,
and train influences whether I make it to my appointment,
and bake just finalizes the model and does some additional computation.
So the specific syntax of this is not really the important part.
Pomegranate just happens to be one of several different libraries
that can all be used for similar purposes,
and you could describe and define a library for yourself
that implemented similar things.
But the key idea here is that someone can
design a library for a general Bayesian network that
has nodes that are based upon its parents,
and then all a programmer needs to do, using one of those libraries,
is to define what those nodes and what those probability distributions are,
and we can begin to do some interesting logic based on it.
So let's try doing that conditional or joint probability
calculation that we saw us do by hand before by going into likelihood.py
where here I'm importing the model that I justified a moment ago.
And here I'd just like to calculate model.probability,
which calculates the probability for a given observation,
and I'd like to calculate the probability of no rain,
no track maintenance, my train is on time,
and I'm able to attend the meeting--
so sort of the optimal scenario, that there's
no rain and no maintenance on the track, my train is on time,
and I'm able to attend the meeting.
What is the probability that all of that actually happens?
And I can calculate that using the library
and just print out its probability.
And so I'll go ahead and run Python of likelihood.py,
and I see that, OK, the probability is about 0.34.
So about a third of the time, everything goes right for me, in this case--
no rain, no track maintenance, train is on time,
and I'm able to attend the meeting.
But I could experiment with this, try and calculate other probabilities
as well.
What's the probability that everything goes right up until the train
but I still miss my meeting-- so no rain, no track maintenance, train
is on time, but I miss the appointment.
Let's calculate that probability, and that has a probability of about 0.04.
So about 4% of the time the train will be on time,
there won't be any rain, no track maintenance,
and yet I'll still miss the meeting.
And so this is really just an implementation
of the calculation of the joint probabilities that we did before.
What this library is likely doing is first
figuring out the probability of no rain, then figuring
that the probability of no track maintenance
given no rain, then the probability that my train is
on time given both of these values, and then the probability
that I miss my appointment given that I know that the train was on time.
So this, again, is the calculation of that joint probability.
And turns out we can also begin to have our computer solve inference problems,
as well, to begin to infer, based on information, evidence
that we see, what is the likelihood of other variables also being true?
So let's go into inference.py, for example, where here I'm,
again, importing that exact same model from before,
importing all the nodes and all the edges
and the probability distribution that is encoded there, as well.
And now there's a function for doing some sort of prediction.
And here, into this model, I pass in the evidence that I observe.
So here I've encoded into this Python program the evidence
that I have observed.
I have observed the fact that the train is delayed,
and that is the value for one of the four random variables
inside of this Bayesian network.
And using that information, I would like to be
able to draw inspiration and figure out inferences
about the values of the other random variables
that are inside of my Bayesian network.
I would like to make predictions about everything else.
So all of the actual computational logic is happening in just these three lines
where I'm making this call to this prediction.
Down below, I'm just iterating over all of the states and all the predictions
and just printing them out so that we can visually see what the results are.
But let's find out, given the train is delayed,
what can I predict about the values of the other random variables?
Let's go ahead and run Python inference.py.
I run that.
And all right, here is the result that I get.
Given the fact that I know that the train is delayed--
this is evidence that I have observed--
well, given that there is a 45% chance or a 46% chance
that there was no rain, a 31% chance there
was light rain, a 23% chance there was heavy rain,
I can see a probability distribution over track maintenance
and a probability distribution over whether I'm
able to attend or miss my appointment.
Now, we know that whether I attend or miss
the appointment, that is only dependent upon the train being delayed
or not delayed.
It shouldn't depend on anything else.
So let's imagine, for example, that I knew that there was heavy rain.
That shouldn't affect the distribution for making the appointment.
And indeed, if I go up here and add some evidence,
say that I know that the value of rain is heavy--
that is evidence that I now have access to.
I now have two pieces of evidence.
I know that the rain is heavy, and I know that my train is delayed.
I can calculate the probability by running this inference procedure again
and seeing the result.
I know that the rain is heavy.
I know my train is delayed.
The probability distribution for track maintenance changed.
Given that I know that there is heavy rain,
now it's more likely that there is no track maintenance, 88% as
opposed to 64% from here before.
And now what is the probability that I make the appointment?
Well, that's the same as before.
It's still going to be attend the appointment with probability 0.6,
miss the appointment with probability 0.4,
because it was only dependent upon whether or not my train was on time
or delayed.
And so this here is implementing that idea of that inference algorithm
to be able to figure out, based on the evidence
that I have, what can we infer about the values of the other variables that
exist as well?
So inference by enumeration is one way of doing this inference procedure,
just looping over all of the values the hidden variables could take on
and figuring out what the probability is.
Now, it turns out this is not particularly efficient,
and there are definitely optimizations you can make by avoiding repeated work
if you're calculating the same sort of probability multiple times.
There are ways of optimizing the program to avoid having to recalculate
the same probabilities again and again.
But even then, as the number of variables
get large, as the number of possible values those variables could take on
get large, we're going to start to have to do
a lot of computation, a lot of calculation,
to be able to do this inference.
And at that point, you might start to get
unreasonable in terms of the amount of time
that it would take to be able to do this sort exact inference.
And it's for that reason that oftentimes when
it comes towards probability and things we're not entirely sure about,
we don't always care about doing exact inference
and knowing exactly what the probability is.
But if we can approximate the inference procedure,
do some sort of approximate inference, that that can be pretty good as well,
that if I don't know the exact probability
but I have a general sense for the probability,
that I can get increasingly accurate with more time, that that's probably
pretty good, especially if I can get that to happen even faster.
So how could I do approximate inference inside of a Bayesian network?
Well, one method is through a procedure known as sampling.
In the process of sampling, I'm going to take a sample of all of the variables
inside of this Bayesian network here.
And how am I going to sample?
Well, I'm going to sample one of the values from each of these nodes
according to their probability distribution.
So how might I take a sample of all these nodes?
Well, I'll start at the root.
I'll start with rain.
Here's the distribution for rain, and I'll go ahead
and, using a random number generator or something like it,
randomly pick one of these three values.
I'll pick none with probability 0.7, light with probability 0.2,
and heavy with probability 0.1.
So I'll randomly just pick one of them according to that distribution,
and maybe, in this case, I pick none, for example.
Then I do the same thing for the other variable.
Maintenance also as a probability distribution.
And I am going to sample--
now, there are three probability distributions here,
but I'm only going to sample from this first row
here because I've observed already in my sample that the value of rain is none.
So
Given that rain is none, I'm going to sample from this distribution to say,
all right, what should the value of maintenance be?
And in this case, maintenance is going to be, let's just
say, yes, which happens 40% of the time in the event that there is no rain,
for example.
And we'll sample all of the rest of the nodes in this way,
as well, that I want to sample from the train distribution,
and I'll sample from this first row here where there is no rain,
but there is track maintenance.
And I'll sample 80% of the time, I'll say the train is on time.
20% of the time, I'll say the train is delayed.
And finally, we'll do the same thing for whether I make it to my appointment
or not.
Did I attend or miss the appointment?
We'll sample based on this distribution and maybe say that in this case
I attend the appointment, which happens 90% of the time when
the train is actually on time.
So by going through these nodes, I can very quickly just do some sampling
and get a sample of the possible values that
could come up from going through this entire Bayesian network
according to those probability distributions.
And where this becomes powerful is if I do this not once,
but I do this thousands or tens of thousands of times
and generate a whole bunch of samples, all using this distribution.
I get different samples.
Maybe some of them are the same.
But I get a value for each of the possible variables that could come up.
And so then, if I'm ever faced with a question, a question like,
what is the probability that the train is on time,
you could do an exact inference procedure.
This is no different than the inference problem we had before
where I could just marginalize, look at all the possible other values
of the variables and do the computation of inference
by enumeration to find out this probability exactly.
But I could also, if I don't care about the exact probability, just sample it.
Approximate it to get close.
And this is a powerful tool in AI where we
don't need to be right 100% of the time or we don't need to be exactly right.
If we just need to be right with some probability,
we can often do some more effectively, more efficiently.
And so here, now, are all of those possible samples.
I'll sort of highlight the ones where the train is on time.
I'm ignoring the ones where the train is delayed.
And in this case, there's six out of eight
of the samples have the train is arriving on time.
And so maybe, in this case, I can say that, in six out of eight cases,
that's the likelihood that the train is on time.
And with eight samples, that might not be a great prediction.
But if I had thousands upon thousands of samples,
then this could be a much better inference procedure
to be able to do these sorts of calculations.
So this is a direct sampling method to just do a bunch of samples
and then figure out what the probability of some event is.
Now, this from before was an unconditional probability.
What is the probability that the train is on time?
And I did that by looking at all the samples and figuring out,
right here, the ones where the train is on time.
But sometimes what I'll want to calculate
is not an unconditional probability, but rather a conditional probability,
something like, what is the probability that there
is light rain given that the train is on time, something to that effect.
And to do that kind of calculation, well, what I might do is here
are all the samples that I have, and I want
to calculate a probability distribution given
that I know that the train is on time.
So to be able to do that, I can kind of look at the two cases
where the train was delayed and ignore or reject them,
sort of exclude them from the possible samples that I'm considering.
And now I want to look at these remaining cases
where the train is on time.
Here are the cases where there is light rain.
And now I say, OK, these are two out of the six possible cases.
That can give me an approximation for the probability
of light rain given the fact that I know the train was on time.
And I did that in almost exactly the same way
just by adding an additional step, by saying that,
all right, when I take each sample, let me
reject all of the samples that don't match my evidence
and only consider the samples that do match
what it is that I have in my evidence that I want
to make some sort of calculation about.
And it turns out, using the libraries that we've had for Bayesian networks,
we can begin to implement this same sort of idea,
implement rejection sampling, which is what this method is called,
to be able to figure out some probability, not via direct inference,
but instead by sampling.
So what I have here is a program called sample.py--
imports the exact same model.
And what I define first is a program to generate a sample.
And the way I generate a sample is just by looping over all of the states.
The states need to be in some sort of order
to make sure I'm looping in the correct order.
But effectively, if it is a conditional distribution,
I'm going to sample based on the parents.
And otherwise, I'm just going to directly sample the variable,
like rain, which has no parents-- it's just an unconditional distribution--
and keep track of all those parent samples and return the final sample.
The exact syntax of this, again, not particularly important.
It just happens to be part of the implementation
details of this particular library.
The interesting logic is done below.
Now that I have the ability to generate a sample,
if I want to know the distribution of the appointment random variable given
that the train is delayed, well, then I can begin to do calculations like this.
Let me take 10,000 samples and assemble all my results
in this list called data.
I'll go ahead and loop n times-- in this case, 10,000 times.
I'll generate a sample, and I want to know the distribution of appointment
given that the train is delayed.
So according to rejection sampling, I'm only
going to consider samples where the train is delayed.
If the train's not delayed, I'm not going to consider those values at all.
So I'm going to say, all right, if I take the sample,
look at the value of the train random variable,
if the train is delayed, well, let me go ahead and add to my data
that I'm collecting the value of the appointment random variable
that it took on in this particular sample.
So I'm only considering the samples where the train is delayed
and, for each of those samples, considering
what the value of appointment is.
And then at the end, I'm using a Python class called counter,
which quickly counts up all the values inside
of a data set so I can take this list of data
and figure out how many times was my appointment made,
and how many times was my appointment missed?
And so this here, with just a couple of lines of code,
is an implementation of rejection sampling.
And I can run it by going ahead and running Python sample.py.
And when I do that, here is the result I get.
This is the result of the counter.
1,251 times I was able to attend the meeting,
and 856 times I was able to miss the meeting.
And you can imagine, by doing more and more samples,
I'll be able to get a better and better, more accurate result.
And this is a randomized process.
It's going to be an approximation of the probability.
If I run it a different time, you'll notice the numbers are similar--
1,272 and 905-- but they're not identical
because there's some randomization, some likelihood that things
might be higher or lower, and so this is why we generally want to try and use
more samples so that we can have a greater amount of confidence
in our result, be more sure about the result
that we're getting of whether or not it accurately reflects or represents
the actual underlying probabilities that are
inherent inside of this distribution.
And so this, then, was an instance of rejection sampling.
And it turns out, there are a number of other sampling methods
that you could use to begin to try to sample.
One problem that rejection sampling has is
that if the evidence you're looking for is a fairly unlikely event, well,
you're going to be rejecting a lot of samples.
Like, if I'm looking for the probability of x given some evidence e,
if e is very unlikely to occur-- like, occurs maybe one every 1,000 times--
then I'm only going to be considering one out of every 1,000 samples
that I do, which is a pretty inefficient method for trying
to do this sort of calculation.
I'm throwing away a lot of samples, and it takes computational effort
to be able to generate those samples, so I'd
like to not have to do something like that.
So there are other sampling methods that can try and address this.
One such sampling method is called likelihood weighting.
In likelihood weighting, we follow a slightly different procedure,
and the goal is to avoid needing to throw out
samples that didn't match the evidence.
And so what we'll do is we'll start by fixing the values for the evidence
variables.
Rather than sample everything, we're going
to fix the values of the evidence variables and not sample those.
Then we're going to sample all the other non-evidence variables in the same way,
just using the Bayesian network, looking at the probability distributions,
sampling all the non-evidence variables.
But then what we need to do is weight each sample by its likelihood.
If our evidence is really unlikely, we want
to make sure that we've taken into account, how likely was the evidence
to actually show up in the sample?
If I have a sample where the evidence was much more
likely to show up than another sample, then I
want to weight the more likely one higher.
So we're going to weight each sample by its likelihood
where likelihood is just defined as the probability of all of the evidence.
Given all the evidence we have, what is the probability
that it would happen in that particular sample?
So before, all of our samples were weighted equally.
They all had a weight of one when we were calculating the overall average.
In this case, we're going to weight each sample,
multiply each sample by its likelihood in order
to get the more accurate distribution.
So what would this look like?
Well, if I asked the same question, what is the probability of light rain given
that the train is on time, when I do the sampling procedure
and start by trying to sample, I'm going to start
by fixing the evidence variable.
I'm already going to have in my sample the train is on time.
That way, I don't have to throw out anything.
I'm only sampling things where I know the value of the variables that
are my evidence are what I expect them to be.
So I'll go ahead and sample from rain, and maybe this time I
sample light rain instead of no rain.
Then I'll sample from track maintenance and say maybe, yes, there's
track maintenance.
Then for train, well, I've already fixed it in place.
Train was an evidence variable, so I'm not going to bother sampling again.
I'll just go ahead and move on.
I'll move on to appointment and go ahead and sample from appointment as well.
So now I've generated a sample.
I've generated a sample by fixing this evidence variable
and sampling the other three.
And the last step is now weighting the sample.
How much weight should it have?
And the weight is based on how probable is it that the train was actually
on time, this evidence actually happened,
given the values of these other variables, light rain and the fact
that, yes, there was track maintenance?
Well, to do that, I can just go back to the train variable
and say, all right, if there was light rain and track
maintenance, the likelihood of my evidence,
the likelihood that my train was on time, is 0.6.
And so this particular sample would have a weight of 0.6.
And I could repeat the sampling procedure again and again.
Each time, every sample would be given a weight
according to the probability of the evidence that I see associated with it.
And there are other sampling methods that exist, as well, but all of them
are designed to try and get at the same idea,
to approximate the inference procedure of figuring out
the value of a variable.
So we've now dealt with probability as it
pertains to particular variables that have these discrete values.
But what we haven't really considered is how
values might change over time, that we've considered something
like a variable for rain where rain can take on values of none or light
rain or heavy rain, but, in practice, usually when
we consider values for variables like rain, we like to consider it for,
over time, how do the values of these variables change?
What do we deal with when we're dealing with uncertainty over a period of time?
Which can come up in the context of weather, for example--
if I have sunny days and I have rainy days.
And I'd like to know not just what is the probability that it's raining now,
but what is the probability that it rains tomorrow or the day
after that or the day after that?
And so to do this, we're going to introduce
a slightly different kind of model.
But here we're going to have a random variable, not just one for the weather,
but for every possible time step.
And you can define time step however you like.
A simple way is just to use days as your time step.
And so we can define a variable called x sub t, which
is going to be the weather at time t.
So x sub zero might be the weather on day zero,
x sub one might be the weather on day one, so on and so forth,
x sub two is the weather on day two.
But as you can imagine, if we start to do
this over longer and longer periods of time,
there's an incredible amount of data that might go into this.
If you're keeping track of data about the weather for a year,
now suddenly you might be trying to predict the weather tomorrow given
365 days of previous pieces of evidence, and that's
a lot of evidence to have to deal with and manipulate and calculate.
Probably nobody knows what the exact conditional probability
distribution is for all of those combinations of variables.
And so when we're trying to do this inference inside of a computer, when
we're trying to reasonably do this sort of analysis,
it's helpful to make some simplifying assumptions,
some assumptions about the problem that we can just assume
are true to make our lives a little bit easier.
Even if they're not totally accurate assumptions,
if they're close to accurate or approximate,
they're usually pretty good.
And the assumption we're going to make is called the Markov assumption,
which is the assumption that the current state depends only on a finite fixed
number of previous states.
So the current day's weather depends not on all of the previous day's weather
for all of history, but the current day's weather I
can predict just based on yesterday's weather
or just based on the last two days' weather or the last three days'
weather.
But oftentimes, we're going to deal with just the one previous state helps
to predict this current state.
And by putting a whole bunch of these random variables
together, using this Markov assumption, we
can create what's called a Markov chain where a Markov chain is just
some sequence of random variables where each of the variable's distribution
follows that Markov assumption.
And so we'll do an example of this where the Markov assumption is
I can predict the weather.
Is it sunny or rainy?
And we'll just consider those two possibilities for now,
even though there are other types of weather.
But I can predict each day's weather just on the prior day's weather.
Using today's weather, I can come up with a probability distribution
for tomorrow's weather.
And here's what this weather might look like.
It's formatted in terms of a matrix, as you might describe it,
as sort of rows and columns of values where on the left-hand side
I have today's webinar, represented by the variable x sub t.
And then over here in the columns, I have tomorrow's weather,
represented by the variable x sub t plus one, t plus one day's weather instead.
And what this matrix is saying is if today is sunny,
well, then, it's more likely than not that tomorrow is also sunny.
Oftentimes the weather stays consistent for multiple days in a row.
And for example, let's say that if today is sunny,
our model says that tomorrow, with probability 0.8, it will also be sunny,
and with probability 0.2 it will be raining.
And likewise, if today is raining, then it's
more likely than not that tomorrow is also raining.
With probability 0.7, it'll be raining.
With probability 0.3, it will be sunny.
So this matrix, this description of how it
is we transition from one state to the next state,
is what we're going to call the transition model.
And using the transition model, you can begin
to construct this Markov chain by just predicting, given today's weather,
what's the likelihood of tomorrow's weather happening?
And you can imagine doing a similar sampling
procedure where you take this information,
you sample what tomorrow's weather is going to be,
using that you sample the next day's weather,
and the result of that is you can form this Markov chain of x zero,
time day zero is sunny, the next day is sunny,
maybe the next day it changes to raining, then raining, then raining.
And the pattern that this Markov chain follows,
given the distribution that we had access to,
this transition model here, is that when it's sunny,
it tends to stay sunny for a little while.
The next couple days tend to be sunny too.
And when it's raining, it tends to be raining as well.
And so you get a Markov chain that looks like this.
And you can do analysis on this.
You can say, given that today is raining,
what is the probability that tomorrow it's raining,
or you can begin to ask probability questions,
like what is the probability of this sequence of five values-- sun, sun,
rain, rain, rain--
and answer those sorts of questions too.
And it turns out there are, again, many Python libraries for interacting
with models like this of probabilities that have distributions
and random variables that are based on previous variables
according to this Markov assumption.
And pomegranate 2 has ways of dealing with these sorts of variables.
So I'll go ahead and go into the chain directory
where I have some information about Markov chains.
And here I've defined a file called model.py where
I've defined in a very similar syntax.
And again, the exact syntax doesn't matter so much as the idea
that I'm encoding this information into a Python program
so that the program access to these distributions.
I've here defined some starting distributions.
So every Markov model begins at some point in time,
and I need to give it some starting distribution.
And so we'll just say, you know what, to start,
you can pick 50/50 between sunny and rainy.
We'll say it's sunny 50% the time, rainy 50% of the time.
And then down below, I've here defined the transition model,
how it is that I transition from one day to the next.
And here I've encoded that exact same matrix from before,
that if it was sunny today, then with probability 0.8
it will be sunny tomorrow, and it will be raining tomorrow
with probability 0.2.
And I likewise have another distribution for if it was raining today instead.
And so that alone defines the Markov model.
You can begin to answer questions using that model.
But one thing I'll just do is sample from the Markov chain.
And it turns out there is a method built into this Markov chain library that
allows me to sample 50 states from the chain,
basically just simulating 50 instances of weather.
And so let me go ahead and run this, Python model.py.
And when I run it, what I get is it is going to sample from this Markov chain
50 states, 50 days worth of weather that it's just going to randomly sample.
And you can imagine sampling many times to be able to get more data
to be able to do more analysis.
But here, for example, it's sunny two days
a row, rainy a whole bunch of days in a row before it changes back to sun.
And so you get this model that follows the distribution that we originally
described, that follows the distribution of sunny days
tend to lead to more sunny days, rainy days tend to lead to more rainy days.
And that, then, is the Markov model.
And Markov models rely on us knowing the values of these individual states.
I know that today is sunny or that today is rainy, and using that information,
I can draw some sort of inference about what tomorrow is going to be like.
But in practice, this often isn't the case.
It often isn't the case that I know for certain
what the exact state of the world is.
Oftentimes the state of the world is exactly unknown,
but I'm able to somehow sense some information about that state
that a robot or an AI doesn't have exact knowledge about the world around it,
but it has some sort of sensor, whether that sensor
is a camera or sensors that detect distance
or just a microphone that is sensing audio, for example.
It is sensing data, and using that data, that data is somehow
related to the state of the world even if it doesn't actually
know, our AI doesn't know, what the underlying true state of the world
actually is.
And for that, we need to get into the world of sensor models,
the way of describing how it is that we translate
what the hidden state, the underlying true state of the world
is with what the observation, what it is that the AI knows or the AI has access
to, actually is.
And so for example, a hidden state might be a robot's position.
If a robot is exploring new, uncharted territory,
the robot likely doesn't know exactly where it is.
But it does have an observation.
It has robot sensor data where it can sense
how far away are possible obstacles around it, and using that information,
using the observed information that it has,
it can infer something about the hidden state,
because what the true hidden state is influences those observations.
Whatever the robot's true position is affects
or has some effect upon what the sensor data the robot is able to collect
is, even if the robot doesn't actually know for certain
what its true position is.
Likewise, if you think about a voice recognition or a speech recognition
program that listens to you and is able to respond to you, something like Alexa
or what Apple and Google are doing with their voice recognition as well,
that you might imagine that the hidden state, the underlying state,
is what words are actually spoken.
The true nature of the world contains you
saying a particular sequence of words.
But your phone or your smart home device doesn't know for sure
exactly what words you said.
The only observation that the AI has access to is some audio wave forms.
And those audio wave forms are, of course, dependent
upon this hidden state, and you can infer,
based on those audio wave forms, what the words spoken likely were,
but you might not know with 100% certainty what
that hidden state actually is.
And it might be a task to try and predict.
Given this observation, given these audio away forms,
can you figure out what the actual words spoken are?
Likewise, you might imagine on a website.
True user engagement might be information you don't directly
have access to, but you can observe data, like website or app
analytics about how often was this button clicked
or how often are people interacting with a page in a particular way.
And you can use that to infer things about your users as well.
So this type of problem comes up all the time when we're dealing with AI
and trying to infer things about the world,
that often AI doesn't really know the hidden true state of the world.
All that AI has access to is some observation
that is related to the hidden true state, but it's not direct.
There might be some noise there.
The audio wave form might have some additional noise
that might be difficult to parse.
The sensor data might not be exactly correct.
There's some noise that might not allow you
to conclude with certainty what the hidden state is, but can
allow you to infer what it might be.
And so the simple example we'll take a look at here
is imagining the hidden state as the weather, whether it's sunny or rainy
or not, and imagine you are programming an AI inside of a building that maybe
has access to just a camera to inside the building,
and all you have access to is an observation as to
whether or not employees are bringing an umbrella into the building or not.
You can detect whether it's an umbrella or not,
and so you might have an observation as to whether or not an umbrella is
brought into the building or not.
And using that information, you want to predict
whether it's sunny or rainy, even if you don't
know what the underlying weather is.
So the underlying weather might be sunny or rainy.
And if it's raining, obviously people are more likely to bring an umbrella.
And so whether or not people bring an umbrella,
your observation tells you something about the hidden state.
And of course, this is a bit of a contrived example,
but the idea here is to think about this more broadly
in terms of more generally, any time you observe something,
it having to do with some underlying hidden state.
And so to try and model this type of idea where we have these hidden states
and observations, rather than just use a Markov model, which
has state, state, state, state, each of which
is connected by that transition matrix that we described before,
we're going to use what we call a hidden Markov model--
very similar to a Markov model, but this is
going to allow us to model a system that has hidden states
that we don't directly observe along with some observed event
that we do actually see.
And so in addition to that transition model
that we still need of saying, given the underlying state of the world,
if it's sunny or rainy, what's the probability of tomorrow's weather,
we also need another model, that given some state is
going to give us an observation of green,
yes, someone brings an umbrella into the office, or red,
no, nobody brings umbrellas into the office.
And so the observation might be that if it's sunny,
then odds are nobody is going to bring an umbrella to the office.
But maybe some people are just being cautious
and they do bring an umbrella to the office anyways.
And if it's raining, with much higher probability,
then people are going to bring umbrellas into the office.
But maybe, if the rain was unexpected, people didn't bring an umbrella,
and so they might have some other probability as well.
So using the observations, you can begin to predict,
with reasonable likelihood, what the underlying state is
even if you don't actually get to observe the underlying state,
if you don't get to see what the hidden state is actually equal to.
This here we'll often call the sensor model.
It's also often called the emission probabilities
because the state, the underlying state, emits some sort of emission
that you then observe.
And so that can be another way of describing that same idea.
And the sensor Markov assumption that we're going to use
is this assumption that the evidence variable, the thing we observe,
the emission that gets produced, depends only on the corresponding state,
meaning I can predict whether or not people will bring umbrellas
or not entirely dependent just on whether it is sunny or rainy today.
Of course, again, this assumption might not hold in practice,
that in practice it might depend--
whether or not people bring umbrellas might
depend not just on today's weather, but also on yesterday's weather
and the day before.
But for simplification purposes, it can be
helpful to apply the sort of assumption just
to allow us to be able to reason about these probabilities a little more
easily.
And if we're able to approximate it, we can still often get a very good answer.
And so what these hidden Markov models end up looking like is a little
something like this, where now, rather than just have one chain of states--
like, sun, sun, rain, rain, rain--
we instead have this upper level, which is the underlying state of the world,
is it sunny or is it rainy, and those are connected by that transition
matrix we described before.
But each of these states produces an emission,
produces an observation that I see, that on this day it was sunny,
and people didn't bring umbrellas, and on this day it was sunny,
but people did bring umbrellas, and on this day it was raining
and people did bring umbrellas, and so on and so forth.
And so each of these underlying states, represented
by x sub t for x sub 1, 0, 1, 2, so on and so forth,
produces some sort of observation or emission,
which is what the E stands for--
E sub 0, E sub 1, E sub 2, so on and so forth.
And so this, too, is a way of trying to represent this idea.
And what you want to think about is that these underlying states
are the true nature of the world, the robot's position as it moves over time,
and that produces some sort of sensor data that might be observed,
or what people are actually saying and using
the emission data of what audio wave forms do you detect in order to process
that data and try and figure it out.
And there are a number of possible tasks that you might want
to do given this kind of information.
And one of the simplest is trying to infer something
about the future or the past or about these sort of hidden states
that might exist.
And so the tasks that you'll often see--
and we're not going to go into the mathematics of these tasks,
but they're all based on this same idea of conditional probabilities
and using the probability distributions we have
to draw these sorts of conclusions.
One task is called filtering, which is, given observations from the start
until now, calculate the distribution for the current state,
meaning given information about from the beginning of time
until now, on which days did people bring an umbrella
or not bring an umbrella, can I calculate
the probability of the current state, that today is it sunny
or is it raining?
Another task that might be possible is prediction,
which is looking towards the future.
Given observations about people bringing umbrellas
from the beginning of when we started counting time until now,
can I figure out the distribution that tomorrow is it sunny or is it raining?
And you can also go backwards, as well, by a smoothing where I can say,
given observations from start until now, calculate the distributions
for some past state.
I know that today people brought umbrellas and tomorrow people brought
umbrellas, and so given two days' worth of data of people bringing umbrellas,
what's the probability that yesterday it was raining?
And that I know that people brought umbrellas today,
that might inform that decision, as well.
It might influence those probabilities.
And there's also a most likely explanation task,
in addition to other tasks that might exist as well,
which is combining some of these given observations
from the start up until now, figuring out the most likely sequence of states,
and this is what we're going to take a look at now, this idea
that if I have all these observations-- umbrella,
no umbrella, umbrella, no umbrella-- can I
calculate the most likely states of sun, rain, sun, rain, and whatnot that
actually represented the true weather that would produce these observations?
And this is quite common when you're trying
to do something like voice recognition, for example, that you have
these emissions of audio wave forms and you would like to calculate,
based on all of the observations that you have,
what is the most likely sequence of actual words
or syllables or sounds that the user actually made when they were speaking
to this particular device, or other tasks that
might come up in that context as well.
And so we can try this out by going ahead and going into the HMM
directory, HMM for Hidden Markov Model.
And here what I've done is I've defined a model where this model first
defines my possible state, sun and rain, along with their emission
probabilities, the observation model or the emission model,
where here, given that I know that it's sunny, the probability that I
see people bring an umbrella is 0.2.
The probability of no umbrella is 0.8.
And likewise, if it's raining, then people
are more likely to bring an umbrella.
Umbrella has a probability of 0.9.
No umbrella has probably of 0.1.
So the actual underlying hidden states, those states are sun and rain.
But the things that I observe, the observations that I can see,
are either umbrella or no umbrella as the things that I observe as a result.
So this, then, I also need to add to it a transition matrix, same as before,
saying that if today is sunny, then tomorrow is more likely to be sunny,
and if today is rainy, then tomorrow is more likely to be raining.
As with before, I give it some starting probabilities,
saying, at first, 50/50 chance for whether it's sunny or rainy,
and then I can create the model based on that information.
Again, the exact syntax of this is not so important
so much as it is the data that I am now encoding into a program, such
that now I can begin to do some inference.
So I can give my program, for example, a list of observations--
umbrella, umbrella, no umbrella, umbrella, umbrella, so on and so forth,
no umbrella, no umbrella.
And I would like to calculate, I would like
to figure out, the most likely explanation for these observations.
What is likely?
Was it rain, rain?
Is this rain or is it more likely that this was actually sunny
and then it switched back to it being rainy?
And that's an interesting question.
We might not be sure because it might just
be that it just so happened on this rainy day people decided not to bring
an umbrella or it could be that it switched from rainy to sunny back
to rainy, which doesn't seem too likely, but it certainly could happen.
And using the data we give to the Hidden Markov Model,
our model can begin to predict these answers, can begin to figure it out.
So we're going to go ahead and just predict these observations.
And then for each of those predictions, go ahead
and print out what the prediction is.
And this library just so happens to have a function called
predict that does this prediction process for me.
So I run Python sequence.py, and the result I get is this.
This is the prediction based on the observations of what
all of those states are likely to be, and it's likely to be rain, then rain.
In this case, it thinks that what most likely
happened is that it was sunny for a day and then went back to being rainy.
But in different situations, if it was rainy for longer, maybe,
or if the probabilities were slightly different,
you might imagine that it's more likely that it was rainy all the way through,
and it just so happened on one rainy day people decided not to bring umbrellas.
And so here, too, Python libraries can begin
to allow for the sort of inference procedure.
And by taking what we know and by putting it in terms of these tasks
that already exist, these general tasks that work with Hidden Markov Models,
then any time we can take an idea and formulate it as a Hidden Markov Model,
formulate it as something that has hidden
states and observed emissions that result from the states.
Then we can take advantage of these algorithms that
are known to exist for trying to do this sort of inference.
So now we've seen a couple of ways that AI can begin to deal with uncertainty.
We've taken a look at probability and how we can use probability
to describe numerically things that are likely or more likely or less
likely to happen than other events or other variables.
And using that information, we can begin to construct
these standard types of models, things like Bayesian networks
and Markov chains and Hidden Markov Models,
that all allow us to be able to describe how particular events relate
to other events or how the values of particular variables
relate to other variables, not for certain, but with some sort
of probability distribution.
And by formulating things in terms of these models that already exist,
we can take advantage of Python libraries
that implement these sort of models already and allow us just
to be able to use them to produce some sort of resulting effect.
So all of this then allows our AI to begin
to deal with these sort of uncertain problems
so that our AI doesn't need to know things for certain
but can infer based on information it doesn't know.
Next time, we'll take a look at additional types of problems
that we can solve by taking advantage of AI-related algorithms
even beyond the world of the types of problems we've already explored.
We'll see you next time.